Who does Iago tell that Othello badmouthed him to Brabantio?
I think you're referring to the opening of Act 1 scene 2, where Iago tells Othello:
'I lack iniquity
Sometimes to do me service: nine or ten times
I had thought to have yerk'd him here under the ribs.' (my italics)
Iago is feigning indignation at someone talking insultingly of Othello, and claiming that he could hardly prevent himself from attacking that person, to defend Othello's name.
The identity of the 'him' is a little ambiguous. It could be Brabantio, as Iago is making a show of warning Othello about Brabantio's anger at Othello and Desdemona's elopement and marriage.
The other possibility is Roderigo. As a known suitor of Desdemona, he has reason to be resentful of Othello, and spoke openly with Brabantio in revealing Desdemona's flight in the previous scene. Roderigo openly insults Othello to Brabantio ('gross clasps of a lascivious Moor'), whereas Brabantio's anger, at least at that point, was directed more against his daughter ('O she deceives me/ Past thought!').
It makes little difference who is actually meant. The significance of the exchange lies in what it reveals about Iago and Othello. The fact that it was Iago who incited Roderigo and led the coarse allegations against Othello and Desdemona, though hidden from view so he could not be recognised, makes Iago's show of indignation and loyalty here one of the earliest signs of his double-dealing and villainy. It is also the first time the audience sees Othello. His calm response to Iago's speech ('Tis better as it is'), forces the audience to re-assess the unflattering impressions they have been given by what has been said about him so far in the play.