On the one hand, Pip is tempted to show him off to Trabb's boy, yet at the same time he worries that Trabb's boy might reveal some of Pip's humble past and the Avenger would ridicule him afterwards. Anxious also that Miss Havisham might even hear such reports, Pip decides to leave his servant behind.  Clearly, that Pip should be concerned about what his servant thinks or that he is worried about revelations of his past indicates his pretentiousness and insecurity, characteristics upon which the servant causes Pip's conscience to avenge itself upon him for his hypocrisy and shallowness. Hence, the name of "the Avenger" for his manservant.

In Chapter XXVII of Great Expectations, Pip reflects,

All other swindlers upon earth are nothing to the self-swindlers, and with such pretences did I cheat myself.

Part of his pretensions include his hiring of a manservant for his rather dilapidated lodgings at Barnard's Inn.  And, instead of being an asset to Pip, the young man becomes yet another financial burden for an already indigent Pip:

For, after I had made this monster (out of the refuse of my washerwoman´s family) and had clothed him with a blue coat, canary waist-coat, white cravat, creamy breaches, and the boots already mentioned, I had to find him a little to do and a great deal to eat, and with both of these horrible requirements he haunted my existence.

The characterization of the Avenger is demonstrative of Dickens's frequent satirizing of the pretensions of those Victorians who aspired to emulate what he considered a frivolous aristocracy.  In his aspirations to be a gentleman, Pip feels it impinging upon him to hire a servant for the sake of appearances.  However, the situation becomes burdensome and inconvenient for Pip.  For instance, when Pip sees Joe Gargery approaching his apartments, he would hide, except that the Avenger greets the man compromising Pip's escape.

Likewise, in Chapter XXVIII as Pip considers traveling to the Blue Boar, he is racked with indecision whether to take the Avenger with him, or not.  Dickens's humorous passage of hyperbole indicates his satire of Pip's superficiality,

It was tempting to think of that expensive Mercenary publicly airing his boots in the archway of the Blue Boar's posting-yard: it was almost solemn to imagine him casually produced in the tailor's shop and confounding the disrespectful senses of Trabb's boy. On the other hand, Trabb's boy might worm himself into his intimacy and tell him things; or, reckless and desperate wretch as I knew he could be, might hoot him in the High-street. My patroness, too, might hear of him, and not approve. On the whole, I resolved to leave the Avenger behind.