We know from the early lines of the introduction to the Pardoner in "The General Prologue" that the narrating Pilgrim feels unsympathetic toward the Pardoner since he is introduced as the friend of the disliked Summoner. The Pilgrim calls the Pardoner "gentle." This satirically ironic descriptive word has an archaic definition meaning gallant and chivalrous. We know the Pilgrim is speaking satirically, ironically and insincerely because we know who the Pardoner's friends are, "With [the Summoner] rode a gentle Pardoner"; the Pilgrim feels dislike and disdain for the Pardoner. 

As the Pilgrim's description continues, we learn more about how he feels through further indirect inferences. First there is the description of the Pardoner's unorthodoxly long yellow hair that spreads over his shoulders in thin clumps. For an unspecified reason, he won't wear the monk's hood that is a standard part of a cleric's garb (clothing was prescribed by code in Chaucer's day so that a person's occupation and social station in life was apparent by the clothes they wore.) Yet he does wear the "cap," or skull cap, that accompanies a cleric's clothing. Some critics suggest this refusal to wear the hood is a sign of vanity and fashionableness.

Another explanation is needed because his hair can certainly not be considered fashionable or a pique of vanity with its "ounces" (strands) in "colpons" (clumps):

it hung as does a strike of flax.
By ounces hung his locks that he had,
And therewith he his shoulders overspread.
But thin it lay, by colpons, one by one,

It makes more sense to analyze this feature of his behavior as another sign of the insincerity of his calling: he won't wear the hood because he is a cleric in appearance only, not a true cleric who sets himself apart for holy purposes. This analysis tells us more about how the Pilgrim feels about the Pardoner: he disdains the Pardoner's hypocrisy and his manipulation of his ordination for selfish gain.

The most perplexing part is the description of the Pardoner's physical characteristics. The final line asks whether he is a "gelding or a mare." A gelding is a castrated male horse or a human male eunuch. If the Pardoner is a eunuch, that explains both his friendship with the Summoner, no male competition, and his great singing ability, as his voice would have remained in the soprano range. A mare, of course, is an adult female horse. A realistic analysis of this is that the Pilgrim's speculation that the Pardoner is a eunuch is accurate and the gibe about being a mare is another instance of satirical irony. This reveals more about how the Pilgrim feels: he is basically repulsed by the Pardoner, seeing him as a non-definable and corrupt entity.

A voice he had as small as hath a goat.
No beard had he /.../
I trow he were a gelding or a mare.

A final note is that the description is one of the three prescribed ancient Greek methods. The method he uses to describe the Pardoner gives only the most enlightening physical details, then describes the Pardoner's occupation, ending with a glimpse at his motivation: "To winne silver ...." Chaucer chose this technique of description so as to focus on the Pardoner's immorality and his inner and outer corruption. This method of description, ascribed as it is to the Pilgrim, tells us that the Pilgrim's feelings for the Pardoner culminate in actual condemnation of him for actions like preaching well only "To winne silver as he full well could."