Perhaps with the Monk. Chaucer does not name any other character in his description of the doctor, but he does say the doctor is in league with the pharmacists to cheat customers of their money, that the doctor is greedy in his special love of gold, that he doesn't read the Bible very much, and that he dresses very fancily in blue and red garments lined in taffeta.
The monk also dresses very fancifully (more than a monk ought to) and he is very into worldly things, one of which is his gold pin fashioned into a lover's knot. He loves to hunt and has the best horses in order to do this well. He blatantly disregards the old ways of living that monks usually follow (he makes up his own rules). One may conclude from this that he is greedy and doesn't have much time for reading the Bible.
Chaucer picks on the hypocrisy of clergy who do not live purely.