We aren't actually given a physical description of these two characters apart from the obvious detail that they are both white. Mr. Brown is said to wear glasses, but apart from that the main description we receive from them regards how they tried to establish Christianity in the tribe and the different approaches that they took. The best comparison between them comes in Chapters Twenty One and Twenty Two, which clearly detail their approach to the indigenous religion. Note what we are told about Mr. Brown:
This growing feeling was due to Mr. Brown, the white missinoary, who was very firm in restraining his flock from provoking the wrath of the clan... Mr. Brown preached against... excess of zeal. Every thing was possible, he told his energetic flock, but everything was not expedient. And so Mr. Brown came to be respected even by the clan, because he trod softly on its faith.
The way that Mr. Brown was respected even by the clan is demonstrated through the many friendships that he formed and the respect with which he was held. However, Mr. Brown's replacement, Mr. Smith was very different:
He condemned openly Mr. Brown's policy of compromise and accommodation. He saw things as black and white. And black was evil. He saw the world as a battlefield in which the children of light were locked in mortal conflict with the sons of darkness. He spoke in his sermons about sheep and goats and about wheat and tares. He believed in slaying the prophets of Baal.
Thus we can see that the massive difference between Mr. Brown and Mr. Smith is that Mr. Smith was far more antagonistic and harsh towards the indigenous religion, whereas Mr. Brown adopted a more softly-softly approach.