One of the major themes of this poem is that the superior person, anyone who is fit to be called a "man," is someone who will do their best at all times. They will work hard and never give up, even when they are surrounded by lesser people who do not appreciate their work and who may even destroy it. This is the meaning of the lines you cite.
Kipling is saying that you may do and say great things and other people may distort or destroy them. They might do these things either because they are fools or because they are bad people. Either way, the superior "man" just takes that in stride. The superior man sees his work or his words destroyed and just gets back to work.
So, these lines are saying that you never give up and you never allow other people to affect what you do. If you know you are doing right, you just keep doing it, even if other people are destroying your work and even if you are worn out and tired from your efforts.