The words "a tattered coat upon a stick" suggest a scarecrow. The coat is worn out, and it will become more tattered as it hangs in a field in all kinds of weather. The coat is hanging on a stick because it is tattered, and it is tattered because it is flapping on a stick in the wind. If a man is old enough and feeble enough his appearance will actually be rather frightening, because he is the living image of what everyone who sees him is destined to become if he or she manages to live as long. He is of no use to society anymore. He is just taking up space. It really doesn't matter what country he is leaving. Any country is no country for old men. The only interest others can have in him is wondering when he will die. Old people are often poor and wear clothing that is worn out and even patched. Or else they have become so forgetful that they are unaware of what they look like. Their clothes are usually long out of fashion, but they either don't realize this or else don't care. Hanging on a stick also suggests the emaciated look that some old people acquire from the illnesses and physical wasting that accompany old age.
When Yeats writes, "...unless / Soul clap its hands and sing, and louder sing / For every tatter in its mortal dress," he is suggesting that the soul has no accompaniment for its singing but the clapping of its own hands. This is a sign of the poverty that often accompanies old age.
By "sailing to Byzantium," Yeats means escaping from reality into the world of art. In his case this would mean escaping into the creation of lyric poetry. Many poets have used their work to escape their spiritual suffering. John Keats is a good example. He was haunted by the fear of death, and he often uses his painful emotions as the subject matter for his poems. Yeats would like to forget about himself completely and to become metamorphized into a work of art. This is made more apparent when he writes:
O sages standing in God's holy fire
As in the gold mosaic of a wall...
He is saying the exact opposite of what most people would see in Byzantine mosaics. They would see the gold mosaics representing the sages standing in a holy fire, whereas Yeats sees them as real, living saints standing in a real holy fire which just happens resembles a gold mosaic. This is the transformation Yeats would like to achieve for himself through his imagination and his art. The only escape from the pains and fears of old age is into the world of art.
Once out of nature I shall never take
My bodily form from any natural thing...
He actually believes he can escape from nature and from death by transforming himself into a beautiful work of art. In a sense, he has succeeded, because he died long ago but has left the best part of himself behind in his transcendent poetry.