The fence is given a specific significance in the dialogue of the play when it is referred to as something that keeps things out and also keeps things in. Regarding Troy and his family, the fence carries this meaning and more.
Troy's character, in particular, has a set of symbolic relations to the fence. As a baseball player, Troy's specialty was hitting home runs - hitting the ball over the fence. This fact has bearing on Troy's character and is mentioned with explicit reference to the idea that Troy was no pleased unless he hit the ball all the way over the fence.
Also, Troy feels that he was kept out of major league baseball because of the color of his skin. He was kept on the wrong side of metaphorical fence.
Most significantly, Troy's character relates to fences in an emotional sense.
Unable to open up to those that he loves, Troy keeps much of his emotion inside, building imaginary fences between himself and his family and friends.
For fifteen years of his life, Troy was in prison, kept inside the fences there. Outside of prison, he feels that his life continues to be delineated and limited by various forces and factors and he attempts to exceed these limitations.
We see this in his insistence that his companies policy change regarding who can and who cannot drive the trucks. We see it in his continued bitterness regarding his athletic career. We see this also in his adultery.
Finally, we see Troy's desire to escape or exceed his limitations in his fixation on death and his sense that he has a chance to defeat this natural and inevitable opponent. Death is the fence that limits Troy in a way that is both actual and symbolic. He repeatedly challenges death and recounts stories of his encounters with death.
At the end of his life, after ruining his marriage, losing his one friend and destroying his relationship with Cory, death is the only figure left inside the fence with Troy.
Troy finally succeeds in isolating himself from his wife, his brother, his sons, and his friend.
Having build up barriers between himself and his family, Troy is alone with his opponent. He dies, challenging death, taking a swing with his baseball bat in an effort to hit one more over the fence.