For Rose, the symbol of the fence has a protective meaning. She craves safety and security. These are the things she believed she would get from a relationship with Troy. These are the things she married him for.
For Cory, a character facing one significant limitation, the fence can be seen as a representation of his relationship with his father. No affection passes between father and son. Troy also works to restrict and constrain Cory's actions and even his hopes, fencing his son within defined limits.
We can see Troy's relationship to fences in the same emotional sense:
Unable to open up to those that he loves, Troy keeps much of his emotion inside, building imaginary fences between himself and his family and friends.
Furthermore, as a baseball player, Troy's specialty was hitting home runs - hitting the ball over the fence. This fact has bearing on Troy's character and is mentioned with explicit reference to the idea that Troy was not pleased unless he hit the ball all the way over the fence.
Also, Troy feels that he was kept out of major league baseball because of the color of his skin. He was kept on the wrong side of the metaphorical fence.
Over the course of his life, Troy repeatedly finds himself fenced in by limits - in prison; in his baseball career; in his marriage; in his mortality. We can clearly see Troy's desire to escape or exceed his limitations in his fixation on death.
Death is the fence that limits Troy in a way that is both actual and symbolic. He repeatedly challenges death and recounts stories of his encounters with death.
At the end of his life, after ruining his marriage, losing his one friend and destroying his relationship with Cory, death is the only figure left inside the fence with Troy.
Troy finally succeeds in isolating himself from his wife, his brother, his sons, and his friend.
Having built up barriers between himself and his family, Troy is alone with his opponent. He dies, challenging death, taking a swing with his baseball bat in an effort to hit one more over the fence.