How does Gregor's transformation symbolize problems in his family and how the demands placed on him have worn him down?
Prior to his transformation, Gregor was faced with the prospect of paying off his family's debts by working at a job he hates, all the while attempting to win acceptance from his bully of a father. Gregor seems to have a compassionate sister, Grete, but even she eventually turns on him. Gregor is a self-sacrificing character; after his transformation, his family can no longer depend on him and particularly his father now feels completely justified in berating his son as a worthless insect; literally and figuratively.
After Gregor's transformation the familial tensions grow exponentially. Being an insect, trapped in his room, Gregor becomes a burden rather than a breadwinner. But, in his isolation and freedom of this role of the sacrificer, Gregor is able to escape. Only through isolating himself from the world (in his room and then in death) is Gregor able to escape his role in his family.
To be more direct to your question, I don't think Gregor's transformation symbolizes the problems in his family. The radical nature of the transformation does show how ingrained the family's function is. They are more concerned with the mundane daily concerns than with the fact that their son is a giant insect. Gregor has been so worn down that a transformation of such significance is the only way to extricate him from this situation.
There are so many interpretations of this story. One perspective regarding the transformation, in this context, is that even the most radical change might not affect the habitual, repetitive function of a somewhat robotic society. And sadly, that isolation from that world may be the only way to escape that routine. Kafka was known for this kind existential look where the individual is on an endless quest of meaning, acceptance or entrance into a more open world. Check out his short story "Before the Law" and his novel, The Castle.