Troy says that he sees Alberta as a release from all his troubles, a way of escape. However, Troy has a history of chasing women, although he has, at least before Alberta, always been faithful to Rose. When he says that his affair with Alberta is "an escape," it's possible that the escape he means is into his past and that his motivations are connected to a desire to return to his younger days.
This is understandable, given his conflict with his sons. Cory, for instance, has a bright future ahead of him, because he has been awarded a football scholarship. But Troy, who once had an ambition to play baseball professionally, thinks that Cory is wasting his time and undermines his efforts, getting him dismissed from the football team.
It's clear that Troy is jealous of his son's success. His affair with Alberta can be seen as a way of asserting his independence and manhood, separate from Rose and his family. Troy's rhetoric about how he suffered as a boy and how his own children should suffer in the same way to build character is a kind of self-mythologizing that again makes Troy the measure of success for his children. The fact that his children are different from him and in some ways have surpassed him could be a motivating factor for his affair.