The author comes to blame himself for his homosexuality. It's not surprising that he should feel this way; he's simply internalized all the negative, hateful messages concerning homosexuality that he's received from his father's church throughout the whole of his short life. From an early age, he's been taught that there's something seriously wrong with being gay. In response, the author has repeatedly tried to banish his homosexual feelings, but without success.
This has made him feel like a complete moral failure, as if it's his fault that he still harbors what to his father are "unnatural" passions. As such, Conley goes through the whole of his early life experiencing what he describes as a "constant guilty ache" that runs through his whole body, which he comes to accept as second nature.