I think it is a little bit of both. Clearly Mr. Boone has pent up anger over his wife leaving; he killed a dog because of it! He is angry about the affair, as anyone would be. Punishing his wife by keeping her letters is probably a subconscious jab at his wife.
However, Christopher's father is dedicated to his son. As Christopher's mother admits, Ed Boone is much more patient with Christopher's outbursts and behavioral issues. He understands Christopher's needs and has adapted to them much more readily and willingly than his mother. For example, knowing his son does not like to be touched, he developed the special finger touch to show emotion. Mr. Boone realizes that his wife and her new boyfriend would not be as accomodating as he is. Christopher needs his same space and his same routines, and his mother cannot provide that to him.