Evidently the relationship between the speaker of the poem and his father was somewhat troubled when the speaker was a child. The speaker tells us that "No one ever thanked" his father, and even how he would speak "indifferently" to his father, despite all of the sacrifices his father makes for his comfort.
It seems as though the father is not the type of person to express his love with hugs and emotional warmth; the father expresses his love with actions, like polishing the speaker's shoes for him or lighting the fires so that his family does not have to suffer in the cold as he does. The speaker says that, once the house was warm, he would rise from his bed to get dressed, though he would fear "the chronic angers of the house." I interpret this to mean that he fears his father's anger; perhaps the two of them disagree and argue quite a lot, as we know that the speaker behaves ungratefully toward the father. Perhaps there is lots of arguing in the house in general, among all the members of the family, and so the speaker fears the time when everyone comes together. He admits, in the end, that he did not understand his father's actions to be expressions of love. Understood this way, "house" seems to stand in for the people who live in the house, and this is an example of metonymy, when something associated with a thing is substituted for the thing itself.