I would say that the antagonist in this story is the idea of growing up, or coming of age. The passage of time, things that change, all act as the force which takes the role of antagonist.
The story is based on a memory of the author who is looking back, with nostaglia, at his childhood days and feeling warmed by them.
"Told in flashback, the narrator relates the bittersweet nature of coming of age. Once removed from his best friend and sent to military school, he states that ''Home is where my friend is, and there I never go." He recognizes the symbolic innocence of his younger days when he "[expects] to see, rather like hearts, a lost pair of kites hurrying toward heaven."