Although it is likely after the fact, there is no direct indication that the narrator is recalling these events from a point in time that is specifically "years later." However, this is a striking moment in his life, so it is reasonable to assume that this is something he will remember into adulthood.
There is a progression of events in Araby which leads the narrator from a state of mind of youthful idealism to adolescent disillusionment. The narrator is so overcome by his crush on Mangan's sister that all other things in his life are absorbed into the world of that infatuation. He waits in the parlour every morning to catch a glimpse of her like it is a religious ritual. He thought of her while saying his prayers and even thought of her in places and at times which were "hostile to romance." He describes walking through the streets amidst drunks, prostitutes, street-singers, all a chaos of sound, but these ugly sights and sounds become part of his idealized world where his only motivation is a spiritually romantic adoration of Mangan's sister:
These noises converged in a single sensation of life for me: I imagined that I bore my chalice safely through a throng of foes.
In his idealism, he perceives her as something perfect. When he offers to buy her something at the bazaar, he frames the gesture with the spiritual and romantic significance of a quest. Even the name "Araby" with its verbal connection to the East (Arab) becomes like an enchanted part of this idealized world.
This idyllic world is shattered when the narrator realizes the bazaar is not exotic or enchanted; it is just a means to make money. He chastises himself for falling under the spell of his own idealized illusion. The reason the narrator is so affected by the events in this story (and why he might remember them years later) is that he imbued these events in his life with the spiritual romanticism that derived from his adoration of Mangan's sister. Under that spell of adoration, everything became more significant and fantastic.
The events become more striking in retrospect when he has his epiphany that the spiritually romantic spell he put upon himself was an illusion.