The narrator ends the story with feelings of "anguish and anger." He is angry at himself for allowing his feelings for Mangan's sister to dictate his actions. He is also angry at the bazaar because he realizes it is not some exotic place where he'll find some treasure to present to Mangan's sister; the bazaar is just an excuse to make some money.
When the narrator gets to "Araby," most of it has closed and most of it is in darkness. He approaches a cafe where a woman and two men are talking and counting money. The woman is English and speaks to the narrator "out of a sense of duty." This is not indicative of the "magic" or exoticism of the implication the narrator had of the bazaar prior to actually getting there. The word "Araby" ("Arab") suggests a non-Christian, Eastern mystery; something the narrator has not experienced, having grown up in a Catholic household. The magic of "Araby" is created by the narrator as this Eastern, exotic land; his sensual idealizations of Mangan's sister coincide with his exotic notions of the bazaar. Ultimately, this magical quality does not hold up for him because his experience with the business-like English (not exotic) woman leaves him feeling angry at himself and the bazaar.
Some scholars have also suggested that because the English woman was "not encouraging" the narrator equates her reaction with a potential reaction from Mangan's sister. The narrator also feels that he's failed in his quest to get her something from the bazaar. Perhaps more importantly, the stark and bland reality of the bazaar leaves the narrator feeling that his "quest" was not so adventuresome or interesting.