There are two direct references to religion in James Joyce's "Araby." The first reference is to a priest who used to live in the unnamed narrator's house. The second way the unnamed narrator references religion is the way he talks about Mangan's sister, the object of his affection.
In the first two Dubliners stories, "The Sisters" and "An Encounter," references to priests are primarily negative. In this story, however, the reference to the priest is positive. This priest "left all his money to institutions and the furniture of his house to his sister." References to priests in "The Sisters" and "An Encounter" focus on the strict religiousness of these men; in "Araby," the "very charitable" priest reads possibly anti-Catholic and secular books such as The Devout Communicant and Vidocq's The Memoirs.
The second reference the unnamed narrator makes in the story is conflating Mangan's sister with religion. As if being a Christian crusader, he imagines himself bearing his "chalice safely through a throng of foes." In addition, her name "sprang to my lips at moments in strange prayers and praises which I myself did not understand." These ideas could both be seen as blasphemous in Dublin at the time "Araby" was published.