The details you need to answer your question are in the second and third stanzas of the poem. Here's the first four stanzas, to give you the full context, of "The Ballad of Father Gilligan," by Yeats. They include the details you ask about:
The old priest Peter Gilligan
Was weary night and day;
For half his flock were in their beds,
Or under green sods lay.
Once, while he nodded on a chair,
At the moth-hour of eve,
Another poor man sent for him,
And he began to grieve.
‘I have no rest, nor joy, nor peace,
For people die and die’;
And after cried he, ‘God forgive!
My body spake, not I!’
He knelt, and leaning on the chair
He prayed and fell asleep;
And the moth-hour went from the fields,
And stars began to peep.
I've emboldened the lines that include what the priest sees as his grievous sin, and italicized his repentence. Actually, his reaction is probably quite natural. Members of his flock are dying faster than he can deliver the Last Rites to them. He is dozing off in his chair when he gets summoned by yet still another dying parishioner. He is overworked and exhausted, but when he reacts to the summons in a very human way, he sees his words and behavior as not fitting for a priest. He grieves and explodes, figuratively, and then is immediatley sorry for what he says right after he says it.
God, in the poem, though, seems to understand. While the priest is praying, begging for forgiveness, he falls asleep and misses the man's dying moments, but God sends an angel in the priest's place to administer the Last Rites.