Since it is satire, we know that the author is not suggesting that people should eat babies. However, the speaker could go either way. You could argue that the speaker is being satirical and not serious, or that the speaker is serious and Swift is satirical.
When you read to the end of the "proposal" you will notice that the speaker suggests other reasonable ways the relationship between the rich and the poor could be made more fair, and of course this is what Swift is actually suggesting should be done in regards to the way the English treat the Irish. He is clearly pro-Irish and critical of the English.
I agree that the speaker's social allegiance lies with the poor of Ireland. The point that pohnpei brings up about the author's feelings about the abuse and oppression which has been inflicted upon the poor would support that his narrator would have the same "feelings."
I would say that it lies with the Irish people and perhaps with poorer people in general. The author clearly thinks that the Irish (and particularly those who are poor) have been abused and that the English and the well-off have been essentially living off of them. That seems to indicate his allegiance is with the Irish.