Jared Diamond takes a neutral view of religion for his work, explaining the effects rather than passing judgement. When speaking of the "kleptocracy," the ruling class that benefits from the lower-class's hard work, Diamond mentions that religion is often used as a pretext to invade other regions, to forcibly tax wealth and products from workers, and to quell rebellion under the guise of a moral necessity. However, he also points out that religion works to unite people, giving people a common culture instead of being a disparate group of disconnected individuals:
...religion helps solve the problem of how unrelated individuals are to live together without killing each other -- by providing them with a bond not based on kinship. Second, it gives people a motive, other than genetic self-interest, for sacrificing their lives on behalf of others.
(Diamond, Guns, Germs, and Steel, Google Books)
This allows for the formation of larger societies instead of isolated farmsteads, and gives people the mental ability to both quantify and justify their sacrifices for "the greater good." While the kleptocracy often uses religion for its own benefit, not caring about the effects on the "average Joe" mentioned in the question, religion has helped the formation and organization of working societies instead of fiefdoms.