The first sentence of Nathaniel Hawthorne's The Scarlet Letter says just about everything that must be said about the Puritan lifestyle, at least according to Hawthorne.
A throng of bearded men, in sad-colored garments and gray, steeple-crowned hats, intermixed with women, some wearing hoods, and others bareheaded, was assembled in front of a wooden edifice, the door of which was heavily timbered with oak, and studded with iron spikes.
A throng of people, probably a good portion of the community, was gathered in their "sad-colored" clothing in front of a door which we find out in the next sentence belongs to a prison. These somber Puritans have gathered not for church but to see the spectacle which will emerge from the prison door.
That spectacle is the adulteress, Hester Prynne, and her newborn daughter, and the people are here to ensure that she receives a suitably harsh punishment: standing on the scaffold in her shame, bearing the insults directed at her, wearing a giant "A" on her chest, and listening to haranguing sermons about her sin.
Some of the ladies in town think this is not enough:
"The magistrates are God-fearing gentlemen, but merciful overmuch,—that is a truth," added a third autumnal matron. "At the very least, they should have put the brand of a hot iron on Hester Prynne's forehead. Madame Hester would have winced at that, I warrant me. But she,—the naughty baggage,—little will she care what they put upon the bodice of her gown! Why, look you, she may cover it with a brooch, or such like heathenish adornment, and so walk the streets as brave as ever!"
The Puritan lifestyle is one of guilt and punishment, hidden sin and hypocrisy. Though of course everyone in town commits sin, those whose sins become public are shamed, ostracized, or killed. Church is where the Puritans go to have those beliefs reinforced, and the forest is where they go to commit their secret sins. While the sins of adultery and witchcraft are called out, put on display, and punished, hypocrisy, gossip, and treating one's neighbor as less than oneself are perfectly acceptable in this Puritan community.
Just as their thoughts center around guilt, shame, and sin (either in others or in themselves) so do their lives. They do not pursue pleasure except illicitly and they do not tolerate those who fail to conform to the strict moral code of their community, as demonstrated by the throng waiting for Hester outside the prison door.