Goodman Brown is surprised by all of the people that he sees in the forest consorting with the Devil, because previously Goodman Brown thought that each of those people were good Puritans. Not only good Puritans, but highly revered, respected Puritans for their general goodness and work within the community in tasks that helped bolster the faith of the rest of the community. Take the following quote about Goodie Cloyse.
As he spoke he pointed his staff at a female figure on the path, in whom Goodman Brown recognized a very pious and exemplary dame, who had taught him his catechism in youth, and was still his moral and spiritual adviser, jointly with the minister and Deacon Gookin.
Goodman Brown recalls that Goodie Cloyse is very "pious." But further details are given about her religious dedication and spiritual health. The woman doesn't only go to church on Sundays. She teaches catechism to the young people of the community, she serves as a spiritual adviser, and she works closely with the minister. The minister!
Every person that Goodman Brown comes across in the forest is somebody that he had previously thought was a wonderful person of God and surely couldn't be tempted and tainted by the Devil. Goodman Brown most adamantly believes that even if everybody in the town is with the Devil, his wife, Faith, must absolutely still be innocent. Unfortunately for Goodman Brown, his faith in Faith is destroyed.
They did so; and, by the blaze of the hell-kindled torches, the wretched man beheld his Faith, and the wife her husband, trembling before that unhallowed altar.
After his horrific night, Goodman Brown is forever changed. He doesn't trust anybody anymore. He is suspicious of every person in the town, and he can no longer fully love his wife. Those feelings stay with him the rest of his life, and he lives as a lonely, jaded, and spiritually dead man.
Often, waking suddenly at midnight, he shrank from the bosom of Faith; and at morning or eventide, when the family knelt down at prayer, he scowled and muttered to himself, and gazed sternly at his wife, and turned away.