You are right to focus on the importance of setting in this excellent short story, and in particular the way in which the forest is described indicates the kind of evil actions that will occur there and the evil intentions of Goodman Brown in seeking to venture there in the first place. This is of course compared to the village of Salem. Note how the path that Goodman Brown takes into the forest is described:
He had taken a dreary road, darkened by all the gloomiest trees of the forest, which barely stood aside to let the narrow path creep through, and closed immediately behind. It was all as lonely as could be; and there is this peculiarity in such a solitude, that the traveller knows not who may be concealed by the innumerable trunks and the thick boughs overhead; so that with lonely footsteps he may yet be passing through an unseen multitude.
Note the way that the open streets of the village, where everybody can be seen, and which are well lit, are compared to the "narrow path" of the forest, which is immediately enclosed by the dark trees, which are "gloomy." The road itself is said to be "dreary" and there are so many trees that even though you appear to be plunged into solitude, you could in fact be surronded by an "unseen multitude." The fear and danger in such a position is clear and obvious, and presents us with a massive contrast to the relatively safe village of Salem.