There is an ambiguity within "Young Goodman Brown" as to whether his vision in the forest actually happened to him, or whether it was a dream. The story does not express a clear answer to this question either way.
As far as this question is concerned, however, it might be useful to consider the story's resolution, rather than the vision in the forest (because there, that ambiguity between imagination and reality does not exist, and one can more clearly discern the ways through which the one influences the other).
Goodman Brown's vision in the forest has destroyed his capacity to trust in the goodness of others. From that point on, he can only see wickedness and sin around him. This we see in his return to Salem and his treatment of the people there. When the minister gives him a blessing, he recoils from it. Meanwhile, Hawthorne writes of his encounter with Goody Cloyse:
Goody Cloyse, that excellent old Christian, stood in the early sunshine at her own lattice, catechizing a little girl who had brought her a pint of morning's milk. Goodman Brown snatched away the child as from the grasp of the fiend himself.
Finally, there is his reunion with his wife, Faith, who meets him joyfully. Hawthorne writes of his response:
Goodman Brown looked sternly and sadly into her face, and passed on without a greeting.
From the beginning of the story, Goodman Brown's expectations of the people around him have shaped his perception of the world. This is a factor which would continue to hold true at its end. After his vision in the forest, Brown has become a cynic, and those cynical expectations will proceed to shape his perception of reality. The result is to cast the remainder of his life in misery.