This exclamation has a double meaning. Goodman Brown's wife's name is Faith, and he loves her dearly. She is the picture of innocence as she bids him goodbye, with her pink ribbons blowing in the breeze. Goodman Brown holds on to this image and his love for his wife as he traverses through the woods. By entering the forest, he chooses to leave Faith behind, both literally and symbolically. As he tries desperately to resist temptation, he exclaims, "With Heaven above, and Faith below, I will yet stand firm against the devil!" Then he hears many voices he recognizes from the village, among them his own sweet wife. He then cries, "My Faith is gone!…There is no good on earth; and sin is but a name. Come, devil! for to thee is this world given." When he realizes that even his wife is a cohort of the Devil, he gives up not only on her, but also on his religious faith as well.
It must be remembered that this story is an allegory and is full of symbolism and double or underlying meanings. Young Goodman Brown's wife Faith is actually symbolc of Brown's faith in God. The pink ribbon worn by Faith symbolizes the life line between Goodman Brown and his faith in God. The falling of the pink ribbon symbolism the severing of this life line, thus Goodman Brown declares, "My Faith is gone!"
Hope this helps.