It is hard to divorce "Young Goodman Brown" from Hawthorne's influence regarding his criticism of Puritan culture. Since Formalist criticism is not interested in biography or historical background, "Young Goodman Brown" deviates from what the Formalist critic would call a strictly literary text. Formalists were interested in the structure and form of the literary text. They were not interested in the author's feelings, the reader's response or any historical, political additions which refer to something outside of the text itself. "Young Goodman Brown" also deviates from a strictly Formalist approach in that it has elements of Romanticism which focus on the character's emotions and this indirectly and directly appeals to the feelings of the reader: 

Had goodman Brown fallen asleep in the forest, and only dreamed a wild dream of a witch-meeting? 
Be it so, if you will. 

However, "Young Goodman Brown" does have many literary qualities that make it ripe for a Formalist analysis. Formalists focus on the artistic structure and form of literature: these elements are what separate literature from other kinds of writing. Symbols, irony, plot, archetypes, and tropes are a few of these literary elements. 

In the story, we don't know if Brown's journey is real or if it is a dream. Whether dream or reality, Brown gets the message. But this ambiguity, or this paradox, is a literary quality which you would not find in other kinds of writing such as journalism. "Young Goodman Brown" is also allegorical, another literary device in which characters and events symbolize concepts (i.e. good and evil). 

Formalism was the attempt to make the study of literature into a science. One could then use formalist analysis to understand any work of literature from any period. Logically following, works from different periods in history could have similar forms. "Young Goodman Brown" has elements of the epic structure found in The Odyssey or Dante's Inferno. A hero goes on a dangerous journey and comes back changed. The hero character and the journey element of epic literature are literary devices, formal techniques of characterization and structure.