A critical analysis has the same basic elements as any other sort of essay, needing a thesis that you support throughout the essay in your body paragraphs. Because this is a literary analysis, you will need to have a thesis that is the important point about the story that you want the reader to understand as he or she reads your essay. That point is one you will want to explore, either through just the evidence of the story itself, or with the story and some external sources, too.
Let me give you a few examples of the kinds of thesis you might explore in this story. One theme in the story is the treatment of the wife by the husband, how paternalistic it is. Another is that women who are mentally ill are "punished" for their feelings. Yet another possibility is a contrast with how mental illness was treated then and how it is treated today. Or you could contrast the relationships between men and women then and now, if you think they have changed any. You might also think about how the author uses symbolism in the story. What does the yellow wallpaper represent to the woman? Any of these would make a great thesis for your essay, and you will be able to think of some other ideas, too. (Some teachers want only the story analyzed, with no outside information, but this is something you can clarify with your teacher.)
Once you have a thesis, you must think about how to support it, and incorporate that support into your thesis statement. For example, I might decide to write about the paternalistic treatment of the wife. So, my thesis statement could be this:
The paternalistic treatment of the wife by her husband is shown throughout "The Yellow Wallpaper" in his disrespect for her feelings, his banishing her to an isolated room, and in his taking away all of her responsibilities.
Now, for each of points, I will write a paragraph about what in the story supports each point.
To organize your essay, you will need an introduction that leads the reader into your main idea, an introduction that names the story and its author and that ends with your thesis statement. Then will come your body paragraphs, each discussing one idea that supports your thesis. Finally, you need a conclusion that reminds the reader what your thesis is and that reviews, briefly, for the reader, the points you have made.
Just remember that this is an essay like any other essay in its structure. Analyze the story and stick to the structure.