An effective to a story may comprise different sorts of specifics for different sorts of stories. The main requirement for an ending to be "effective" is for it to be consistent with the story as it has been crafted from the beginning. To judge effectiveness, consider things like the following: Does the structure of the story support a given ending? Do the characters act in keeping with their traits and/or development? In light of this definition, I think Oates' ending is effective.
I agree with #2 in the way that the ending the author has created clearly suggests that Anna is due for another meaningless relationship at some point in the future due to her inability to the way that she depends on others to fulfill her. However, I don't necessarily think that this makes the ending "ineffective." If you are the kind of person that expects a happy ending to your stories, then you will probably consider this ending to be ineffective, but I think this story points towards the way in which it is dangerous to look for completion in another person and how secure and mature relationships often occur between two people who are complete in themselves.
Joyce Carol Oates' writing is described as:
…fiction [that] explores the conflict inherent in the attempt to define oneself in a turbulent society.
In her short story, "The Lady with the Pet Dog," I believe the end of the story is effective only with regard to the person Anna is. Anna finally decides that she will look at her adulterous affair as a kind of "true marriage." From Anna's standpoint, this is an acceptable way to direct her life and her future. However, the real question is whether or not she is simply kidding herself by believing that her lover is someone who only exists in her imagination. She is struggling to find herself in a world as she wants it to be, not in the world in which she actually lives—with real people.
In the story we learn that Anna is not happy with marriage or her extra-marital affair, to the point that she contemplates suicide. One disconcerting element is that Anna goes from seeing no future with her lover, to being a woman who now sees a complete and happy future with the same man. How quickly she is able to convince herself that they can make the relationship work.
In addition, Anna, unfortunately, does not look within herself for satisfaction, but relies on a man for this. She can never be happy if she depends upon another person because his actions ("perfect" and "imperfect") will end up defining her life. At some point we can expect that she will resent the man—though it is really not his fault but her own. She cannot find completion with another person until she can see herself as a complete person when she is alone. She has decided she will have the very thing she told herself could never be hers.
Like an addict, she looks for salvation in the very things that hurt her.
The character of Anna will see her choice (which surprises and "stuns" her lover) as an answer to all her problems. However, their relationship is built upon the shaky foundation of two people who communicate on a physical level, but who do not really know each other—there is no substance to their relationship. Anna may be pleased, but I believe the reader may see that she is heading for disaster on the path she has chosen, and therefore, for the reader, the story's ending is not effective.