I think the main difference between the two is "where" meaning happens in a literary work.

In psychoanalytic criticism, Freud's notion of the unconscious is the ultimate origin of literary meaning. That is, psychoanalytic interpretation seeks to discover meanings within the text placed their by the author's subconscious. This does not mean that art is accidental in any way or outside the artist's control, but it does suggest that the imagery and themes that make the essence of a literary work may include or embody meanings other than those expressly intended by the author. Unearthing these unconscious meanings is the work of the psychoanalytic critic.

Reader response criticism emerges from a different philosophical background, that of phenomenology. In this case, the critic focuses on the experience of reading, and sees literary meaning as something produced by the interplay of different contexts (or "horizons" of experience) as the reader encounters the text. For instance, there is the horizon of the text itself, and the experience one has with being engrossed in a story; there is also the horizon of the reader's life, in which the act of reading the text is a part; included here would be ways in which one's life experience alters how one perceives the text. A third horizon would be the reader's experience of other texts, and the ways in which the reading of other works influences the current reading. A fourth could be the history of other readings of the text (the history of how Hamlet has been understood, for example). By trying to address the complex mental process of reading in an intentional way, reader response critics hope to explain how literary meaning is produced, and the nature of its multiplicity and indeterminacy.