Hamlet was pretty upset with the world; his father had just died, he suspected his uncle of killing him, and his mother had insulted his father's name and honor by marrying his uncle so quickly. So, he is upset at the world, and especially angry with his mother. He can't fathom how she would, in just two month's time, be able to get over his father so quickly, when she loved him so much, and then go and marry someone else. Because she did this, Hamlet starts doubting the fidelity and purity of all women, not just his mom. If his mom could be so fickle and frail, couldn't any woman be that way? He spreads his anger out over all women, not just his mother, and unfortunately, that includes Ophelia.
So, he is hating all women, and Ophelia comes up to him and gives him all of his presents that he had given her back. She tells him that she is returning them, and basically signals that their relationship was over. This confirms Hamlet's bitterness towards women; here was Ophelia, a woman, rejecting him and being wishy-washy. So, in order to vent some of his pent-up rage against women, his mother (who he suspects might be listening to the conversation) and Ophelia's rejection, he unleashes a rather mean tirade against Opehlia. He rants that all women should never marry, because they just bear children who will grow into corrupt men. He says that beauty is a just a charm to draw men in. He says that there are no honest women and that all women should become nuns in order to not create any more confusion or children in the world. All of this is just Hamlet, venting his frustration at his mom and all women in general, and Ophelia happens to be the victim of it.
Shakespeare doesn't really explain Ophelia's reaction to Hamlet's behavior; later, she goes mad, and we can't help but wonder if Hamlet's cruelty played a small part of that. I hope that those thoughts help; good luck!