What does Hamlet mean when he said to Ophelia "go thee to a nunnery"?
What I want to know is if Hamlet says that because he does not love Ophelia, or instead because he loves her so much and he does not want to hurt her..
This is also a notorious example of how Shakespeare played to all levels of his audience -- in this case, the pit, always appreciative of a double entendre. It is fairly clear that Ophelia has slept with Hamlet, as her distracted words in Act IV Scene 5 make clear:
By Gis and by Saint Charity,
Alack, and fie for shame!
Young men will do't if they come to't
By Cock, they are to blame.
Quoth she, 'Before you tumbled me,
You promis'd me to wed.'
'So would I 'a' done, by yonder sun,
An thou hadst not come to my bed.'
She has come to believe that the reason Hamlet has not married her is because she yielded to him too soon. Given this, it is hard to believe that the other meaning of "nunnery" in Elizabethan times is not in play here -- "whorehouse." (There is another double entendre in "by Cock" -- an euphemism for "by God," but the first attested use of "cock" for penis is in Shakespeare's time, 1618 to be precise.)
Why would Hamlet be so crude? We must consider the context. The scene in Act III Scene 1 which he says "get thee to a nunnery" comes immediately after his famous "to be or not to be" soliloquy, in which he questions his very existence. When he sees Ophelia at the conclusion, he is overjoyed:
...Thus conscience does make cowards of us all,
And thus the native hue of resolution
Is sicklied o'er with the pale cast of thought,
And enterprises of great pitch and moment
With this regard their currents turn awry
And lose the name of action. Soft you now!
The fair Ophelia! Nymph, in thy orisons
Be all my sins remembered.
And how does she respond? She dumps him on the spot:
My lord, I have remembrances of yours
That I have longed long to redeliver.
I pray you, now receive them.
Little wonder that his reaction is violently negative -- he denies that he had ever given her any love-tokens ("rememberances"), and launches into a slashing attack on Ophelia's honesty, intelligence, and virtue, the final result of which will be to drive her insane.
Hamlet admits in this little conversation that he "loved Ophelia once". He quickly follows with "I loved you not." I don't think this means he never loved her, or that he loved her once and no longer does...he is acting crazy for the benefit of those listening and still trying to warn Ophelia, for whom I think he still has feelings, against the evil that men harbor within themselves. He tells Ophelia that men are sinners and that if she marries, she, like his mother, will breed sinners into the world. He goes on to tell her that all men are "arrant knaves" and that she would be better off if she never heard or believed any of them (where best to do that than in a nunnery, safe from the company of men?) He tells her also that if she decides not to go to a nunnery, that she should marry a fool--wise men make monsters of women (I'm sure he is referring to his mother who has perhaps helped Claudius become King by getting rid of Hamlet's father).
He tells her several times to go to the nunnery. My opinion is that he is trying to protect her from a life that might end up similar to his mother's. That indicates that he cares for her a great deal, otherwise, why would he care what happens to Ophelia?