After he meets the ghost, Hamlet tells Horatio he will be pretending madness, so we as an audience know what he is doing. However, the members of the Danish court, including Claudius, do not understand what is going on. When the audience knows what characters in a play do not, this is called dramatic irony.
Hamlet's feigned madness works to his own advantage, as it keeps the courtiers Claudius has assigned to spy on him guessing about what is wrong with him. He does not want Claudius to guess that he suspects that he killed King Hamlet. In fact, Hamlet creates enough confusion with his erratic, "mad," behavior that Claudius is relieved when Hamlet wants to put on a play. Claudius encourages this activity (which he should be very worried about) because Hamlet's feigned madness has successfully deceived him.
Ophelia, however, is confused and distressed by Hamlet's behavior. She doesn't know that Hamlet is only pretending to be mad and doesn't know how to interpret his behavior toward her. What we learn about her real madness is that true insanity can have dire consequences: Ophelia, unlike Hamlet, is driven to suicide by the madness around her.
Feigned madness is a useful ploy in this play; real madness is tragic.