Shakespeare is known for inserting a play within a play, and he certainly puts the form to good use in Hamlet. I assume the speech to which you refer is in Act III scene ii, after Hamlet has berated himself for failing to conjure up the passion to follow through with his plan to avenge his father. Here he gives the players some advice, reminding them not to overact. Instead, they are to see their job as holding "mirror up to nature"and presenting a realistic and believable story.
This speech is important for several reasons. First, Hamlet demonstrates his acting experience. One of the central questions of the play is whether or not Hamlet is mad or "mad in craft." After he meets the Ghost, he tells his friends he may be acting mad; and in this speech we learn his view of and expertise in acting. Second, Hamlet is eager for the play to help him "catch the conscience" of Claudius. If the story is told in a less than realistic manner, he cannot be sure of a true response from the King. Hamlet speaks, the Players listen, and the King reacts.