First, the paperweight represents a time before the Party was in power, a time when beauty was something to be valued and spend time thinking about and admiring. Now, beauty is not valued and everything is dingy. People don't think and spend time doing things like looking at a beautiful piece of coral inside glass. Winston marvels at having a thing that could ultimately get him killed. It is not the first illegal item he has purchased, but it is when he buys this item that he contemplates doing much more than simply writing in a journal. It is also something he thinks about using as a murder weapon on Julia before they become lovers.
The nursery rhyme is not as old as Winston would like to think and it rather foreshadows his destruction and death later on, "here comes a chopper to chop off your head." While his head isn't literally chopped off, it is somewhat figuratively chopped off int he sense that Room 101 and his experience inside the Ministry of Love has changed Winston. He no longer committs thought crime on the other side of the experience and he ends up losing in the end because he declares his undying loyalty and love to Big Brother right before he is shot in the back of the head.