Why is Winston interested in antiques? Why does winston want to piece the fragment of the "Rhyme St. Clements" together?
In my opinion, the point here is that Winston wants to have something that he can actually call his own. He wants to have a life of some sort that is his private life rather than something that is allowed to him by the government.
In this society, the government has taken control of pretty much all of a person's life. There is nothing that can make a person an individual. This is one of the major things that bothers Winston. He wants to be an individual with individual feelings and a personal life. Because of this, he wants to have things like antique objects and he wants to remember the rhyme -- both of these will make him more of his own person.
Perhaps because it is his job to destroy and re-write the past, WInston is desperate to find any link to what the world was like before Big Brother and the Party and everything else that is so incredibly dominant.
Those links might be in the form of physical things like antiques or even just the room above the shop where he feels some connection to this other or previous world, or they might be mental things like remembering what his parents were like or the particular rhyme because it opens up possibilities or thoughts outside of Big Brother's control.