In addition, Winston and Julia feel safe in this room. They often comment that no harm can come to them here. There are no telescreens, no posters of Big Brother, no hidden mircrophones, no rules to obey. They are in a pocket of safety, a pocket of emotional stability and physical connection that is forbidden by the Party. They are meeting in the room above Mr. Charrington's shop in the Proletarian region which is against the law...their relationship and the risks they are taking are their own rebellion against the Party. This is why Winston is so enamored by things of the past--the paperweight, the rhyme about the church bells, the makeup and perfume the Prole women wear, etc.
Symbolically, Winston and Julia's relationship is like the paperweight. It is beautiful, forbidden, fragile, and the coral is protected in the depths of the clear and smooth glass. But, like the paperweight which is smashed by the Party police when they sweep suddenly into the room where Winston and Julia meet, their relationship has been visible all along, and smashed right along with the paperweight itself. In the end, the paperweight is extinct, and so are Winston and Julia...just like the extinct animals in the quote.