I think all of these symbols that you mention do represent the past. However, I can think of a few more.
First, I think the songs that are sung definitely represent the past. Of course, one of them does talk about St. Clement's Church, but I think the songs themselves are a symbol of the past.
Second, and this is connected to the songs, I would say that the prole woman with the wide hips is also a symbol of the past.
Finally, I would say that Winston's dreams of his mother and family qualify as well.