What bothers Winston is openly stated in Chapter 7:
What most afflicted him with the sense of nightmare was that he had never clearly understood why the huge imposture was undertaken. The immediate advantages of falsifying the past were obvious, but the ultimate motive was mysterious.
So Winston is not so bothered by the fact that the erasure of history is happening, but that he does not understand why it must be so. This remains the central question of the novel.
This quote occurs in the context of Winston remembering the survivors of the great purges in which the original leaders of the Revolution were exterminated. The three men were arrested, confessed to treason, and were eventually pardoned by the Party and given sinecures(cushy jobs). Later all three were rearrested and executed as a warning to others. But five years later Winston came upon a photo proving that their confessions had been lies. This would have been extremely damaging to the Party. Winston destroyed the photo by throwing it down the memory hole. The effect of this was to destroy the past, of course.
All of this is written in Winston's journal. Underlying all of Winston’s writing is the hope that the mind is the one thing the Party cannot control; yet he is aware that the Party is powerful enough to declare 2 + 2 = 5 and that the statement will be believed. The fact that the mind is a shaper of reality is a powerful message in this novel. If all minds accept the new version fo the past, it will become truth.
I'm not sure there is a single answer to this. I would suggest that Winston sees a part of him, the greedy, selfish youth who took his sister's candy for himself and ran away. Besides being a selfish act, it is the last time he saw his mother and sister, although he does remember his mother protecting his sister, perhaps against him. Selfishness is probably part of youth; as you get older, there are often regrets about our selfish acts, particuarly, I would imagine, if we acted that way in our last contact with someone we loved.
Perhaps, and here's where I am "guessing," Winston has a foretaste of the selfishness that is going to lead him to denry Julia in the end. Toward the end of the chapter, Winston and Julia admit to each other that the party will break them, but that the party will never be able to make them stop loving each other. Could Winston already have understood that self-interest could make one do almost anything? I doubt that this happens on the conscious level, but it may be there bubbling under the surface.
And in the end, he is right.