The Giver stays behind to help the people of the community deal with the return of the memories.
The purpose of the Receiver of Memory is to collect all of the memories from the history of the community. This way, the people never remember pain, hunger or suffering. They don’t remember love or family either. When Jonas became The Receiver and started taking memories from The Giver, those memories rested with him. If he leaves the community without giving them to someone else, they will be released to the people. The people will experience pain for the first time, and be very confused. This is why they need The Giver, because he can counsel them and help them through the tough times of coming to terms with the memories, and the changes that will occur in the community as a result.