Guilt does have a role to play throughout many different parts of the book. After Pip first helps the convict Magwich by bringing him food and supplies, the next weeks are spent wracked with guilt; he is sure that someone will find out, and that Joe especially will find fault in him for helping a prisoner. Guilt makes Pip jump at small noises, quail in fear when news comes, and it even interrupts his sleep. Eventually, his guilt over helping the escaped convict does fade to the background, but for quite some time, Pip was ridden with a guilty conscience over the entire thing.
Guilt comes into play again after he has received his fortune and lived in London for a while. He realizes that he has been treating Joe and Biddy horribly. He never talks to them, hardly ever writes, and when Joe visits, Pip is awfully embarrassed by Joe's simple ways and embarrassing familiarity. Pip doesn't treat them very well, and he knows this--it is a constant force, eating away at the back of his mind. His guilt keeps him from fully enjoying the benefits of his new life, because he feels like he is a bad person, even if he is now a "gentleman." When Pip does finally return home, and gets sick, he is so driven by guilt that in his feverish ramblings he focuses quite a bit on it. After that, he repents his bad behavior and learns to treasure his family for the prize that they are.
One last rather unexpected area that guilt comes in is in relationship to Magwich's capture and death. When Magwich arrives and reveals himself as Pip's benefactor, Pip is, at first, horrified, embarrassed and shocked. Eventually though, he comes around, feels bad for Magwich, and as the old man is captured, Pip feels guilty for having treated him so horribly and having thought of him in such negative terms. He stays by Magwich's side until his death; guilt and fondness drive him to be a good friend to him until the end.
Those are a few ways that guilt has a hand in the plot and character development of the novel; I hope that helped. Good luck!