Silas Marner moves to Raveloe in shame after being falsely accused of stealing from his church. When Silas moves to Raveloe, he leaves behind a full life in Lantern Yard and a community who once viewed Silas as a young man with a lot of potential. After William Dane frames Silas for theft, Silas loses the respect of his community as well as his own innocent faith in humanity and the woman he loves; at this point, he has no choice but to leave Lantern Yard for a new life in Raveloe.

Silas's life in Raveloe is isolated, as he chooses to concentrate on his work and the earning of gold. He lives away from the community and away from any kind of lively interaction with other people. These choices make Silas a lonely man, but they reflect his need to protect himself from being hurt again.

The main reason Silas moved to Raveloe has to do with his "friend" Wiliam Dane.  In the town where he lived before, he and Dane were religious buddies and they attended the same church.  Dane made it seem that they were great friends, but at one point he decided that he had had enough of pretending.

So he betrayed Silas on every level possible, he had Silas blamed for stealing money from the church (something that Dane arranged) and because of the social fall out over this, Silas lost his fiancee and Dane ended up swooping in to rescue her and married her himself.

Because of this betrayal, Silas was far too angry and bitter to live in the same town so he moved to Raveloe to escape the memories and the social stigma he faces.