The first reaction is shock. From the very start of the novel, we are told that Silas never ventured into the Rainbow despite it being the town's public house.
..he never strolled into the village to drink a pint at the Rainbow or to gossip at the wheelwright's; he sought no man or woman, save for the purposes of his calling, or in order to supply himself with necessaries..
Therefore the fact that Silas, of all people, shows up in a place where he has not set foot in the 15 years that he has been at Raveloe, is a big deal. The townspeople literally feel that they are staring at a ghost, and not at Silas. That is the degree of shock that they actually feel.
This all happens in chapter 7, after Silas discovers the theft of his gold. In complete desperation Silas steps out of his cottage and ends up in the Rainbow, looking for Squire Cass and other people, to look into the theft. Being that Silas is, literally, about to have a breakdown, we get a clear description of what exactly takes place in the inn as Silas shows up in the doorway:
...the pale, thin figure of Silas Marner was suddenly seen standing in the warm light, uttering no word, but looking round at the company with his strange, unearthly eyes..
Just imagine this for a moment. Silas, who is already notorious for his big eyes, his exotic looks, and his strange personality, is just standing there looking quite distressed at the faces of the pub's frequent visitors. It is no wonder they felt it was some ethereal version of Silas that was at the door, and not the enigmatic weaver himself.
...every man present, not excepting even the skeptical farrier, had an impression that he saw, not Silas Marner in the flesh, but an apparition...
There is an awkward silence that follows for a few moments, as the shock is still fresh in those who are witnessing Silas.
However, the landlord comes over and welcomes Silas in, clearly noticing that Silas is not well at all. When Silas requests that "the Justice—and Squire Cass—and Mr. Crackenthorp" are summoned, the landlord can tell that Silas is out of his mind. He asks Jem Rodney to calm Silas down but, in the end, the Rainbow guests simply have Silas sit down to tell them what happened. As they slowly start warming up to Silas, due to the genuine state of alarm that he showed, the feeling still remained:
The reality of ghosts remained still an open question
The people still felt uneasy and still wondered whether that was really Silas inside the Rainbow Inn.