One of the central aspects of A Christmas Carol is the fact that Scrooge is a very mean old man at the beginning of the story and has changed drastically by the end.

In the first page of the story, Dickens writes,

Hard and sharp as flint, from which no steel had ever struck out generous fire; secret, and self-contained, and solitary as an oyster.

This description of Scrooge explains why he might have been ignored and avoided. It says he is solitary as an oyster, and as we read through the story, it's clear that his solitude is somewhat by choice but would be difficult to change anyway, since the people around Scrooge have determined that he is better off alone.

People avoided interacting with Scrooge. In fact, the story says, even beggars didn't dare ask him for help, since everyone knew his reputation and knew he was greedy and jealous with his money.

No one spoke to him on the street, and he had no friends or family inviting him for meals or special events.

The text says that even dogs seemed to avoid him!

Although people do business with Scrooge, they do not socialize with the man.  In Stave I, Dickens writes that no one would stop him on the street to say, "with gladsome looks, 'My dear Scrooge, how are you? When will you come to see me?"  Nor do any of the beggars ask for a "trifle"; even the children do not bother to approach him and ask what hour it is; not a man or woman his entire life asks directions of the miserly Ebenezer Scrooge.

With humor, Dickens adds that blind men's dogs recognize him, but they pull their owners into doorways or into courtyards to hide them as they wag their tails at Scrooge as if to utter,

"No eye at all is better than an evil eye, dark master!"

This avoidance of people does not hurt Scrooge; rather, he likes that people are not in his way.