The quest that Oskar undertakes is precipitated by a key that is found in an envelope with the name "Black" on it. This causes Oskar to think that this key was left deliberately for him by his father for him to find the lock that is opened with it. This is the first "Black" that is important to Oskar, as he sees this Black as a direct communication to him from his dead father, and believes that he has been left with a quest to complete.
Of course, what Oskar quickly realises is that there are a large number of people with the last name of Black in his area, but out of the many that he meets, he does find two who are particularly important to him. The first is Mr. A. R. Black, who is a gentleman who has secluded himself from the world and deliberately switched his hearing aids off. In a very moving moment in the novel (one of many), Oskar switches his hearing aids back on for him so that he can participate in the world once again:
Mr. Black grabbed at his ears and made a bunch of weird sounds. He started crying--not out of happiness, I could tell, but not out of sadness, either.
Mr. Black comes to join Oskar in his quest, and loves the way that Oskar has helped him reconnect with the world. He becomes an almost surrogate grandfather to the boy.
The final Black that is important to Oskar is the first Black he finds. Abby Black tells Oskar that her ex-husband might be able to help him, and Oskar discovers that Abby's former husband sold the vase where he found the key to his father as a present for his wife. The key, Oskar is told, belongs to William and is not a secret key left for Oskar by his father. This solves the mystery. These therefore are the three Blacks that are important to Oskar in the novel.