The first significance of the New York City setting is the plot detail which was based on the actual historical event of the bombing of the Twin Towers. Foer chooses this event as the catalyst for the death of the protagonist's father. In many ways, this detail could have been created in almost any manner or fashion, as long as the death was sudden and unexpected. The protagonist himself never really processes the fact that his father was killed in a terrorist attack, but instead seems unable to escape his curiosity, wondering exactly how his father died. This character quality could have been developed in another manner of death, therefore, in many ways, the NYC setting seems to be more of a social commentary that would appeal to readers, even if it was not absolutely necessary for the development of the plot.
Another significance of New York City is its size. As Oskar Schell goes in search of the mystery of Mr. or Mrs. "Black," it is particularly important that he does not live in a rural town where everyone knows everyone, and "Black may or may not be a common name. In New York City, Oskar's idea to visit every single Black in the phone book is an impossible task. His journey takes him to every single one of the five boroughs, and in some ways, this could be considered the author's tribute to a city he knows and loves.
Finally, New York City is an American icon. It was such even before September 11, but even more so afterward. In many ways, setting a modern novel in New York City could almost be considered cliche, at this point. However, because readers experience (or re-experience as the case may be) the city from the curious and eccentric eyes of nine year old Oskar, it is almost like the experience is made completely new. Perhaps the author wished to take a universal experience, base it around a well known and well loved place, but then tell a story with all of this that had never before been told.