I would only add to what mwestwood already noted that Whitman was very disppointed at the end of his life by the economic and social horrors of "the Gilded Age" (Wikipedia has a good introduction to this age; I posted a link below). The poet of the common man, who tried to celebrate the Democratic ideal, who was able to see all of us as leaves of grass, saw the vision take a horrible turn as class divisions sharpened and many economic downturns (recessions/panics/depressions) served to separate the haves from the have nots.
If you have access to it, I would suggest you watch Ken Burns "New York," particularly the 3rd and 4th DVD. It will provide you with a visceral experience of the New York that Whitman so loved.
Since Walt Whitman (1819-1892) lived throughout most of the 19th century, he saw many changes take place in America, chief among them the Industrial Revolution which gave birth to a rising middle class. In the North there were many factories and much overcrowding and poverty in the cities. Poor children rarely went to school as they had to work. They often starved, were infected with lice, or died in childhood of disease. In the South there were wealthy landowners or poor sharecroppers and slaves--no middle class.
Whitman was the second son of a housebuilder, so his family was part of the new middle class. Within this class there were divisions, as well: the industrialists believed in free trade and "laissez-faire." There were two basic feelings of the middle class toward the lower class: The lower class was morally inferior and/or intellectually inferior, or the lower class was victimized and needed instruction in values, morals, and social reform. Whitman certainly fell in the perimeter of the latter way of thinking. When he moved to New Orleans as a young man, he was appalled when he first saw the slave market. Upon his return to New York as a journalist, he founded a "free soil" newspaper, the Brooklyn Freeman. During the Civil War, Whitman helped care for wounded soldiers, one of whom was his own brother. After the Civil War, he vowed to write and "purge" and "cleanse" his life. He lived on a clerk's salary and later moved to Camden, New Jersey and cared for his aging mother.
The Romantic Movement in literature certainly fit Whitman with its emphasis upon feelings, imagination and intuition as a means of acquiring knowledge, its conviction that poetry is superior to science, and its distrust of industry and city life as well as an interest in the more "natural" past and in the supernatural.