There are several considerations in answering this question.
First, we need to keep in mind that the sexuality of Walt Whitman is still debated. Some scholars believe that he was homosexual or bisexual, but others are not so certain. His relationship with Peter Doyle was close, but apart from this we do not know much. Moreover, we are not certain, if he ever has sexual relations with the same gender. Hence, we should trod carefully.
Secondly, we need to consider his dates - May 31, 1819 to March 26 1892. During this time, there was still a stigma attached to homosexuality. And if a person cared about public perception, then one's homosexuality was a private affair.
With that said, Whitman was a person who was also in the public eye and cared about politics. For this reason alone, if he was homosexual, there was some uneasiness in the public eye.