I would argue that the main trait that Odysseus displays that was not seen as being heroic in ancient Greece was the way that his actions were at times dominated by incredible arrogance and how he therefore rejects the wise and sage advice given to him by other characters because he believes he knows best.
A great example of this can be found in Book 12 of this epic classic, when Circe counsels Odysseus on how to pass Scylla and Charybdis. Even though Circe tells Odysseus not to try and pick a fight with the immortal Scylla, as nobody can vanquish this beast, Odysseus first of all does not tell his men about Scylla and secondly ignores Circe's advice and arms himself in an attempt to try and kill Scylla and gain glory. He loses six of his best men as a result, and the text makes it clear how much this haunts him:
Right in her doorway she ate them up. They were screaming
and reaching out their hands to me in this horrid encounter.
That was the most pitiful scene that these eyes have looked on
in my sufferings as I explored the routes over the water.
Odysseus therefore shows himself to be a man who at various points in his journey is overwhelmed by arrogance and a desire to gain glory by attempting impossible feats. Often this gamble pays off and he shows he is able to conquer unconquerable forces. However, at other points, his arrogance gets too much for him and innocents die and suffer as a result.