This is a great question. Odysseus has been fairly faithful to his men and his wife. So, I do not want to give an impression that Odysseus was characterized by a characteristic of betrayal. However, Odysseus has not been a model of complete faithfulness.
Perhaps the most egregious instance of Odysseus's unfaithfulness is his time with Calypso, the nymph. He stayed with her for seven years after a ship wreck, as a love slave. To be sure, Odysseus was a prisoner, but he still capitulated. They were lovers.
Penelope could have made the same argument as Odysseus. She could have stated that she was besieged by suitors and that she had to give in, but she resisted with great cunning. So, from this perspective, some can say that Odysseus betrayed Penelope to a certain extent.