The antagonist in "The Adulterous Woman" is actually Janine's husband Marcel. His antagonist's role is established from the beginning of the story as he persuades Janine to go on a trip she is not interested in, which results in Janine's open--though private--negative assessment of Marcel's qualities and of her life with him. The culmination of his status as the antagonist occurs when he stays asleep at the hotel while she goes again to the ruins of the fort, running faster and faster to get to the place where she can relive the freedom and joy she felt there the first time she, with Marcel accompanying, visited it.
The other men in the story act in an antagonistic manner although they are not antagonists in the story. The soldier in the bus and the other men who fail to notice Janine's feminine presence and power represent an antagonism toward Janine's female freedom and authenticity. This undergirds Camus's thematic concern for the discovery and expression of personal authenticity in an inauthentic life filled with habit, which drowns out an authentic understanding of self and an authentic life.