Of the two men, Alec d'Urberville is obviously the worse one in his treatment of and general attitude toward women. He rapes Tess and later makes no apology or attempt to help her with their child. As in most nineteenth-century fiction, the actual events are presented ambiguously. English and American writers of that period were far too reticent to describe explicitly a man forcing himself upon a woman. It's clear that Alec blames his own behavior on Tess, as in the later episode when he forces her to swear that she will never "tempt him again." Though Hardy is generally considered a liberal and progressive writer, one cannot be sure if it is his view as well that Tess has caused her own misfortune. Probably, as with others of his time, he is simply reporting the manner in which society treats women, without necessarily judging this situation one way or another.
Angel Clare, unlike Alec, seems to be depicted as a basically virtuous man (as the symbolism of his name indicates). But to a modern reader, his double-standard attitude to Tess makes him come off almost as badly. Instead of trying to understand and sympathize with Tess when she makes her revelation to him, he coldly dismisses her—in spite of his own admission of a past relationship. Again, it's not clear if Hardy intends this to be seen as a monumental instance of hypocrisy, or if he's just telling it like it is, so to speak: society at that time (and unfortunately still today to some extent) has established one standard for men's conduct, and another for women's. It of course doesn't help that what occurs between Alec and Tess is never described explicitly, and that Tess herself can't tell Angel that she was forced into an affair with Alec. But in any case, both Alec and Angel, even by nineteenth-century standards, are cruel in their behavior toward women.