Frank and Millie have been carrying on an adulterous liaison. Frank, Andrew's work colleague, is everything that Andrew isn't: handsome, witty, well liked, and charismatic. It is precisely such qualities that draw Andrew's wife, Millie, toward him and lead to their conducting an illicit affair.
However, the affair isn't destined to last. As time goes by, Frank starts to feel rather sorry for "Old Crock," to use Andrew's none-too-flattering nickname. He gradually becomes more sympathetic to his more senior colleague, which eventually leads to his breaking off the affair with Millie.
Unlike Millie, Frank bears no animosity toward Andrew and so finds her callousness toward her husband rather unpleasant, to say the least. When Frank says how sorry he feels for Andrew, it's notable that Millie responds by saying that she's the one he should be feeling sorry for. It's such selfishness and cold indifference to the feelings of others that eventually leads Frank to break things off with Millie. For good measure, he encourages Andrew to leave her, albeit without success.