There are several different ways of looking at this question, and in the third book of the series (World of Wonders) Leisl explains some of the reasons she used that phrase. This was spoken by the "brazen head" (which was Leisl's distorted voice from backstage) in Eisengrim's show after Boy Staunton was found dead, to a theatre audience. The person who shouted out the question "Who killed Boy Staunton?" was the dead man's son, David, who was drunk and distraught (this is explained in the second book in the series The Manticore.) Leisl, in order to preserve the mystery of the show, had to come up with a fittingly cryptic answer on the spot (something she was good at), and make it sound plausible without actually implicating anyone specifically.
The full text of the answer is "He was killed by the usual cabal: by himself, first of all; by the woman he knew; by the woman he did not know; by the man who granted his inmost wish; and by the inevitable fifth, who was the keeper of his conscience and keeper of the stone." If we assume that "the woman he knew" was Boy Staunton's wife, Denyse Hornick, that would fit with one view of his death. Denyse arranged for Boy to become Lieutenant Governor of the province, which led him to suicide (possibly) because he discovered he could no longer face his life. So if Denyse was the woman he knew, the woman he did not know would be Leisl. Leisl, who never met Boy, was part of the reason that Eisengrim's show came back to Canada (Eisengrim was, understandably, never that interested in touring Canada, as it was the place of his horrible childhood). Leisl wanted to see Dunstan, her sometime lover, and since Leisl was a major investor and the impressario of the show she could influence Eisengrim. If Eisengrim had never come to Canada, in all likelihood Boy would never have learned that he was responsible for Mary Dempster's madness, and he might never have committed suicide (or been murdered by Eisengrim, depending on which version you believe.)
There is another possiblity of the "woman he knew, and the woman he did not know." Boy's first wife, Leola Cruikshank, was an utter failure in his eyes. Though she was beautiful, she was simple and rather unintelligent, and very unsuited to being the social leader that Boy wanted her to be. Boy fundamentally misunderstood his wife, and made her very unhappy (and was unfaithful to her) for many years. This woman, perhaps, drove him to further flights of egotism; a woman of stronger and more determined character might have swayed Boy away from his greater excesses. So Leola could be the woman he didn't know, and the marriage to her further deepened the bad parts of his character because of his inability to know her. The woman he did know could be represented by his second wife, Denyse. She was of very similar character to Boy; so much so in that they had such unbridled ambition that actually made Boy question his own grasping social climbing. Perhaps Denyse's own quest for power and prestige, which he recognized as his own character trait, made him re-examine his own motives for ambition. Upon this examination his life was found wanting and hopeless, and, perhaps, he took his own life because of it (or allowed himself to be murdered, perhaps.) The phrase the "woman he didn't know" was supposed to be cryptic in the first place, and can be thought of many different ways.
The woman he didn't know was Mary Dempster - the snowball that contained the rock was what caused Mrs. Dempster's simplicity, leading to Paul running away from home. Paul was not the reason that Boy wanted to kill himself, but Paul presented the opportunity when he hypnotized him and let him drive into the lake. (I may be fuzzy on the actual death, but you get the idea) I understand that there must be different opinions on who the woman is, but that seems like the most logical choice to me.