In this scene in Act 2, Miss Prism and Doctor Chasuble discuss their views on marriage and the Primitive church with Miss Prism giving her stern opinions in what Wilde describes as her "sententious" ways.
One of Prism's opinions is that Doctor Chasuble should be married. He disagrees because he believes in the original tenets of the church which go against matrimony.
It is obvious that Miss Prism has a romantic interest in Dr. Chasuble. It is also clear that Miss Prism is of a much stronger nature than she lets out and that she is willing to get Dr. Chasuble's attention and perhaps even make him fall in love with her. Yet, she cannot do any of this as Chasuble continues to reject the idea of being married. To this, Miss Prism says that celibacy in men creates temptations in society for women:
...by persistently remaining single, a man converts himself into a permanent public temptation. Men should be more careful; this very celibacy leads weaker vessels astray.
This is Prism's advice to Chasuble: his decision to not marry will put him at a greater risk of creating lust or bad desires in women (perhaps Miss Prism herself?). She also criticizes celibacy (for the same personal reasons), and strongly continues to advocate that Chasuble marries since, once married, he will have a woman devoted to him only. This is where she says the quote you refer to. She means to indicate that married men lose all their appeal, thus all their tempting qualities, except in the eyes of their own wives, which, she suggests, is the correct way of the world.
No married man is ever attractive, except to his wife
Again, Miss Prism uses this as a way to convince Chasuble that marriage is a good thing, that it brings loyalty with it, and that makes him less "dangerous" to the outside world.
By saying those words, Miss Prism also becomes Wilde's own mouthpiece. Wilde was well-known in his society for how much he loathed marriage, even though he was married himself. We find marriage epigrams in the majority of Wilde's comedies of manners. His disdain for marriage, the idea of marriage, and the expectations bestowed upon marriage by middle class Victorians are consistent targets of ridicule in Wilde's works.