A fine question. The short answer is, we don't know. The facts of the case are hidden from us as the minister's face is hidden from the congregation.
The somewhat longer answer is, perhaps—it is certainly implied that they have a relationship. Their spirits are seen walking arm and arm, as if they were lovers, and he only starts wearing the veil after she dies. They could have been lovers. It is also possible that he longed for her without acting, and is still ashamed of this.