The Duke seems to be an absolutely cold, unloving, insensitive, and cruel person who would be dissatisfied with a wife regardless of what she was like. He complains that she was too friendly and not sufficiently dignified, but it may have been simply that he got tired of her and wanted to try another wife with another dowry. He describes his last Duchess disapprovingly, but ironically everything he says about her only makes her seem more charming and lovable. In one place he tells his visitor that "...such stuff ... was cause enough / For calling up that spot of joy." He seems to dislike the idea that a wife of his should be joyful at all. He seems to want a beautiful youthful bride who would be just as cold as he is, but this is close to an impossibility, and the reader (as well as the Duke's visitor) must be appalled at the thought of what will happen to his next Duchess if the marriage is arranged.
The Duke is evidently not looking at his visitor while he is talking but looking at the portrait of his murdered wife and recalling how she displeased him with the very qualities in her person that Fra Pandolf managed to capture in his painting. The Duke calls the portrait "a wonder," but he could not see that his young wife was a much greater wonder, being real and alive and possibly even in love with her aristocratic husband, or at least excited by her new status as a Duchess.