The speaker, the Duke, notes that when Fra Pandolf painted the portrait of his (Duke's) last duchess, he was present. The Duke felt he needed to be present because he wanted to control every aspect of the Duchess's life. He is jealous and controlling. He says:
Sir, ’twas not
Her husband’s presence only, called that spot
Of joy into the Duchess’ cheek;
He means that it was not only his presence that caused the Duchess to blush. He implies that it was also the presence of Fra Pandolf because the Duchess liked to be looked at. His claim is that the Duchess is flirtatious even though there is nothing in the poem to support this. He is even jealous of a friar ("Fra" Pandolf). The speaker reveals more about his jealous, controlling treatment of the Duchess than he does about the Duchess herself.
In the portrait, the Duchess has an "earnest glance." This means she has a serious look and is probably not smiling. The speaker's companion asks why there is no smile. The Duke replies:
Oh, sir, she smiled, no doubt,
Whene’er I passed her; but who passed without
Much the same smile? This grew; I gave commands;
Then all smiles stopped together.
She is not smiling in the painting because the Duke was present while her portrait was being painted. The Duke even admits that as he became more demanding and commanding, she stopped smiling. Although his focus is on her portrait, he ends up describing a portrait of himself as a jealous, oppressive husband. Ironically, he is speaking to an emissary of a count in hopes of marrying the count's daughter.