There most definitely is a special significance to the sea-horse reference. Through the entire poem, the Duke's obesession with control is revealed. He even had his first wife murdered because she didn't behave how he wanted her to.
The most telling words are that Neptune was "taming a sea-horse"; this appeals greatly to the Duke's desire for taming and controlling all around him, especially things like the rare sea-horse. He tamed his rare beauty of a first wife, didn't he? Secondly, he had it "cast in bronze"; if the sea-horse is cast in bronze, it is trapped forever, just like his other precious acquisition, his first wife, who is forever trapped, with her engaging pose, behind the curtain.
Browning wanted to indicate that this room contained a number of works of art. It seems to be the Duke's private study or library. He has brought the visitor up here for a confidential discussion about the dowry of the girl he expects to marry. In the monologue it would not have been convenient for him to refer to other works of art in the room, but calling attention to the sculpture of Neptune taming a sea horse is a way of suggesting that there are paintings and sculptures in various places. The Duke also seems to be trying to delay the visitor's hasty withdrawal. The Duke is obviously surprised when the visitor suddenly jumps up and starts to leave without a word of explanation or apology. The Duke says, "Will't please you rise?" And then, "Nay, we'll go / Together down, sir." The visitor was going to leave the Duke behind in his hurry to get away from him. The Duke is not satisfied that they have agreed about the terms of the dowry. He didn't expect this interview to end so abruptly. He asks the visitor to "Notice Neptune, though." He would like to hold the man there for further conversation. The actual sum of the dowry has not been settled. They only came up here away from "the company below" on the pretext of looking at art but for the main purpose of talking about money. If the Duke could have enticed the visitor back to look at Neptune, he might then have directed his attention to other artworks and drawn him into further conversation about the dowry. The Duke doesn't realize what a terrible impression he has made on this representative of his fiancee's father and that he may have even talked himself out of the marriage.
One special significance to the last three lines
Notice Neptune, though,
Taming a sea-horse, thought a rarity,
Which Claus of Innsbruck cast in bronze for me.
is that the narrator's cold, calculating, and vain nature is clearly revealed. He has turned from speaking with a subdued passion of bitterness:
... Sir, 't was not
Her husband's presence only, called that spot
Of joy into the Duchess' cheek / ... /
... She had
A heart - how shall I say? - too soon made glad,
to speaking of his future bride and her dowry ("no just pretence / Of mine for dowry will be disallowed") to speaking of a statue of Neptune that some artist worthy of name-dropping made especially for him. This clearly shows that, while logical and rational, he has not one grain of human feeling in his cold controlling heart; it is as cold as Neptune and the sea, as controlling as the taming of the magnificent sea-horse.
In addition to the cogent points already made, another indication of the subservience of the duchess is the symbolism of her being the seahorse, a unique but delicate creature who dwells only in the realm of Neptune. Thus, the duchess, like the seahorse, is confined to the world of its lord and master.
All true. This is the last image the Duke leaves with the ambassador for his next duchess as they walk the upstairs gallery, and it's the last thing he shows him for a reason. As has already been stated, the image is one of power (god) and taming (the much more docile sea horse) and bronze (heavy, unbending, powerful). It was, of course, made just for him (arrogance) by a master (which, of course, is the only one worthy of working for this Duke). This statue screams out loud all the subtleties of everything else the Duke has already said. If the ambassador misses this message, he's not too bright.
I would only add that it's interesting that the bronze was cast "for me." Consider that the second word of the poem is "my," and the last word is "me" and it's clear that the entire story is framed with the control that is clearly all the Duke is about.
There is also a suggestion of his power; after all, how difficult would it be for Neptune to tame a sea horse :)
Sources are listed in the first entry.