Why did Marianne Moore invent the nonce form for the poem "The Fish"?
Concerning Marianne Moore's "The Fish," your question is a bit problematic, which is probably why no editors have answered it. I've been waiting to see if someone else would, since I'm not sure of my answer. Since no one else has before now, either, I thought I would try to help you out.
The question is problematic for two reasons. First, I can't ever really speak for an author. I can't go back and read Moore's mind. Unless I have some external evidence, such as comments she made about the poem or her writing process, I really can't say why she does anything in the poem. I can speak in terms of functions and effects, but I can't speak for the author.
Secondly, I've never heard of something called the "nonce form." The term nonce is most often used concerning literature to designate nonsense words created for a specific purpose or poem. Lewis Carroll's "Jabberwocky" is a poem that uses nonce words, such as "brillig" and "slithy." But I don't see any of those in Moore's work.
Nonce is also associated with a single use or occassion or purpose for things other than words. Moore's poem, however, seems to suggest vastness and universality and endless strands of time.
In short, I'm not sure how the term "nonce" applies to "The Fish," and, specifically, I don't see how the term applies to the form of the poem, which is what you ask about.
Concerning the form, enotes Study Guide on the subject says the following:
“The Fish” is written in rhymed syllabic verse: organized in eight five-line stanzas, the poem is rhymed a-a-b-b-c. The syllabic pattern for each stanza is 1, 3, 9, 6, 8. In addition to end rhymes, Moore uses a variety of internal rhymes including slant rhyme, off rhyme, consonance, and alliteration. The sound of her poem mimics its subject matter. In the second through fourth stanzas, for example, the “s” sound dominates, echoing the ripple and splash of water itself. Her organization of the poem into syllabic units also provides the poem with its visual shape. Like the sea it describes, the poem ebbs and flows, the number of syllables expanding and contracting with each line.
Doing something different for a poem doesn't necessarily create a new form. For instance, Moore arranges the lines is a visual pattern that seems to represent the ebb and flow of the sea. This is referred to as concrete imagery: the poem looks like its subject. That doesn't mean that Moore created the form, though. She creates the specific concrete imagery in this poem--the ebb and flow--and it represents the scene she describes, but that doesn't mean she created the form of using concrete imagery. She creates the form of the poem, but not the idea of concrete imagery. At least not as far as I know.