Is the speakers's attitude toward the wall not necessarily Frost's?
It is always assumed that the first-person narrator in any poem is the author, until the poet gives evidence otherwise. For example, Shakespeare’s sonnet are virtually always interpreted as being his voice, even part of his biography, because he gives no evidence otherwise; but by contrast, look at Browning’s My Last Duchess. Even the title shows that Browning is creating a fictive persona for the first-person narrator (since Browning was not a duke). Since Frost lived in New England, where stone walls, apple orchards, etc. are common, and since no contrary evidence is presented in the poem itself, and since the personal tone of the poem’s observations implies a real-life exchange between real-life landowners, the reader has no reason to imagine any other persona than the author’s.
Unless some biographer finds external evidence that Frost could not have meant himself, or that the sentiment in the poem is contradictory to Frost’s own attitude about good fences, there is no support for another view. But the idea of questioning the assumption in a first-person poem is intelligent and healthy.