The activity can be seen as an excuse for the speaker to bond with his neighbor for a brief period once a year. He probably sees it as an opportunity to be social and to renew the link they have. There is, furthermore, a clear suggestion that the speaker sees the meeting as an opportunity to persuade his neighbor that they do not need a wall between them after all. He alludes to the fact that a wall is used for protection, either inside or out, but it seems as if neither he nor his neighbor has anything to protect. The speaker's subtle sarcasm suggests that none of them has anything to fear about the other.
He is all pine and I am apple orchard.
My apple trees will never get across
And eat the cones under his pines.
In addition, the speaker makes a point of stating that his neighbor's contention that "Good fences make good neighbours" does not make any sense, because none of them have livestock, such as cows, which may wander from one's area into the other's and cause damage.
It is clear that the wall creates some tension between the two men. The one sees the wall as a purposeless barrier, while the other insists on maintaining a stubbornly traditional approach by referring to the cliched notion that a well-constructed and well-maintained fence ensures good relations between neighbors. In this instance, though, it evidently does not, since our two protagonists are not in agreement about its presence. The speaker's annoyance is, however, contained, and it is his composure and cordial acceptance of his neighbor's idea, rather than the existence of a wall, that prevents any open conflict.