The work being done, as described by the poem "Mending Wall," would be work with which Frost would be very familiar. The ground in New England is very rocky. In order to have some areas of relatively rock-free soil for growing crops or grazing animals, farmers have to collect the rocks and put them somewhere else. The easy and useful way to deal with those rocks is to use them to erect rock wall fences between fields. The walls are held together by the weight and shape of the rocks piled carefully on each other - "We have to use a spell to make them balance."
The poem starts with spring's arrival, which reveals the stones that have worked their way to surface level as the ground has shifted over the winter, and shows the places where the stones already in the wall have fallen which "makes gaps even two can pass abreast."
The poem also recognizes the traditional New Englander's reserved manner, not saying any more than is essential.