The significance of the grave is relevant to how the man deals with his frustration and grief. He has quickly and deliberately dug the grave, which has helped him relieve some of his tension and frustration and sadness. His wife simply does not see that this has HELPED him and that his digging of the grave is not some trivial task that he gets through. In a way, this was a small form of therapy for the man. The wife's way of grieving is her own, and it is understandable and warranted, of course.
In this dramatic poem, the child's grave symbolizes how a man and a woman deal differently with the tragic loss of their child. For the father, the grave is something practical, something he must dig in order to have a proper burial. For the woman, however, the grave signifies not only her baby's death, but the fact that her husband does not seem to care about what has happened. She believes that since he is able to "make the gravel leap and leap in air" and then come inside and "talk about your everyday concerns" means that he "couldn't care" about what has happened. They have shared a tragic experience together, but, because they grieve in different ways, they grieve alone.