Definitely. I think you need to be aware of the way that Frost presents the ambiguity over the wall. The speaker is very open about his distaste for walls, saying that he doesn't love them and he presents the wall in a way that seems to emphasise the way that it divides people. Note the following example:
And on a day we meet to walk the line
And set the wall between us once again.
We keep the wall between us as we go.
Note how the wall is what separates the speaker from his neighbour and prevents them from being together. Even when the speaker feels that they do not need a wall as their territory is clearly marked by the different trees, the neighbour presents his opposing argument:
"Good fences make good neighbours."
However, this leads the speaker to make a series of very thoughtful and pertinent questions and statements, such as this:
Before I built a wall I'd ask to know
What I was walling in or walling out,
And to whom I was like to give offence.
This quote above all cements the way in which this poem is about human divisions. Building any form of wall means that you are implicitly walling in some people and walling out others, thereby causing offence. Frost therefore symbolically writes about how we built walls, real or metaphorical, that may give us a sense of security but also separate us and prevent communication and true togetherness.