The speaker does not like walls because he doesn’t feel that they are needed, but his neighbor prefers him.

When the speaker says, “Something there is that doesn't love a wall” (line 1), he is describing the fact that walls seem to deteriorate over time, and have to be maintained.  He does not see the need to repair the wall between his far and his neighbor’s.  After all, neither of them has livestock.

He is all pine and I am apple orchard.

My apple trees will never get across 

And eat the cones under his pines, I tell him. (lines 24-6)

The neighbor just grunts that, “Good fences make good neighbors” (line 27).  He believes that if your neighbor sticks on his side of the fence while you stick to yours, there will not be conflict between you.

The gruff neighbor prefers his own personal space.  He does not seem inclined to be friendly or make overtures.  The speaker, on the other hand, both decries the need for maintaining the wall and the symbolic isolation it represents.

While most of us agree that it is important to be a good neighbor, the definition of such a thing is widely open to interpretation.  As we see in this poem, two neighbors can have widely different variations on the concept.