If you look at the line 2, "And sorry I could not travel both," it does not say it was a difficult decision, but that the speaker was sorry that the two paths could not be travelled both at the same time. The speaker pauses to consider both paths. If you look at line 10, "Had worn them really about the same," while one path is less travelled, they both have the same wear, which would imply that while the more travelled can be seen as such, the less travelled at one time was worn down by travel.
For the speaker, the challenge is to go with the more well travelled or not. No matter what the decision, it would have made the difference because either path was equal.