Robert Frost's poem The Road Not Taken has an enquiring tone and the speaker - who is not necessarily the poet - questions his decision at every step. He even suggests that in years to come he "shall be telling this with a sigh" and there is a suggestion that he will still be wondering whether this was the best decision.
Any decision in life has consequences and opportunities of its own. However, it also has potential missed opportunities. "The difference" for the speaker in The Road Not Taken is that whatever results from his decision will make or have made all the difference in his life. He knows that he is unlikely to even know what the other "road" may have resulted in because "I doubted if I should ever come back."
By using the title of The Road Not Taken, the speaker is placing equal emphasis on the path he did not take as much as the one he did. His choice really did make "all the difference."