In the story, Brown (a "good man") goes into the woods at night on a mysterious errand. His journey is literal and allegorical. He meets a man who resembles an older version of himself, but also resembles the devil. He embarks on this journey to resist the devil's temptations; in doing so, he will try to resist becoming a sinner, like the devil himself. This is why it's fitting that the devil resembles Brown as an older man. Brown is there to resist becoming a sinner as he progresses through his life.
In embarking on this journey, Brown will test his faith in himself and in his community. Brown literally leaves his wife, Faith, to test himself in this way. Brown allegorically leaves his faith in confronting the potential evil in himself and in his fellow townspeople. The question becomes: can he leave his faith (be tempted, witness and/or take part in evil) but then come back to his faith/Faith?
Brown is dismayed when he discovers that the most ethical members of his community are consorting with the devil. He learns that they sin. He is completely distraught to find his wife, Faith, at the devil's meeting. Before he can see if Faith has accepted the invitation to embrace sin, he finds himself alone in the woods. He walks back into town and ignores Faith even though she is overjoyed to see him. He had abandoned Faith by leaving her. As a result of his journey, he has lost his religious faith. He has lost his faith that others can be moral and righteous Christians. When he returns, he cannot embrace his Faith (wife) or his faith (religious conviction that people are inherently good). His conclusion is that if they are all sinners, then so is Faith. He grows old in gloom because he has lost faith in humanity.