The reader should know Jimmy has changed through direct quotations from the text:

Jimmy Valentine looked into [Abigail's] eyes, forgot what he was, and became another man.

It took the love of an amazing woman to change Jimmy from a criminal into an upstanding citizen.  Yes, Jimmy did a "job" that landed him in jail, but he worked hard even in prison (fixing shoes) and never needed to be "watched" in order to assure his good behavior.  Further, all of his actions afterward point to the fact the Jimmy is no longer a criminal.  Jimmy loves Abigail, and as a result, opens a very successful shoe store in the town and prospered through business and trade in those much-needed goods.

Mr. Ralph Spencer, the phoenix that arose from Jimmy Valentine's ashes--ashes left by the flame of a sudden and alterative attack of love--remained in Elmore, and prospered. He opened a shoe-store and secured a good run of trade.

It is also important to note another particular way that Jimmy uses his past "expertise" in order to become a hero:  Jimmy opens an unbreakable safe in order to save a little girl.  When the little girl becomes locked inside the safe in an unfortunate accident, everyone is scared and baffled as to how to help.  Jimmy shows up as a hero, using his "criminal" skills to save the girl.  At the same time, his actions reveal (especially to the detective) who Jimmy really is.

In conclusion, it is also important to realize that the detective, Ben Price, agrees with us about Jimmy's "change" in character.  Even though the reader fully expects Price to arrest Jimmy Valentine (turned Ralph Spencer), instead Price reveals that he knows Jimmy's true identity, but values his citizenship and goodness so highly that Price does not make the arrest.

How has Jimmy changed in 'A Retrieved Reformation'?

Jimmy's change was triggered by his falling in love at first sight with the beautiful and highly respectable Annabel Adams. It should be noted, however, that he was already beginning to realize that a life of crime is always a mistake. It means that the criminal becomes too well known to law enforcement officers, such as Ben Price. Judges impose harsher sentences. It is easier to get caught and harder to get out. 

He had served nearly ten months of a four year sentence. He had expected to stay only about three months, at the longest.

He also realizes that he is getting to be too well known as the best safecracker in the business. This is brought home to him when the warden seems to know all about him and when his old pal Mike Dolan asks:

“Got anything on?” 

Obviously too many people know about him and too many are talking about him. So Jimmy decides to move all the way to Arkansas, establish a business as a "front," and start operating in a virgin territory. But by chance he sees and falls in love with Annabel Adams. 

Jimmy Valentine looked into her eyes, forgot what he was, and became another man. 

He "became another man," but he was unconsciously preparing to make a radical change before that chance encounter. Annabel is a "catalyst' rather than a "cause" of his reformation. He knows he cannot win her love and remain a criminal. His experience in the shoe shop in prison gives him the idea of opening a shoe shop in Elmore, Arkansas. He is successful because of his sharp mind and winning personality. O. Henry is making the point that a man who has the talent to be a successful criminal can use the same talent to be successful if he goes straight.

In the end, Jimmy's future looks bright. He is a successful and respected citizen, engaged to marry a beautiful girl, accepted by her family and all their friends, on his way to becoming a family man, homeowner, churchgoer, and pillar of the community. He has changed his name to Ralph Spencer in order to put "Jimmy Valentine" well behind him.The change that has taken place in the old Jimmy Valentine is spelled out in the letter he writes a friend to whom he is bequeathing his set of custom-designed and custom-made safecracking tools.

Say, Billy, I've quit the old business—a year ago. I've got a nice store. I'm making an honest living, and I'm going to marry the finest girl on earth two weeks from now. It's the only life, Billy—the straight one. I wouldn't touch a dollar of another man's money now for a million. After I get married I'm going to sell out and go West, where there won't be so much danger of having old scores brought up against me. I tell you, Billy, she's an angel. She believes in me; and I wouldn't do another crooked thing for the whole world. 

For a few tense moments when Ben Price shows up at the bank, it looks as if Jimmy's dreams are  about to be smashed to ruins. But his good deed in saving the life of the little girl accidentally locked in the bank vault, makes such an impression on his nemesis that he is allowed to "retrieve" his reformation and go ahead with his idyllic new life.

What have we learned about Jimmy in "A Retrieved Reformation"?

We learn that although Jimmy used to be a bank robber, he was reformed once he met the love of his life.

Jimmy seems to have been an excellent criminal with good political connections.  He serves 10 months of a four year sentence in jail, but he expected to serve much less. 

When a man with as many friends on the outside as Jimmy Valentine had is received in the "stir" it is hardly worth while to cut his hair.

He thought his friends would get him out of prison before it would be necessary to cut his hair, as they usually do when a prisoner enters the system.  Jimmy was there long enough.  The warden even says that he is “not a bad fellow at heart,” and he just needs to stop cracking safes.

Jimmy has no intention of reforming.  He goes to get a new start, but soon there were robberies with his signature.  Investigator Ben Price took notice.

Jimmy’s reformation began when he arrived in the little town of Elmore, Arkansas.

A young lady crossed the street, passed him at the corner and entered a door over which was the sign, "The Elmore Bank." Jimmy Valentine looked into her eyes, forgot what he was, and became another man. 

It was love at first sight.  It did not take long for Jimmy to become reformed, but he really was.  Ironically, he proves it by cracking a safe.  When his lovely lady’s little sister locks herself inside the safe in the bank, no one knows the combination.  Jimmy has to use his skills to rescue her.  Instead of arresting him, Price realizes that sacrifice he made and decides to let him go.  He feels that Jimmy has retrieved his reformation, and will never crack another safe to steal.