Jimmy's suitcase is so heavy because it is full of all his safecracking tools, which are made of tempered steel. O. Henry makes frequent references to this suitcase throughout the story. In his first such reference, the author gives a full description of the contents of the suitcase so that the reader will know what they are.
Pulling out from the wall a folding-bed, Jimmy slid back a panel in the wall and dragged out a dust-covered suit-case. He opened this and gazed fondly at the finest set of burglar's tools in the East. It was a complete set, made of specially tempered steel, the latest designs in drills, punches, braces and bits, jimmies, clamps, and augers, with two or three novelties, invented by Jimmy himself, in which he took pride. Over nine hundred dollars they had cost him to have made at—, a place where they make such things for the profession.
After that O. Henry only mentions the suitcase from time to time to keep reminding the reader that Jimmy has it with him. It will become crucial to the story at the end, because Jimmy will open it to take out the necessary tools to rescue the little girl from the bank vault. At the same time, the suitcase and its contents will expose him to his fiancee and her family as a safecracker and not the honest businessman he is pretending to be. Furthermore, the suitcase could be used as evidence against him if Ben Price arrested him for the three bank jobs Jimmy pulled after getting out of the penitentiary. O. Henry arranges the plot in such a way that Jimmy will have the suitcase with him when he goes to see the new bank vault with Annabel's father and all the others. O. Henry inserts the following bit of action and dialogue for the purpose of reminding the reader--yet again--that Jimmy has the suitcase with him.
Jimmy set his suit-case down. Annabel, whose heart was bubbling with happiness and lively youth, put on Jimmy's hat, and picked up the suit-case. “Wouldn't I make a nice drummer?” said Annabel. “My! Ralph, how heavy it is. Feels like it was full of gold bricks.”
By opening that suitcase Jimmy is making a heroic sacrifice for the sake of a little girl who would certainly die if he didn't open the supposedly impregnable door of that vault. The suitcase which was once his pride and joy has become a big handicap since he decided to go straight. He wants to get rid of it. He was planning to take it to Little Rock, Arkansas on the train and give it to an old friend. That explains why he has it with him when he goes into the bank with Annabel and her whole family. Fortunately for Jimmy, Ben Price sees his act of heroism and decides he is a thoroughly reformed man. The author does not state what happens to the suitcase after Jimmy leaves it behind. No doubt Ben Price takes charge of it and dumps it in the river.
It might be said that the suitcase is represented as being so heavy because it symbolizes Jimmy's past. He is having a hard time getting free of that criminal past, just as he is having a hard time getting rid of that suitcase.