To Jimmy Valentine, the presence of Ben Price at the bank indicates that he has again been on Jimmy's trail and is there to arrest him. 

Recalling that it was the acumen and persistence of Detective Ben Price that put him in prison in the first place, it is only reasonable that Jimmy Valentine infers that the presence of Price in the Elmore Bank is purposeful. After all, Elmore, Arkansas, is not a well-known town.

Among other factors that enter into Valentine's thinking that Ben Price is there to arrest him for the robberies since his release from prison are the following:

1. Price is probably very angry that Jimmy received a pardon from the governor after he spent untold hours investigating Jimmy's crimes and tracking him down. This pardon would be considered by Price as a professional insult.

2. Price may also feel personal umbrage by the pardoning of Valentine because he underwent the risk of personal harm to have wrestled Jimmy into arrest in his room in a little town on the state line--

There on the floor was still Ben Price's collar-button that had been torn from that eminent detective's shirt-band when they had overpowered Jimmy to arrest him.

Indeed, the remembrance of all the conditions of his arrest, temporary imprisonment, and release serve to determine Jimmy Valentine's conclusion that Ben Price has the single motive of arresting him, and hopefully, arresting him in the act of a crime.