The bank manager is obviously annoyed because the narrator causes so much trouble about opening what is a very small account. The narrator, presumably Stephen Leacock himself, only wants to deposit fifty-six dollars and then another fifty dollars a month. Leacock claims to be totally ignorant about banking matters. He should have gone directly to one of the tellers, or else to one of the clerks. Instead he asks to see the manager and gets conducted into that man's private office because it is assumed that he is there on a very important matter. The manager himself assumes that Leacock is an operative from Pinkerton's, a famous private-detective agency which was heavily involved in protecting banks in Leacock's day. (Ben Price, the detective in O. Henry's short story "A Retrieved Reformation," evidently worked for the Pinkerton National Detective Agency.) The manager feels alarmed by Leacock's mysterious manner.

 “You are one of Pinkerton’s men, I presume,” he said.

When Leacock tells him he is not a detective but simply wants to open an account, we read:

The manager looked relieved, but still serious; he concluded now that I was a son of Baron Rothschild, or a young Gould.

Sir Nathan Rothschild (1840-1915) was a member of the enormously powerful Rothschild banking family in Europe. Jay Gould (1838-1892), one of the American "robber barons," was among the richest men of his era. He made his fortune in railroad development. Leacock is obviously young, so the manager might take him for a junior member of the Rothschild or Gould family. When the manager finds out that the narrator is only bringing fifty-six dollars to his imposing institution, he shows his annoyance by his body language and tone of voice.

The manager got up and opened the door. He called to the accountant.
"Mr. Montgomery," he said, unkindly loud, "this gentleman is opening an account; he will deposit fifty-six dollars. Good morning."
"Good morning," I said, and stepped into the safe.
"Come out," said the manager coldly, and showed me the other way.

Leacock is so befuddled after his embarrassing interview with the bank manager that he causes more trouble when he is turned over to one of the clerks. The narrator ends up depositing his fifty-six dollars and then inadvertently writing a check to withdraw the entire amount. At the end of the story he tells us:

Since then I bank no more. I keep my money in cash in my trousers pocket, and my savings in silver dollars in a sock.