Mr. Loisel means that government officials and important people will be at the ball.
Mr. Loisel is a clerk for the Ministry of Public Instruction. This is a government job, but not a very important or high-ranking one. When Mr. Loisel comes home with an invitation to Ministry ball, he expects his wife to be thrilled because she always wanted to be wealthy, and wealthy people will be there.
“… I had great trouble to get it. Every one wants to go; it is very select, and they are not giving many invitations to clerks. The whole official world will be there."
She is actually not happy, which confuses him. She is going to be at the ball with all of the important people, and it just reminds her of what she does not have. She does not have the right dress or jewels, so she will stand out among the fancy people.
Mr. Loisel's comment is really just another example of his cluelessness. His wife is selfish and arrogant, and he does not get it. She is never going to be happy, because she only wants what she can't have. Unless he grows rich, she will stay unhappy.