Because one of the themes of Gatsby's life is this attempt to break into the posh set and of course in the end to work things out so that Daisy runs off with him, he is incredibly concerned about his image. He has worked so hard to cultivate this very cultured image and he could ruin that by allowing others to know how he got his money.
It is absolutely unacceptable among Daisy's crowd to have had to work for your money, especially if it is tied to anything improper or illegal. It is perfectly acceptable to have huge parties with alcohol, etc., but to be connected to smuggling it would be a mark on his character that nothing could undo.
As the previous post says, definitely rude, but Gatsby can ill afford to tell the truth. Of course we know that he also struggles to lie to Nick (as everyone does according to Nick) so he also can't just make something up, so the response is a rude one instead.