What Wilson means by a four-letter man is a college athlete who goes out for many different sports. At the upper-class Ivy League colleges like Harvard and Yale, it was common for athletes to wear varsity sweaters with symbols sewn on them to show which sports they participated in. To have four letters on a varsity sweater would be unusual and distinctive. Varsity-level college athletes were often described as “lettermen,” and varsity sweaters and jackets were often called “lettermen sweaters” and “lettermen jackets.” The sports would probably be football, tennis, and perhaps swimming, golf, track, boxing, and almost certainly rowing. The varsity sweaters and jackets would be in the school colors. There are still plenty of them sold, as can be seen on Google. Hemingway never went to college and was probably both envious and contemptuous of men who did. Since Robert Wilson is an Englishman, he must have known about these varsity sweaters and jackets because they were imitations of the same sorts of garments worn by athletes in the upper-class English schools like Oxford and Cambridge, where cricket would have been included.
Well, it is an intentionally obscure reference, but the Dictionary of Sex says that it is a discreet but insulting way to refer to a male homosexual (h-o-m-o are the four letters).
One academic argues that it is "the c word" (female genitals--trying not to be crude here). Either would fit, given Hemingway's hypermasculinity, but I prefer the first answer.
"Four Letter man" is 1930's slang for stupid, the four letters being "D U M B"
It is also a slang term (as Per Cassell's dictionary of Slang) [1920's+]meaning an unpleasant person, the four letters being S-H-*-T - or worse.
The idea of it being "H-O-M-O" for "gay" isn't supported by the dates, since it isn't referenced with that usage until the 1940s, years after the publication of this story.
Although the google search returns 'homo' most often, this definition does not seem to fit the context of Hemingway's story. Perhpas the definition "a contemptible man" provides a better rendering of Hemingway's intent. (From probertencyclopaedia.com.)