The men in the boat are very frustrated by the knowledge that after fighting so hard for their lives, and getting so close to shore, they could still die. This seems extremely unfair to them.
"If I am going to be drowned---if I am going to be drowned---if I am going to be drowned, why, in the name of the seven mad gods who rule the sea, was I allowed to come thus far and contemplate the sand and trees?"
Of course, that is the entire point of the story, that life is often unfair. The ending of the story exemplifies this point because of all the men who in the boat, Billy was the most deserving. He had done most of the rowing and seemed the strongest of the group. However, Billy is the only character to drown, indicating the unfairness of both life and fate.