Martin's introduction to the Morse family is difficult for him mainly because of social and class differences. He's self-conscious about being of the working class and not well educated. The Morses are upper-middle-class and "refined": a social stratum Martin is, until this point, unfamiliar with.
Martin has primarily been a sailor, as the narrative indicates from his clothes. But the other sentences you have quoted indicate how, despite his exterior roughness, Martin is sensitive and vulnerable, filled with anxiety about his visit to the Morse home. He needs apologetically to tell Arthur that "you know I didn't want to come." These thoughts are somewhat like Bigger Thomas's in Native Son when he first goes to the Dalton house to be interviewed: he feels that he "should not have come here," though in Bigger's case, it's both a class difference and racial bigotry that cause his obvious discomfort.
When Arthur says "we're just homely people," it's both true and not true. They welcome Martin into their home, but until Martin makes a "success" of himself, he's unacceptable as a suitor for their daughter Ruth. And yet after he's become a famous author and has money, the situation changes completely, and Ruth comes to his apartment, seemingly by herself. Oddly, Martin is no longer interested in her, though he has been in love with her for years. He gently turns her down, and when he escorts her from his apartment he sees a man hiding in a doorway and is sure it's Ruth's brother. Martin reflects that when he had no money, he was considered not good enough to be seen with Ruth, but now that he's rich and famous, "her brother brings her to me." It's a reversal of the dynamic that existed in the opening chapter described in the sentences you have quoted.