Yank's original definition of "belonging" in The Hairy Apeis based on his physical prowess and his place in the world; he shovels coal on a steamer and takes pride in his abilities, limited though they are.
Dem boids don't amount to nothin'. Dey're just baggage. Who makes dis old tub run? Ain't it us guys? Well den, we belong, don't we? We belong and dey don't. Dat's all.
Yank's perceptions change as he realizes that he only belongs in the engine room of a steamship; he can't "belong" in the outside world. He tries and fails to "belong" to city life, to prison, and then to a union; his concept has failed him because he can't reconcile his emotional states with the reality outside his engine room. Speaking at the end to the gorilla, Yank says:
Youse can sit and dope dream in de past, green woods, de jungle and de rest of it. Den yuh belong and dey don’t. Den yuh kin laugh at 'em, see? Yuh're de champ of de woild. But me -- I ain't got no past to tink in, nor nothin' dat's coming', on'y what's now -- and dat don't belong.
(Quotes: O'Neill, The Hairy Ape, eoneill.com)
Yank's final realization is that he has had nothing to look back on, and nothing to bring forth as a position of superiority; the gorilla can "dream in the past" and remember belonging in its own world, but Yank can't even remember his own belonging because he has changed and messed it up so badly. He used to belong in the engine room, but being scorned by Mildred and failing to adapt in the city has taught him that his life was small and petty, and he had nothing to offer except his strength -- and that can be had from any number of men. His "belonging" is little more than an illusion in his own head, created to cover his insecurities.