As the story opens, it is explained that Santiago, the old man, has had a run of bad luck fishing. People around him call him "Salao," or unlucky, and the boy Manolin who used to help him has been admonished by his parents. However, he is a fixture in the community, and is even respected by some:
They sat on the Terrace and many of the fishermen made fun of the old man and he was not angry. Others, of the older fishermen, looked at him and were sad. But they did not show it and they spoke politely about the current and the depths they had drifted their lines at and the steady good weather and of what they had seen.
(Hemingway, The Old Man and the Sea, Google Books)
The older fishermen know that bad luck can happen to anyone, and they take pains to be friendly, because of superstition and fear of karma. However, others tease Santiago, probably not out of spite, but because it is a culturally accepted form of entertainment; Santiago has been around for so long and is so safe that teasing him is all right. For his part, Santiago is not concerned with their teasing, partly because he knows that his luck must change with time, and partly because he knows that their words are meaningless.