Ulrich von Gradwitz regarded Georg Znaeym as an interloper for two reasons.
First, there was a court case a few generations ago between the grandfathers of Ulrich and Georg over disputed land. The courts favored Ulrich's family. Therefore, the land belonged to them. However, the Znaeym family never accepted the ruling. So, they continued to use the land. When they did this, Ulrich's family saw them an interlopers, who were trespassing on their land.
Second and more importantly, when Ulrich became the leader of his family, he took the trespassing of the Znaeyms personally. He loathed them from deep with his heart. He hated Georg so much that he wanted to meet him face to face to kill him. So, even more than the court case, Ulrich saw Georg as an interloper on account of his pride and anger. Here is what the text says:
The neighbor feud had grown into a personal one since Ulrich had come to be head of his family; if there was a man in the world whom he detested and wished ill to, it was Georg Znaeym, the inheritor of the quarrel and the tireless game snatcher and raider of the disputed border forest.