Your question clearly points to four of the most well-known words in the English language when we think of literature! I think it is important to discuss what Marlow thinks of what Kurtz says and how he interpreted it. As the boat travels swiftly down the river, Kurtz's life is fast running out, and it seems that almost to the end he remains self-deceived. However, just before he dies, it is "as though a veil had been rent" and Marlow reads on his face contradictory emotions of pride and ruthless power on the one hand, but on the other hand, terror and hopeless despair. Kurtz's last moment is one of "complete knowledge" we are told, when he exclaims "The horror! The horror!" Marlow tells us that he interprets Kurtz's final words as "a judgement upon the adventures of his soul on this earth." Whatever the truth Kurtz has glimpsed, Marlow sees its perception as a "moral victory". In other words, he approves of Kurtz's achievement of consciousness; that is why he keeps thinking of him as a remarkable man and remains loyal to him.
Of course, there are critics that use Marlow's position as an unreliable narrator to question his interpretation of these final words, but to me, they do present a moment of self-knowledge where Kurtz comprehends the depths into which he had fallen, and also presents a damning condemnation of the whole colonial enterprise.
As Marlow travels to the Inner Station, he learns not only of Kurtz's legendary status as a trader in ivory, but he also learns about what the jungle can do to a person. His first glimpse of Kurtz's shack (through the binoculars) reveals the human skulls, all (but one) facing the shack, as if the dead worship Kurtz like the natives do.
"The horror! The horror!" is Conrad's wish that King Leopold II recognize what is really happening in the Congo, i.e., how the exploitation of the people and resources leads to destruction and death. Kurtz, the symbolic representation of Leopold, sees what he has done and repents for his crimes with his dying breath.