This is a good question and it gets to the heart of the book in one direct way. However, before I delve into this issue any further, let me give you the context.
As the men are in the bunk, the men complain about Candy's dog. He is old, smelly, and basically useless. They all think that the dog should be put down. Candy cannot stand up to the peer pressure. So, he allows Carlson to put down the dog. In a few moments, a shot of a gun is heard and Candy is filled with pain and sorrow. Here is the text:
A shot sounded in the distance. The men looked quickly at the old man. Every head turned toward him.
For a moment he continued to stare at the ceiling. Then he rolled slowly over and faced the wall and lay silent.
Later, in a conversation with George, Candy says that he should have put down the dog himself. He regrets letting a stranger do it. Candy feels this way because he has been with the dog since the dog was a puppy. So, in the dogs final seconds, he should have been there. Loyalty, faithfulness and being there until the end are all important. This is why he says the following words.
“I ought to of shot that dog myself, George. I shouldn’t ought to of let no stranger shoot my dog.”
This point also gets to the heart of the story because later on, George will face a similar situation. But this time, he puts down Lennie himself, rather than letting someone else do it.