This scene tells the reader much about the character of the men on the ranch, and is the first instance of the motif of death. It is designed to draw feelings of pity and sorrow from the reader, but also illustrate the harshness and hollow nature of the world in which these men live. The themes of loneliness and the struggle for respect are also evident in this scene.
Candy's dog has clearly reached the end of it's productive life and appears to be genuinely suffering. Carlson, who leads the charge to have it put down, points out how it suffers from rheumatism, has no teeth and causes a terrible smell. The reader is surely familiar with the idea of putting a dog down that is suffering, but it is the way that Carlson pursues the idea (at the behest of Slim) despite Candy's soft, helpless protests that may cause us to feel geat pity and sorry for Candy.
Our feelings for Candy may be heightened by his reaction to Carlson taking the dog away and shooting it,
For a moment he continued to stare at the ceiling. Then he slowly rolled over and faced the wall and lay silent. (p.47)
His reaction and the reaction of the men who sit in a guilty, apprehensive silence until the deed is done tells us much about the stifling culture of masculinity in which they are all bound to live.
In summary, perhaps it is not so much the actual death of the dog itself which draws an emotional response from the reader (it was after all genuinely suffering), but the lack of sensitivity shown by the men to its owner Candy, the lonely old swamper who had a deep affection for his once productive sheep dog.