The Giver is mostly a sympathetic character. He is an old man, prematurely aged by the weight of his position. The memories have aged him. When he begins working with Jonas, we begin to learn more about him. Although he does not treat Jonas with much overt affection at first, over time he and Jonas grow closer.
The Giver is also very careful about how he sends memories to Jonas. He starts with simple and happy memories. Then he has to transmit memories of pain, both because he cannot shelter Jonas forever and because he needs Jonas to take the pain.
However the greatest sympathy for him comes when we learn what happened to his former apprentice, his daughter Rosemary. He tried to protect her, and that led to her suicide. When we learn this, we begin to see into his soul and understand him better.
Finally, when Jonas sees his father kill the newborn and plans his escape, the Giver helps him. This raises our estimation of him, both because he helps Jonas and because he is willing to stay behind and suffer, and help the people through their suffering.