This is a great question. It is absolutely clear, that, as for all of us, Amir's childhood relationship with his father greatly impacts him in his adult years. One central aspect that you have to focus on is how Amir is presented as being so incredibly different to Baba. Note that this is something that causes Baba himself to doubt his own paternity of Amir:
"If I hadn't seen the doctor pull him out of my wife with my own eyes, I'd never believe he's my son."
Throughout his life, Baba is described as a strong character who is eager to stand up and fight against injustice to protect the weak. And yet note how Amir reacts to a number of key situations: he is scared of Assef, and crucially does nothing to help Hassan and prevent his rape. Likewise, when Baba and Amir flee Afghanistan, Amir tries to dissuade his father from challenging the Russian soldier who wants to rape one of the women travelling with them. Amir is constantly aware of how he does not measure up when compared with his father, and this is something that Rahim Khan uses when Amir is an adult to convince him to go back to Kabul and rescue Sohrab:
"You know," Rahim Khan said, "one time, when you weren't around, your father and I were talking. And you know how he always worried about you in those days. I remember he said to me, 'Rahim, a boy who won't stand up for himself becomes a man who can't stand up to anything.' I wonder, is that what you've become?"
It is crucially this haunting recognition of how Amir has not lived up to Baba's larger than life personality that drives him to recognise and accept this part of his legacy and to face Assef and his fears by going to Kabul to rescue Sohrab, and thus dispel the ghosts from his own past.