As mentioned previously, Baba is Hassan's father, making Amir Hassan's half-brother. As a child, Amir is jealous of Baba's affection toward Hassan. Unaware that Hassan is his half-brother, Amir believes he is more deserving of Baba's attention. He does not understand why Baba has so much affection for Hassan, who he believes is merely the son of Baba's closest friend.
In chapter 18, Amir learns that Hassan is his half-brother, which changes his entire perception of his childhood. Looking back, Baba's actions do reflect the secret that he carried, such as him having paid for the expensive surgery to fix Hassan's lip and his emotional devastation when Ali and Hassan left the house.
After growing up believing he had nothing in common with his father, it occurs to Amir that they were similar in a monumental way—having both betrayed their best friends. Not only did Amir allow Hassan to be raped in order to obtain the kite from Assef; in order to please Baba by winning the kite, he betrays another family member. Similarly, Baba accuses Hassan of stealing, which deeply offends Ali.