Macbeth fears Banquo beyond all others, for good reason. First of all, he knows Banquo is brave and capable of acting against him (he "dares" much and has a "dauntless temper of mind"). Then, he knows that Banquo is intelligent enough to get away with whatever he does. He may have the crown, but he does not have security. Banquo's very presence threatens his security, as does the presence of his children. (The wyrd sisters have prophesized that Banquo's children (and grandchildren) will keep the throne). Macbeth may actually begin to hate Banquo at the point where all this hits home: he realizes that he has given his soul for an impermanent temporal prize.