In act 3, scene 1, Macbeth acknowledges that his "fears in Banquo / Stick deep." At this point, he lists several reasons to account for this fear. Firstly, he says that Banquo has a "royalty of nature," or in other words, something noble about him that makes Macbeth afraid. Banquo also has a "dauntless temper," meaning a willingness to take risks, and also "a wisdom that doth guide his valor." In summary, Macbeth is afraid that Banquo is intelligent enough and brave enough to expose and defeat him.
The first of Macbeth's reasons, that Banquo has something royal and noble about him, links back to a prophecy that the three witches made, namely that Banquo would one day be a "father to a line of kings." Macbeth is, therefore, afraid that all the risks he has taken, and all the immoral acts he has committed to become king, might all be for the ultimate benefit of Banquo and Banquo's sons. Macbeth is afraid that he is essentially damning his own soul more for Banquo's benefit, and more for Banquo's legacy, than for his own.
In act 3, scene 4, after Macbeth has had Banquo murdered, Banquo's ghost appears to Macbeth and the latter's fear increases. He starts to fear that all of his murders might have been for nothing, if "charnel houses and our graves must send / Those that we bury back." Up to this point, Macbeth has been secure in the knowledge that those he has murdered, and those that he is still to murder, have been and can be disposed of forever. Now he is faced with the possibility that the likes of King Duncan will not stay buried but may return to haunt him. Essentially, Macbeth here is afraid that his guilt cannot be buried. This is the terror that Banquo's ghost suggests to him.
Banquo's ghost also makes Macbeth afraid that he shall be punished. He says to Lady Macbeth that "blood will have blood" and that the dead will, from beyond the grave, expose "the secret'st man of blood." In other words, the dead will get their revenge, and the guilty, like Macbeth, will be exposed and brought to justice. When one realizes that these are the thoughts that scare Macbeth, it becomes clear that he has become a coward in a moral sense, afraid to face up to the consequences of his own actions.