I think that Banquo is referring to both his own ambitions and his fears about Macbeth when he speaks of his "cursed thoughts." At the beginning of Act 3, Banquo thinks about the witches' prophecy having come true for Macbeth, and he wonders whether or not it will come true for him as well, and if so, when? Yet, at the same time he also thinks that Macbeth has done evil things to ensure that the prophecy would come true for him. Banquo is tormented by these conflicting thoughts because he feels that gathering up his hopes is unfair, especially if it will lead to evil deeds. He banishes these thoughts from his mind as the other characters enter the scene.