Macbeth very likely considered Banquo a threat from the moment he heard the prophecy made to Banquo in act 1, scene 3.
THIRD WITCH. Thou shalt get kings, though thou be none. (1.3.70)
Macbeth seems troubled by the prophecy, and it's the first thing he mentions after the witches leave the scene.
MACBETH. Your children shall be kings. (1.3.89)
The prophecy to Banquo is still on Macbeth's mind even after he's told by Ross and Angus that King Duncan has made him Thane of Cawdor.
MACBETH. (To Banquo) Do you not hope your children shall be kings,
When those that gave the Thane of Cawdor to me
Promised no less to them? (1.3.127-129)
Macbeth becomes involved with Lady Macbeth in their plan to murder Duncan, but once Duncan is dead, Macbeth's thoughts turn again to Banquo.
By act 3, scene 1, Macbeth is king, and he believes that the greatest threat to staying on the throne is Banquo. By the time Macbeth meets Banquo in the scene, he's already decided to have Banquo and Fleance killed, and now he's deciding when and where to have them killed.
MACBETH. Ride you this afternoon? ...
BANQUO. Ay, my good lord.
MACBETH. ...Is' far you ride?
BANQUO. As far, my lord, as will fill up the time
’Twixt this and supper. ...
MACBETH. Fail not our feast.
BANQUO. My lord, I will not.
MACBETH. ...Goes Fleance with you?
BANQUO. Ay, my good lord...
MACBETH. I wish your horses swift and sure of foot,
And so I do commend you to their backs.
Macbeth expresses the depth of his fear of Banquo in his soliloquy in that scene.
MACBETH. To be thus [to be king] is nothing,
But to be safely thus. Our fears in Banquo
Stick deep, and in his royalty of nature
Reigns that which would be fear'd. ’Tis much he dares,
And, to that dauntless temper of his mind,
He hath a wisdom that doth guide his valor
To act in safety. (3.1.52-58)
Although Banquo has given no indication that he intends to usurp Macbeth's throne—and the witches made no prophecy to that effect, in fact telling Banquo that he would not be king—Macbeth is nonetheless concerned that if Banquo decides to take his throne, Banquo would easily succeed in doing so.
Macbeth's thoughts then turn to the other part of the prophecy, that Banquo's children will be kings.
MACBETH. ... For Banquo's issue have I filed my mind,
For them the gracious Duncan have I murdered,
Put rancors in the vessel of my peace
Only for them, and mine eternal jewel
Given to the common enemy of man,
To make them kings, the seed of Banquo kings! (3.1.69-74)
As a side note, King James I of England, for whom Shakespeare is thought to have written Macbeth, believed himself to be a descendant of Banquo. He wasn't, because Banquo is a fictional character, but he nevertheless held to the belief.
Macbeth decides to challenge the prophecy and have Banquo and Fleance killed.
MACBETH. Rather than so, come, Fate, into the list,
And champion me to the utterance! (3.1.75-76)
As fate would have it, the murderers enter the scene at just that moment. It's clear from Macbeth's conversation with the murderers that their entrance isn't coincidental. He's already spoken with them about Banquo.
MACBETH. Was it not yesterday we spoke together?
FIRST MURDERER: It was, so please your Highness. (3.1.78-79)
Within minutes, Macbeth convinces the murderers to kill Banquo and Fleance, which wasn't particularly difficult, because they're called "murderers" for a very good reason. Macbeth says that he'll tell them the time and the place for the murders "Within this hour" (3.1.141), and tells them to wait close by for further instructions.
MACBETH. I'll call upon you straight. Abide within.
It is concluded: Banquo, thy soul's flight,
If it find heaven, must find it out tonight. (3.1.155-157)
The murder of Banquo will haunt Macbeth, figuratively and literally, for the rest of his short reign.