This is a very good question. The text does not provide any answer. We know that Banquo was very suspicious of Macbeth, and he may very likely have confided his suspicions to Fleance, warning him to be very wary of Macbeth, especially since the three witches had prophesied that Banquo's heirs would be the future kings of Scotland. Fleance could have deduced that since the murderers were waiting to ambush him and his father only one mile from Macbeth's castle, it was Macbeth who had set the trap. At the end of Act 3, Scene 3, Banquo calls:
"O treachery! Fly, good Fleance, fly, fly, fly! / Thou mayst revenge--O slave!"
Banquo certainly wouldn't have advised his son to fly to Macbeth's castle. Fleance must have understood that his father wanted him to fly, fly, fly far away, and stay safe until he had time to grow to manhood and then seek revenge. In fact, the word "fly" seems to imply that Banquo wants his son to get as far away as possible.