Macduff goes with Malcolm to England to get military support to reclaim the throne from Macbeth. Lennox does not believe Duncan was murdered by his sons and clearly, Macduff is thinking the same thing. I don’t think it is clear that Macduff realizes the implications of his not attending Macbeth’s banquet. It is clear that Macbeth takes this rebuff as Macduff opposing his right to the throne. Also, Macduff cannot know what the weird sisters have said to Macbeth (the first apparition says beware Macduff); so Macduff can’t know just how seriously Macbeth sees him as a threat. Still, I can see how it makes much more sense for Macduff to take his family with him. But I think this is something he could only see in hindsight. Also, no one except Lady Macbeth was privy to Macbeth’s growing insecurities and madness and so no one but her (sans the weird sisters/witches) could foresee Macbeth’s subsequent killing spree.