How is Macbeth's decision to kill Macduff's family different from his decisions to murder Duncan and Banquo?
In Shakespeare's Macbeth, the murders of Duncan, Banquo, and the ordered murder of Fleance (who is a child) serve strategical purposes. Duncan is killed so Macbeth can take his place, Banquo knows about the witches' predictions so therefore is a threat to Macbeth, and Fleance is in line for the throne, if the witches' predictions are to be believed.
The murder of Macduff's family has no strategic purpose. Macduff is not in line for the throne, or at least there are numerous people in front of him, and much less are any of his heirs. Macbeth is even assured in Act 4.1 that Macduff is no threat to him, though the messages are contradictory--one message is that he should beware of Macduff, while another tells him no man born of woman can harm him. And Macduff is safe in England, where Macbeth can't get at him. Again, no strategic purpose exists.
When Macbeth orders the murder of Macduff's family, he is lashing out, throwing a horrific temper tantrum, if you will. He can't get at Macduff, so he kills his family instead. Psychologically, he is getting at Macduff the only way he can.
This makes these murders even more terrible and more horrific than the murders of Duncan, Banquo, and the ordered murder of Fleance.
When Macbeth kills King Duncan, his purpose is to take the throne for himself. He wants to be king, and Duncan is the only obstacle in his way. Similarly, when Macbeth kills Banquo, his purpose is to protect his crown. As we learned in act I, scene III, Banquo's sons are destined to be kings. In order to prevent this from happening, Macbeth hires assassins who kill Banquo and try (but fail) to kill Fleance.
In contrast, the murder of Macduff's family serves no purpose. In act IV, scene I, it is made clear to Macbeth that Macduff poses no threat to his crown. The second apparition tells Macbeth that "none of woman born" can harm him, leading Macbeth to declare that he does not fear Macduff:
Then live, Macduff. What need I fear of thee?
The only reason that Macbeth kills Macduff's family is because he can. Instead of having a political purpose, like the other murders, it is rooted in Macbeth's growing tyranny.
The murder of Macduff's family is, therefore, a turning point for Macbeth, demonstrating that he is becoming increasingly paranoid and increasingly cruel.
I would argue that this decision differs from the others in that it is much more brutal and much less justifiable. Killing Duncan and (trying to) kill Banquo somehow does not seem so bad.
When Macbeth makes the decisions to kill the two men, it seems more okay to me for a couple of reasons. First, they are grown men. Second, killing them has a direct bearing on Macbeth being king. By contrast, killing Macduff's family seems much more horrible for the opposite reasons. These are women and children and their lives do not directly affect Macbeth's chances. He is killing them to protect a dynasty -- so that they will not be kings after him.
So it really seems to me to be much less "necessary" to kill them -- it shows that he's getting more bloodthirsty and power hungry.
The killing of King Duncan was Macbeth's 'original sin'. The thought of murdering Duncan pre-existed in Macbeth's mind. The prophecy of the 3rd witch on the heath only enhanced his ambition to translate the thought into reality. Even then he dwindled a lot within himself before being goaded to the crime by his wife. Macbeth killed Duncan to realise his ambition to be on the throne of Scotland. He was very substantially assisted by Lady Macbeth in this foul mission.
Banquo was killed primarily because he was an eye-witness to what happened to Macbeth on the heath. Furthermore, Macbeth, suffering from a sense of guilt and inferiority born of his heinous crime, was feeling insecure and undervalued as long as Banquo was there. There was also the fear of losing his kingship to Banquo's son, Fleance. Macbeth contemplated the kilings of Banquo and Fleance all by himself, taking the services of two murders. This time, Macbeth needed no support of Lady Macbeth.
Macbeth's decision to surprise the castle of Macduff and to wipe out the entire family of Macduff was a desperate bid of self-defence, for Macduff was clearly opposing the usurper king. The way Macduff's wife, son and all other relations were butchered, it seems clear that fear-stricken Macbeth was at the nethermost level of moral alienation and degeneration. Of the three murders, the murder of Macduff's family was the most horrific and the most unfeeling act of mad brutality.