Let us remember that they didn't actually do an incredibly good job of it. The first crime, that of regicide or the killing of Duncan, was actually only possible to cover up through the killing by Macbeth of the two grooms that he and his wife had framed for the murder by smearing them with Duncan's blood. This of course, if we examine Act II scene 3 very carefully, makes Macduff very suspicious, as his question of "Wherefore did you so?" makes absolutely clear when he asks why it was Macbeth killed the grooms. Of course, it is just chance that the two sons of Duncan decide to flee, giving Macbeth the perfect opportunity to blame them conveniently, as they are not present to defend themselves.
If we then think about the next crime, the killing of Banquo, the inconvenient appearing of his ghost during the banquet scene of Act III scene 4 really threatens Macbeth's secrecy, and clearly makes his lords very suspicious about his sanity, but also his involvement in the crimes. If we look carefully at Act III scene 6, Lennox makes it clear that he is very suspicious of the "convenient" way in which things have fallen in place for Macbeth:
The gracious Duncan
Was pitied of Macbeth:--marry, he was dead:--
And the right-valiant Banquo walk'd too late;
Whom, you may say (if't please you) Fleance kill'd,
For Fleance fled. Men must not walk too late.
The sarcasm and suspicion in his voice is evident. Of course, after this, Macbeth moves from trying to commit crime and treachery in secret to the open slaughter of Macduff's family, ending any pretense of goodness whatsoever. So, all in all, we could argue that the efforts that Lady Macbeth and Macbeth made to cover up their treachery were not actually that effective, as Macbeth's suspicious killing of the henchman does not go unnoticed, and the ghost of Banquo reduces him to near-madness.