In terms of ethics, the most flawed character is certainly Claudius. Claudius has killed Hamlet's father and married his mother Gertrude. Claudius clearly killed the king to become king himself. Claudius is motivated by greed and power and will remove anyone that gets in his way. When Hamlet becomes a concern for him, both as an attacker and as one who could take the throne, Claudius makes plans to have Hamlet killed as well. Claudius might actually love Gertrude but this is never a central issue in his motivations. He is guided by his own self interests.
Rosencrantz and Guildenstern are flawed in that they conspire with Claudius to spy on Hamlet. They are hypocrites but hardly as flawed, ethically speaking, as Claudius.
Hamlet is flawed as well, but his flaws are more complex. In terms of ethics, one could argue that his "madness as method" is flawed because of how his behavior alienates Ophelia. Hamlet is so upset with Gertrude that he impugns women altogether. His poor treatment of Ophelia leads to her suicide. But Hamlet's flaw that is most written about is his procrastination. This flaw could also be described as his strength. (This is why Hamlet's flaws are more complex than Claudius'.) Hamlet is philosophical. He tries to weigh all the pros and cons of any action of revenge he might take on Claudius. His analyzes and thinks over things so much that he delays killing Claudius multiple times. This is a strength because it shows how intelligent and thoughtful Hamlet is. But even Hamlet is aware of this as a flaw and he chastises himself for delaying revenge:
Why, what an ass I am! This is most brave,
That I, son of a dear father murder'd,
Prompted to my revenge by heaven and hell,
Must, like a whore, unpack my heart with words,
And fall a-cursing like a very drab,
A scullion! (I.ii.556-61)
Hamlet is upset with himself because he has done nothing but think and talk. He earlier calls himself a coward for not showing some action.
His flaw of procrastination does become self-indulgent. When he comes up with the plan to expose the king's guilt with the "play within the play" he intends to expose Claudius in the most dramatic way possible. One could argue that Hamlet delayed things in order to make a show of it, thereby establishing himself as the hero and judge of the crime.
Those suppositions being said, Claudius is the most ethically flawed character. And these flaws create the setting and eventual start of the plot (Hamlet's revenge). Hamlet's flaws keep the play going; or rather, they keep putting off the resolution of the play. This is interesting because whereas Claudius' crime (flaw) initiates the story, Hamlet's flaw (procrastination) delays its end.