Shakespeare's Hamlet is set in Denmark in the Middle Ages, at some time between 1100–1500 CE. Until about 1660, Danish kings were elected. In actual practice, the eldest son of the reigning king usually was elected King, but it wasn't until after 1660 that the throne of Denmark was formally made hereditary.
When Hamlet's father dies, or, more correctly, when Hamlet's father is murdered by Hamlet's uncle, Claudius—although the Danish people didn't know that—Hamlet is a university student in Wittenberg, Germany.
Denmark is under threat of invasion from Fortinbras, the son of the former King of Norway, who was killed in battle by Hamlet's father. Young Fortinbras has vowed to recover all of the territory that Norway lost after his father was killed, and he's rattling his saber at Denmark's border.
Claudius no doubt fought with Hamlet's father in the war against Old Fortinbras and perhaps even showed himself to be a trusted and able commander.
Even though Hamlet would traditionally be elected King, Denmark is in a perilous situation that requires a steady and experienced leader, and, if it becomes necessary, an experienced warrior.
When Hamlet's father died, Hamlet—a student, inexperienced and unproven in statesmanship or war—is over five hundred miles from Denmark, and Claudius—brother of the dead King and an experienced leader and warrior—was in Denmark, probably already living in Elsinore Castle, and ready, willing, and able to take control of the country.
It was an easy decision for the Danish people to make, and Claudius was elected King. To reinforce his claim to the throne, Claudius married his brother's wife and former Queen, Gertrude.
It's interesting to note that at no time during the play does Hamlet challenge Claudius's right to be King, even after Hamlet learns the truth about his father's death. Hamlet challenges Claudius only on moral grounds.
In act 1, scene 2, Claudius is shown already attending to affairs of state and taking steps to ameliorate the rising conflict with young Fortinbras. Claudius seems notably unimpressed with the young man's saber rattling. Rather than confront Fortinbras and risk war, Claudius simply goes over Fortinbras's head and sends emissaries to Fortinbras's uncle in order to resolve the matter peaceably.
Claudius then turns to Laertes and graciously permits him to return to France, from where Laertes willingly came to Denmark, "To show [his] duty in [Claudius's] coronation..." (1.2.55).
All that remains is for Claudius to practice "tough love" and call a petulant Hamlet to task for what appears to Claudius to be "impious stubbornness" and "unmanly grief" (1.2.97) in mourning his father's death.
If this scene is any indication of future behavior—and if the ghost of Hamlet's father hadn't made an appearance in the play and thoroughly upset the status quo—Claudius might well have proven to be a strong and capable King.