Why does Prince Escalus blame himself for the tragedy in Romeo and Juliet?

Prince Escalus is a ruler who believes in a top-down hierarchy, as most people in Shakespeare's era did (see Tillyard's The Elizabethan World Picture). Therefore, he thinks his actions are vitally important for setting the boundaries of what is or is not acceptable in Verona.

At the beginning of the play, Escalus tells the feuding families that "three civil brawls, bred of an airy word ... have thrice disturb'd the quiet of our streets ..." indicating that he has allowed the fighting to get out of hand. Now he says, working to restore order, "your lives shall pay the forfeit." He essentially says he is tired of fooling around and the fighting has to stop or heads will roll.

However, Escalus, a wise and what Shakespeare might call a "meek" ruler, meaning merciful, clearly doesn't want to be passing death sentences on members of the feuding families: for instance, he banishes rather than executes Romeo for killing Tybalt. Tybalt is a hothead, but the fact that he is ready to go back to fighting a day after Escalus has made his threat to execute anyone who feuds means that Escalus's threat is probably not being taken completely seriously—and for good reason, as Escalus backtracks with Romeo one day after his strong words. Escalus may be reminiscent of Vincentio, the Duke of Vienna in Measure for Measure, who has to leave the city in the hands of Angelo to try to restore order after his lax rule.

Escalus says at the end of the play that he has been "winking" at the feud, indicating once again that he is aware that he has not taken a strong enough hand in stopping the fighting. In this play, however, the shock of the young lovers' deaths is enough to restore the peace.

Soon after his arrival at the Capulet tomb with the discovery of the tragedy, Prince Escalus, on having listened to Friar Laurence's testimony, addresses the Capulets and Montagues expressing his disappointment. During his speech, he says the following:

"And I for winking at your discords too
Have lost a brace of kinsmen: all are punish'd."

It is not so much that the Prince blames himself for the tragedy, but he does take responsibility for not having done enough to stop the feud between the families. He says that he was only "winking" at their strife. This suggests that he feels that he had temporarily closed his eyes to their battle, choosing to, at most, ignore the ongoing dispute. The Prince essentially puts forward that he could have done more and that, if he had done so, the present tragedy would not have occurred.

He furthermore proposes that, because of his reasonably dismissive attitude, he had also suffered loss. Two of his kinsmen (Mercutio and Paris) have died. A firmer and much more hard-handed approach to end the quarrel between the two warring factions, might have prevented this unnecessary tragedy. 

One can sympathize with the Prince, for on the whole, he had intervened and had promised both families severe sanctions if they should indulge their whims and strike out at each other. At the beginning of the play (Act 1 scene 1) he had warned both families: 

"If ever you disturb our streets again,
Your lives shall pay the forfeit of the peace."

And again, after both Tybalt and Mercutio were killed in battle, he banished Romeo and warned:

"But I'll amerce you with so strong a fine
That you shall all repent the loss of mine"  and

"I will be deaf to pleading and excuses;
Nor tears nor prayers shall purchase out abuses:
Therefore use none: let Romeo hence in haste,
Else, when he's found, that hour is his last."

He had clearly taken a tough stance, all to no avail.

He says that he turned a blind eye to the severity of the quarrel between the Capulets and the Montagues.  Had he nipped it in the bud - had he imposed large fines or greater punishment on those involved or outlawed the feud altogether - he thinks he could have prevented an increase in hatred between the two families.  Thus, Romeo and Juliet may have had permission to fall in love, or at least not felt that death was their only option.

How much is Prince Escalus of Verona to blame for the events in Romeo and Juliet?

In the Prologue of Shakespeare's Romeo and Juliet, the Chorus tells the audience that two distinguished households of Verona have a long-standing feud and that the enmity between the families has again erupted into violence.

CHORUS. Two households, both alike in dignity,
In fair Verona, where we lay our scene,
From ancient grudge break to new mutiny...(Pro., 1–3)

When Prince Escalus, the prince of Verona, first appears in Shakespeare's Romeo and Juliet, members of these two household—the Montagues and Capulets—are brawling in the streets of Verona.

At first, the combatants ignore Escalus.

PRINCE. Rebellious subjects, enemies to peace,
Profaners of this neighbour-stained steel—
Will they not hear? (1.1.77–79)

When Escalus finally gets their attention, he demands that they throw down their weapons and listen to what he has to say. He tells the people that this is the third such brawl that has recently occurred, and anyone involved in another such brawl will be executed.

PRINCE. ...Gentlemen, for shame! forbear this outrage!
Tybalt, Mercutio, the Prince expressly hath
Forbid this bandying in Verona streets....

If ever you disturb our streets again,
Your lives shall pay the forfeit of the peace. (1.1.85–87, 92–93)

Escalus doesn't say whether he had to break up the other brawls himself, but it's apparent that he doesn't command much respect or obedience from his subjects. Escalus can't even get his people's attention when he first appears on the scene, and it becomes clear through the rest of the play that his threats to execute those who violate his laws are empty.

In act 3, scene 1, Escalus's threat of death doesn't dissuade Mercutio and Tybalt from fighting in the streets.

ROMEO. ...Gentlemen, for shame! forbear this outrage!
Tybalt, Mercutio, the Prince expressly hath
Forbid this bandying in Verona streets... (3.1.85–87)

Mercutio is killed by Tybalt, and Romeo kills Tybalt for killing Mercutio.

When Escalus again appears on the scene, he fails to condemn anyone to death for fighting in the street, but instead banishes Romeo and issues a fine to deter future outbreaks of violence.

PRINCE. And for that offence [Romeo killing Tybalt]
Immediately we do exile him [Romeo] hence...

But I'll amerce you with so strong a fine
That you shall all repent the loss of mine. (3.1.192–193, 196–197)

In the next line, Escalus says, "I will be deaf to pleading and excuses; / Nor tears nor prayers shall purchase out abuses," but he has already deferred to Benvolio and Lord Montague in banishing Romeo instead of sentencing him to death.

In the last line of the scene, Escalus himself sums up his own ineffectiveness and the dire consequences of his tolerance towards wrongdoers.

PRINCE. Mercy but murders, pardoning those that kill. (3.1.203)

Escalus's lax enforcement of his own laws has resulted in the death of two citizens—including Mercutio, one of his own relatives—and the banishment of Romeo.

Because Escalus has not been able to control the feud between the Capulets and the Montagues, Romeo and Juliet must keep their romance and their marriage secret, which ultimately results in their deaths.

After Romeo and Juliet are found dead in the Capulet's tomb, Escalus tries to lay the blame for their deaths on the Capulets and the Montagues, but he finally realizes his own responsibility for their deaths.

PRINCE. ... Where be these enemies? Capulet, Montage,
See what a scourge is laid upon your hate,
That heaven finds means to kill your joys with love!
And I, for winking at you, discords too,
Have lost a brace of kinsmen. All are punish'd. (5.3.302–306)