Romeo kills Tybalt right after he marries Juliet in Act III. He is now family to Tybalt and does not want to fight him, but he can't tell anyone why. Romeo wasn't ever much of a fighter in the first place, and, due to the Prince's ultimatum from Act I, there are many reasons why Romeo wouldn't want to fight Tybalt. Still, Mercutio engages Tybalt in a duel after Tybalt hurls insult after insult at Romeo. In an effort to stop the fight between Tybalt and Mercutio, Romeo comes between them. Mercutio is fatally wounded because of the interference. A few of Mercutio's last words are directed at Romeo:

"Why the devil came you between us? I was

hurt under your arm" (III.1.99-100).

"A plague o' both your houses!

They have made worms' meat of me. I have it,

And soundly too. Your houses!" (III.1.103-105).

Once Mercutio dies, Romeo feels guilty for having come between his friend and Tybalt in the duel, but he also feels guilty that Mercutio was fighting in his stead. In fact, in a moment of reflection, Romeo thinks he would have been in the right mind to fight his own battle against Tybalt if he weren't so in love with Juliet. Romeo realizes this when saying the following:

"My very friend, that got this mortal hurt

in my behalf; my reputation stain'd

With Tybalt's slander,—Tybalt, that an hour

Hath been my cousin! O sweet Juliet,

Thy beauty hath made me effeminate

And in my temper soft'ned valour's steel! (III.1.107-112).

Romeo believes loving Juliet makes him less of a man to the point that he didn't stand up for himself. Romeo believes Mercutio's death is his fault. His goal then changes from keeping the peace to avenging his best friend. It's as if Mercutio's death awakens a rage inside of Romeo that is unlike his personality and he forgets himself. One might say Romeo suffers from the traumatic event by going crazy for a time. As all reason seems to exit Romeo's mind, he attacks Tybalt with great ferocity. Therefore, Romeo is motivated by his own guilt and Mercutio's death when he goes after Tybalt to kill him.

Romeo's motivation in killing Tybalt is to defend the honor of Mercutio. Ironically, it was Romeo's jumping in between the swordplay of Tybalt and Mercutio that indirectly led to Mercutio's death. Had Romeo not been in the way, Mercutio would have likely been able to see and avoid Tybalt's blade.

Romeo, blind with rage, screams at Tybalt:

Now, Tybalt, take the villain back again,
That late thou gavest me; for Mercutio's soul
Is but a little way above our heads,
Staying for thine to keep him company:
Either thou, or I, or both, must go with him.

Here, Romeo is hellbent on ensuring that Mercutio's soul does not travel alone. However, Romeo never pauses to think of the repurcussions of killing Tybalt, who is Juliet's cousin, until it is too late when he excalims:

O, I am fortune's fool!

This is the play's climax when Romeo realizes that he has just murdered the cousin of Juliet, whom he has just married.

Tybalt kills Mercutio, though it is in part, accidental. Tybalt and Mercutio are playing around, but it is a dangerous game they're playing. They're sword fighting, with "live" swords, swords that aren't blunted or tipped. There's anymosity between the two, but they're not really trying to kill each other.

Romeo, in an attempt to make peace and get them to stop fighting, gets in between them and inadvertantly causes a thrust by Tybalt to be missed by Mercutio, and the thrust mortally wounds Mercutio.

Romeo avenges Mercutio and kills Tybalt. The consequence is the banishment of Romeo, which of course leads, eventually, to the tragedy at the conclusion of the play.

Romeo's only motivation for killing Tybalt is the fact that Tybalt has killed Mercutio.  Mercutio was Romeo's good friend and he is enraged and his death.  I, personally, think Romeo feel guilty too because he kind of caused Tybalt to be able to kill Mercutio.

The immediate consequence of this action is that Romeo has to flee from the city or be killed.  The less immediate consequences are more important.  Because he has to flee, he and Juliet both end up dead.

So looking at it like that, it sure seems like a mistake.  He should have let the Prince execute or banish Tybalt.