The Prince feels some responsibility for the death of the two lovers because he has allowed the feud to continue. In the very last scene, the Price gives us insight into his thoughts about this very sentiment:
Capulet, Montague, / See what a scourge is laid upon your hate, / That heaven finds means to kill your joys with love! / And I for winking at your discords too, / Have lost a brace of kinsmen. All are punished. (5.3.304-308)
The key to this passage are in the words "winking" and "brace." Winking in this case means "to look the other way." Brace (a word more common in Shakespeare's time, but still available in dictionaries today) in this case means "two of a kind" or "an important pair." The Prince, then, is chiding himself for looking the other way as the pointless feud between the Capulets and the Montagues continued. Meanwhile, the Prince has lost Mercutio, Tybalt, Paris, Romeo, Juliet, and probably countless others to the fray. In this way, "all are punished."
In regards to Romeo and Paris, there can be no better honor than to place Paris in the tomb of Juliet at his request. This was Paris' dying wish, and Romeo obliges him. In fact, Romeo goes even further:
I'll bury thee in a triumphant grave. / A grave? O, no, a lantern, slaughtered youth, / For here lies Juliet, and her beauty makes / This vault a feasting presence full of light. (5.3.83-86)
Even at the point of death, Romeo is still praising Juliet's beauty! Here he calls her grave a lantern: a beautiful metaphor in honor of his love.