Another thing to consider is that it was unelected judges who ruled segregation unconstitutional, and that guaranteed freedom of speech even when it was unpopular. While they may not always rule in a way we like, they keep us honest when it comes to the Constitution, and are not punished by being removed from office for doing so.
The name for this is an "independent judiciary", as in they are independent of what is the popular will, and can concentrate on what is legal instead. As pohnpei states, this is critical to our maintaining a democracy.
In my opinion, an unelected judiciary is essential to having a fully democratic society.
When thinking about this, please keep in mind that there is more to democracy than simply having elections (what we call "popular sovereignty.")
In order to have a true democracy, a country must also protect the rights of minorities from the majority. It would not be democratic, for instance, to have a law voted on by the people that prohibited Catholics from voting.
So who will protect the rights of minorities from hostile majorities? If anyone can, it will be unelected judges.
Now, of course, this causes tension with the idea of majority rule, but that tension needs to exist -- pure majority rule is not democratic.