The actual phrase was taken from the New Testament, specifically the phrase "for the letter killeth, but the spirit giveth life" (2 Corinthians 3:6). In this context, St. Paul was talking about the difference between legalism, which involves interpreting laws in an extremely literal fashion (something he associated with Judaism, especially with the Pharisees), and the spirit of charity, which attempted to look beyond a narrowly literal interpretation of the law to a more spiritual one (something Paul associated with Jesus). In religion, this often is seen as the difference between a religion grounded in charity or love and one focused on ritual. 

In legal theory, this has been applied to following the spirit versus the letter of legal codes and constitutions, especially when those codes are being applied many centuries after they were written. For example, the Second Amendment to the US Constitution enshrines the right to bear arms, but it was written in 1791, a period in which guns could only shoot one bullet at a time and then required reloading. Thus people question whether the spirit of the amendment is focused on the ability to bear single-shot weapons (which can do only limited damage) or whether it includes assault rifles or nuclear weapons, things not invented in 1791. 

Following the letter of the law refers to the idea of keeping your actions technically legal by making sure that they follow the law.  By contrast, following the spirit of the law refers to acting in ways that are consistent with what the law means, not its exact words.

One example of this difference, perhaps, can be seen in laws that can be consistent with the letter of the Constitution but not with its spirit.  (Of course, saying this requires us to specify what we think the spirit of a law is and that is a subjective matter.)  In the early 1960s, there were states that banned the use of contraceptives.  There is nothing in the Constitution that says that people have the right to use contraceptives.  Therefore, such a law follows the letter of the law.  On the other hand, that sort of law seems to violate the spirit of the Constitution.  The spirit of the Constitution seems to be that people should be able to do as they wish with their private lives because we have a limited government and not a totalitarian one.  The Supreme Court ended up ruling that such laws violated the Constitution because they went against its spirit.