Calhoun believed that the states had the right to nullify a federal law because he believed that the Constitution had been a compact between the various states of the Union and the national government. Therefore, any state had the right to interpret the Constitution and to nullify a law that it felt was contrary to the Constitution.
Calhoun believed that the Constitution was made by the states of the Union, not by the people of the Union. This seemingly small difference had major impacts. It meant that any state had the right to judge whether the national government was living up to its part of the contract. The states had given the federal government certain powers and they could not go against it as long as it kept to those powers. But if the federal government overstepped its bounds, the states had the right to withdraw their support for its actions.
Thus, since the Constitution was a compact between the various states and the national government, each state had the right to nullify laws that violated the compact.
One must wonder if the basis for Calhoun's argument might have been the Tenth Amendment, and he might have believed that it modified the Supremacy Clause of the Constitution.