To answer this, we must look first at the definitions and relative positions of good and evil. God, as the Supreme Being, is what is known as the Ultimate or Highest Good and the source of all goodness. Also, as the Supreme Being, He is the Ultimate Cause of everything; everything that is, He brought into existence. Now, since He is good, all that comes from Him is good, including the existence which He gives to all of creation. It is because of this fact that we are able to say that whatever is, is good, at least insofar as it has existence.
If existence is good, and if evil is not good, evil cannot have existence or being in the sense we spoke of it above. Evil is, rather, an absence of good. That is all. When we do something evil, what we are assigning the word "evil" to is not actually an entity but rather a lack of goodness.
With this philosophical basis, we can understand then where the evil of the devil came from. He was created good but with a free will. He could choose good (which is God) or not good (which we call evil). In choosing evil, he merely rejected good.
If you want to take the question into the matter of why God allowed him to choose evil, we can answer this by an explanation of God's love. God, being self-aware, of course knows that He is the fullness of goodness. He knows that only goodness will satisfy and make happy all His creatures. However, He loves them so perfectly that He wants to give them the dignity of knowing both possibilities--Him and not Him, good and not good--and deciding for themselves. This is why He allows free will to His higher creatures, men and angels. They can choose Him (Goodness), or they can reject Him.