The Temple veil separated the Holy Place of the Temple, where the people worshipped, from the Sanctum Sanctorum, or Holy of Holies, which contained the original tablets of the Ten Commandments and was believed to be the earthly dwelling place of God. Because man was sinful, he could not approach God himself. The Prophet Isaiah illustrated the point:
Surely the arm of the LORD is not too short to save,
nor his ear too dull to hear.
But your iniquities have separated
you from your God;
your sins have hidden his face from you,
so that he will not hear.
The High Priest was allowed to enter the area once per year to offer a sacrifice for atonement; however any other person who entered was immediately struck dead. In fact, tradition holds that the High Priest entered the area with a rope tied around his waist so that, in the event he did something wrong and was struck dead, his assistants could remove him safely.
The veil was constructed of material so thick that two teams of horses could not pull it apart. It marked the point of separation between a Holy God and sinful man. However, Jesus constituted the perfect sacrifice for sin. His death on the cross meant there was no longer any need for this separation to exist. The thickness of the veil indicates that no human hands could have torn it. So the tearing of the veil was God's way of saying that man need no longer be separated from Him or fear His presence.