Language is symbolic in the sense that it consists of a system of arbitrary or conventional signs.
Imagine, for example, that I speak of a "cat." The term cat is a "sign." The sign "cat" can be separated into two components, the noise I make saying the word (or letters on a page), which is a "signifier," and the "signified," or the concept with which the signifier is associated, namely a small furry domestic animal that likes to attack my mouse when I am trying to work.
We understand language as a symbolic system. No one sign is meaningful by itself, but instead each sign accumulates meaning in terms of its similarities and differences to other signs. For example, we might understand "tabby," "Siamese," and "Maine coon" as types of "cat," while "lynx," "bobcat," and "dog" are not domestic cats and thus by thinking about them we clarify the demarcations of our concept of "cat." These demarcations are arbitrary (one could imagine a language that did not distinguish among domestic cat, bobcat, and lynx, for example, or that assigned different signifiers to long- and short-haired cats).
Each language has different conceptual demarcations. Thus when we look at how kinship terms, for example, differ from one language to another, we learn something about how cultures differ.
This is a good question, and the answer is that language is symbolic in many ways.
First, language (written or spoken words) is just a symbol of something else. I can write the word heart, for example, but the word itself is not a heart. I can say the word peanut butter, but the word is certainly not edible. Language itself is symbolic of the words we speak and write.
Second, language is a symbol (a representation) of a culture. The words--both acceptable and unacceptable--which comprise a language reflect the culture of those who speak it. If a language has many definitions for a word such as love, for example, that is an indication of perhaps a more formal and precise culture. If a language includes a lot of acceptable slang, on the other hand, that is an indication of a less formal culture. (Sidenote: look at the words whicih have been added to the dictionary in the past few years to see where America is heading.) People think differently, and this is reflected in both an individual's and a culture's language.
Finally, language is often a symbol (a reflection) of intellect. Complex actions and accomplishments generally require communication, and a shared language is generally the most efficient and effective method of communication.
Language is symbolic in literature, of course, but it is also symbolic in other ways.