What does the authour mean by language as a structure?
``grammar taught me language as a structure. I even learned the elementary categories of philosophy from grammar, things like the concrete, the abstract, the particular and the universal``
The author is talking about the ways in which language organizes or structures how words are put together to convey messages. When you want to ask someone to pass you the textbook, you don't say the words in any order that comes to mind; "Textbook me please to the pass" makes no sense because there is a specific order in which words should be presented to be understood. That order is part of the structure of language.
Another aspect of the structure of language is what your quote refers to as the "categories of philosophy." Some parts of language change in response to different types of use; I see, they saw, he has seen, she will be seeing. These differing forms of the same basic verb (to see) are examples of using the structure of language to correctly communicate the message being presented.