Little Alex loves Beethoven. Although Alex is a thorough-going sociopath in every other respect, having no regard or empathy for the people he terrorizes, gang rapes, beats, and kills for his amusement, his adoration of Beethoven's music is the one light in his dark mind. He even feels a glimmer of identification with other people who like Beethoven, which, for Little Alex, is remarkable.
Ironically, when Little Alex is being reconditioned using the Ludvico method that causes him to become physically ill at the thought of violence, Beethoven plays in the background as he is watching films of the concentration camps and other atrocities. Therefore, Little Alex ends up becoming ill at the sound of Beethoven as well as the idea of violence.
Beethoven symbolizes the humanity within Little Alex. Taking it from him while conditioning him shows how forcing people into conformity can rob them of their humanity and, therefore, dehumanize them. A parallel can be found in the novel Brave New World, of which Burgess was well aware, in which people are conditioned into a "happy" conformity but at the price of any knowledge of artists like Shakespeare or Beethoven.