Culture originally was a term that applied to farming – to cultivate the land. Culture as it applied to the arts referred to a refinement or cultivation of the mind; and in that context, has developed an elitist connotation. “Culture” also distinguished itself as high art because only high art would be the creation of a refined or cultivated mind but also because the institution of art was often guided or funded by high society – aristocracy, governments and so on. It was until industrialization and later, kitsch, decadent and later surrealist art (and later Andy Warhol) that the notion of high art began to be challenged. So, yes, the word “culture,” in terms of art, has always carried an elitist meaning.
Over the course of the 18th-19th centuries as industrialization increased, there was a growing sense of man’s place in building history and society; machines and cities. It was this growing sense of history being a social or collective process, and later the recognition of art from minority or previously oppressed groups, that broadened the term culture to have a wider definition in terms of art and then as a refinement of the social mind (consciousness) in general – Culture became the mentality of a society.
Culture is still used to refer to artistic ability and to a lesser extent, higher society. For example, a city has culture if there are museums, a variety of music; in other words, things to do besides bars and churches. But today, culture is predominantly used to refer to whole (but interacting) systems of thought: The American culture, the culture of Harry Potter fans, etc.
So, culture can still connote an elitist definition. And culture can refer to the characteristics, beliefs and ways certain groups think. The thing is; once a culture establishes itself as a culture, the ways that culture think come to be solidified or established. Ironically, that culture breeds its own elitism. For example, the hard-core Harry Potter fans might act snobby towards others who are fans but don’t devote their life to it.
- Raymond Williams' chapter on "Culture" in Marxism and Literature is a great and short summary of the development of these ideas.