The reason they are TAUGHT togther in Western Civilization courses is that they consist of the background used in studies of European and Anglophone culture in subsequent courses. Thus your question really should be framed as one not about inherent qualities of the cultures but rather about the history of curriculum.
Western civilization is best defined as the civilizations that have contributed to and have sprung from Europe.
When we think of Western civilization, we generally think of the set of cultural attitudes and institutions that have developed on that continent. These attitudes and institutions have spread with colonization and so we would also say that the Americas, as well as Australia and New Zealand are generally part of the West.
In order to understand Western civilization, though, we have to also look back at how thise civilizations came to be. Some things that have influenced this civilization have come from outside of Europe. The most obvious example of this is the Judaeo-Christian tradition. Therefore, the study of Western civilization also includes the study of those civilizations (even outside of Europe) that have had a major impact on Europe.
I would say that 'Western Civilisation' could be roughly defined as the amalgamation of the influence of the following...
Ancient Greece, The Roman Empire, Roman Catholicism, The Renaissance, The Reformation, The Enlightentment, The Industrial Revolution, The British Empire, Capitalism, Feminism and The 60s counter-culture.