Linguistics is the scientific study/analysis of a language, but how can one analyse these languages?
There is indeed a number of languages whose linguistics roots and syntactic rules have lost connectivity to modern languages. These languages may not be extinct, but certainly difficult to analyze precisely due to their radical detour from the common rules of syntax that help us encode information the way that we do.
When a language loses this "connectivity" is mainly due to the original, native users of the language decimating, or separating geographically thus diluting the use of it. However, this is what constitutes the "bread and butter" of linguistics: the search and analysis of language formation, usage, and acquisition.
The languages which are currently considered most complex for an English language learner to correlate are Basque, Hungarian, and Chinese. Compared to currently known languages, these three particularly lack the grammatical rules of syntax which enable the process of encoding information. However this is also true of Arabic language families, as well as of the Japanese, Korean, and Russian languages, among others.
Therefore what is considered "hard" or complex in a language family is its lack of connection to other language families, the lack of grammatical rules, and the inability to find a way to transfer meaning. This is what has made some languages become "extinct" through time, or merely so unique to its native users that it is only used by a specific group. Think Romany, Catalan, or even Gaeilge, which are spoken by groups of a strong cultural and geographical connection to the areas in which the language originates.