In Gulliver's Travels, Dr. Lemuel Gulliver makes four voyages, is ship-wrecked and arrives at four different lands, the most well-known of which is the Land of Lilliput where all the inhabitants are tiny, about six inches tall for an adult male, so Gulliver is a giant in their midst.
Aside from being small, they exhibit to a great extent smallness of minds and morals. When we now use the word "lilliputian," it represents someone who is intellectually or morally defective. Among other criticisms that Gulliver makes while among the Lilliputians, for example, is that the Emperor of Lilliput chooses his ministers not by their innate abilities but by their ability to walk a tightrope. Swift is making a veiled comment on the court of King George I, but he gets away with it because the criticism occurs in the context of a child's story.
In another instance, a fire occurs near the Empresses quarters in the castle, and the only way to put it out and save everyone is for Gulliver to urinate on the fire, which he does. The act of saving the Empress and her retainers is, however, considered to be the ultimate insult to the Empress, and Gulliver's reputation among the Lilliputians is forever damaged and, in fact, his life is in jeopardy from that point until he escapes from Lilliput.
So, you are absolutely correct in saying that they are short--very, very short--but the important aspect of their size is that they are mentally and spiritually "short" as well as physically short.