Wittgenstein said "Of that which we cannot speak we must remain silent." The implication is that in order to "convey" thought we must devise a shared "sign" system with the receiver, be it linguistic, visual, tactile, or what have you. The truth is that what we think of as "thoughts" are our own reactions, emotions, response, etc., and our minds need not put any of those things into a language until we want to convey them. What seem like words in our minds are the instant spontaneous translations from sensations to words, a habit of mind and a mental process that takes place when we learn language, perhaps as early as one or two. To say that a baby is not feeling anything because it cannot put it in words (even in his/her own mind) is nonsense. "Those without language" is a vague term that needs defining.
Curious about this concept, I decided to research the idea. A doctor of linguistics, Arika Okrent, defines the ability to think without the possession of language. This said, she also submits to the fact that there are certain concepts which are lost to those without language.
Okrent's findings suggest that one must first define what is meant by the term "thought." If one defines thought as an experience, everyone experiences life (in one way or another). Simply, even though one cannot express through language something does not mean he or she is unable to think.
For example, while this uses a fiction text, the Creature in Mary Shelley's Frankenstein does not possess language when he first comes to life. He is able to experience pain and happiness. He understands that when he touches fire that it causes pain, and he understands when he hears birds sing he is happy. This, especially the pain, relies upon a person's internal survival instincts. It would be a person's natural reaction to pull his or her hand out of a fire because of the pain it caused.
What this shows is that humans do not need language to experience life and think. Refer back to the example above, the Creature (or any human) would learn that fire hurts. They would then know (think) not to touch it again. The thinking process exists without the possession of language attached.