Some of the answer to your question depends upon who you define as being the tellers of the rumors. Gatsby himself constructed a life history for himself that was based on rumor. He insisted his parents were "some wealthy people in the Middle West," which Nick believes up until he receives the telegram announcing that Henry Gatz is on his way east after learning of his son's death. As Gatsby becomes involved in questionable activities with partners of debatable reputation, he builds and encourages the image that allows him to come closer to realizing his dream of winning Daisy for himself. The tales of incredible adventures and fabulous wealth that all his party-attending friends share among themselves are, in the final analysis, nothing but rumors and wishful thinking.
There are a few mentioned in chapter 3, page 45. These include him being a german spy, being in the American army during the war and that he killed a man. There is also the rumors of him being Oxford eduated.