The music of the jazz age (jazz) perfectly embodies much of what "The Great Gatsby" is about.
Jay Gatsby and the other characters who live in the West Egg represent the new, young money. They are forging their own path and at times are throwing off the conventions of previous decades in order to make their money or be individuals during this time. These West Egg residents are carefree and risk takers. One such risk taker is Jay Gatsby himself. He made his money through illegal bootlegging, and doesn't think Daisy's marriage vows should stand in the way of his affections for her.
Jay throws opulent, over the top parties in an effort to win her affections. Wild parties for people who are young, reckless, and recently into some new money need some kind of music to reflect all of that. Jazz fit that bill. irst it was new. Second it was slightly rebellious because much of the time it was played and heard inside of "speakeasies" (illegal bars). Third, the music itself was quite individualistic. It supported improvisation and produced very distinctive styles and sounds within itself (Miles Davis for example).
Fitzgerald could have left out any reference to music in the novel, but by including the jazz references, it allowed him to better encompass the feelings of the period and characters.