In his inaugural address, given on January 20, 1961, Kennedy most famously used the rhetorical device of chiasmus, which is a fancy way of saying that a phrase is inverted when it is repeated. When he said, "ask not what your country can do for you--ask what you can do for your country," he was using this AB-BA pattern, inverting the order of "country" and "you" the second time he used these two words.
In addition, Kennedy used antithesis--putting two opposing ideas next to each other. For example, in his first line, he said, "We observe not a victory of party but a celebration of freedom--symbolizing an end as well as a beginning--signifying renewal as well as change." The proximity of these opposing ideas made it clear that he was calling for a break with the past. He was at once recognizing an ending as well as celebrating a beginning that had a strong connection to the past.
Kennedy also used anaphora, or the repetition or words or phrases. For example, he said, "Let us begin anew...Let us never negotiate out of fear. But let us never fear to negotiate." The repetition of "let us" is an example of anaphora, while the "Let us know never negotiate out of fear. But let us never fear to negotiate" is another example of chiasmus. The repetition of words allowed Kennedy to place emphasis on them.
Finally, Kennedy used a great deal of alliteration, or starting two words that are close together or next to each other with the same sounds. There are many examples of alliteration in his speech, including "to friend and foe" and "a hard and bitter peace, proud.." Using alliteration made his speech more poetic and memorable.