Kennedy's most prominent failure would have to be not supporting the Cuban rebels who tried to overthrow the Castro regime during the Bay of Pigs invasion. Kennedy did not provide air support to this CIA mission and the mission's disaster led to a loss of American prestige and it strengthened Castro as a leader of the people who stood against American aggression. It was not long after this that Khrushchev stored nuclear missiles on the island in the Cuban Missile Crisis. While that would be Kennedy's best moment as president, the Bay of Pigs fiasco should have either been supported or called off altogether. Kennedy also allowed America to stay in Vietnam by authorizing more advisers and special forces groups to enter the country in a war that would ultimately tarnish the Democratic party for the next twenty years.
Kennedy's greatest failure domestically would be his slowness to get involved in the civil rights movement. It was only after the televised riots that took place in Alabama did Kennedy decide to look into legislating on civil rights. While Kennedy did not share the feelings of segregationists, he also did not want to alienate Southern Democratic congressmen. Kennedy ultimately came around on desegregation and was about to push legislation through Congress before his untimely death in 1963. After Kennedy's assassination, Johnson was able to pass the Civil Rights Act of 1964 by using Kennedy's legacy. The Kennedy administration is hard to judge since a lot of it is thought of in terms of what could have happened instead of what actually happened.