There were two major defeats or setbacks for FDR in his second term. They were the recession of 1937 and the defeat that he suffered at the hands of Congress in the court-packing episode.
In 1937, FDR cut back sharply on government spending. He felt that the Depression had eased and that it was important to reduce the deficit and stop having the government do so much. This is said to have led to a major slump in the economy. This was a serious setback for FDR and the country.
Also in his second term, FDR tried to "pack" the Supreme Court with justices who would be sympathetic to his policies. This attempt was rejected by Congress because they saw it as a grab for power on the part of the presidency.
These were the major setbacks of FDR's second term.