The first one I will look at is the controversy over "Citizen Genet." This was very controversial because it involved both opinions about the French Revolution and nationalistic pride. The US was badly split over whether to support the French Revolution and Genet was trying to make them take sides. He also was seen as insulting Pres. Washington, and people did not like that.
The second is Jay's Treaty. This made many people angry because it seemed to give in to the British and it seemed to be anti-French.
Finally, there is the Embargo Act of Jefferson's presidency. This was controversial because of its impact on the US economy and because of the whole France-GB issue.
Here is a list of foreign policiy events which happened during the Washington, Adams, and Jefferson administrations. I don't know if they were controversial.
Payment to the Babarbary pirates during the Washington Admiinistration.
Raising an Army and building a Navy because of the expected French invasion during the Adams Administration.
War with the Dey of Algiers during the Jefferson Administration.
The Embargo during the Jefferson Administration.
The Louisiana Purchase during the Jefferson Administration.