Most of these presidents stayed out of war because they believed the U.S. was not strong enough militarily to engage in a European war, neither did the U.S. stand to benefit from engaging in such a war.
Although he received pressure from both Thomas Jefferson and Alexander Hamilton to intervene in the Anglo-French wars of the day, George Washington issued his famous Proclamation of Neutrality. In it he avoided use of the word neutral but rather stated that the U.S. would remain "friendly and impartial toward the belligerent powers." When the French Ambassador, Charles Genet, attempted to stir up support for France and even attempted to lead a revolt against Washington, the President threatened to expell him from the country. Genet, in turn, quieted down and remained in the U.S. the rest of his life.
The situation grew more difficult for all three Presidents as U.S. merchants continued to trade with the belligerents and both sides interfered with American shipping, France through Napoleon's Continental System and Great Britain through its Orders in Council. Both were intended to prevent presumed neutral powers from aiding the other side.
Under the Adams administration, war with France seemed inevitable after the XYZ affair; however Adams wanted France to be the instigator if war were declared. When issues with France were resolved, Alexander Hamilton opposed the deal, as he hoped to capture Louisiana and feed his own ego by military glory. Adams threatened to resign if the terms of the treaty were not accepted.
When Americans continued to trade with belligerents during the Jefferson Administration, Jefferson imposed an embargo which prohibited trade outside the U.S. This did not hurt the warring powers, but did cripple American shipping.
Jefferson's successor, James Madison was ultimately unsuccessful in avoiding war, and was President when war against Britain was declared in 1812. The ostensible cause of the war was the British practice of Impressment in which presumed British deserters were removed from American ships and forced to sail (and fight) on British vessels. However, all efforts to prevent war were ultimately unsuccessful because Americans persisted in trading with the belligerent powers. Interference by Britain and France were obvious factors, but private attempts by U.S. citizens to profit from the war ultimately led the U.S. into war. A lesser element was the determination of the war hawks in Congress, including Henry Clay ahnd John C. Calhoun, to make Canada a part of the U.S., a dream that was never close to realization.