Gandalf left at a few points during story. I assume you mean the one of the greatest length, and that is as the dwarves and Bilbo entered Mirkwood.
Gandalf is very specific when he tells the group that they must not leave the path as the enter Mirkwood because of the great evil that has now spread through much of the forest. In the new to DVD movie version, this danger is made clearer by an added sequence that includes Gandalf's counterpart wizard, Radagast the Brown. Radagast had detected a great evil growing in the forest.
In The Hobbit text itself Tolkien tells us "It appeared that Gandalf had been to a great council of the white wizards, masters of lore and good magic; and that they had at last driven the Necromancer from his dark hold in the south of Mirkwood" (201).
I found additional information in Tolkien: The Illustrated Encyclopedia (1991) by David Day. According to that source, Mirkwood forest used to be called "Greenwood the Great" until the darkness became so powerful there.
The source of that darkness was a being known as "the Necromancer of Dol Guldor." Gandalf later found this to be Sauron the Ring Lord. At that time, many had believed Sauron had been defeated when the ring had been cut from his finger by Isildur, but because the one ring had not been destroyed, Sauron's spirit was able to rise once again. He built the fortress at Dol Guldor with the Ringwraiths and quietly built power there for almost 2000 years when he re-entered Mordor.