As you might know, the General Theory of Relativity is basically a geometric theory they tries to describe the properties of a 4 dimensional space (3 spacial dimensions plus one temporal dimension). In this geometric theory a mass (hence gravitation) is warping (curving) the space around it. This warping of the space has long ago (around 1920) been demonstrated by observing that the during a solar eclipse the path of light coming from other stars is curved by the sun when it travels in its proximity. This makes the planets revolving around the sun much like a motorbike is traveling in a circle high on a wall.

What is little known about gravitation is that the General Theory of Relativity also has predicted a second property about it. A mass makes also spin the space (twist or make revolving the space) around it. Experimental evidence for this has been brought by NASA only this decade, although this experiment has been planned about 40 years ago. Imagine you have a spinning wheel on the Earth as in gyroscopes. It will tend to keep the axis of rotation the same direction as long as it spins. Now you move this spinning wheel still spinning high above the Earth. If gravitation makes revolving the space, the axis direction of the spinning wheel will modify. That is exactly what has been observed experimentally.

Now you have the answer to your question. Objects are falling into a black hole following a circular path because gravitation also spins the space around it (not only warps the space).