The rock will continue along the path that it had at the moment the string broke.
Newton's laws indicate that an object in motion tends to stay in motion, unless acted upon by another force. We could reverse this concept to say that a force acts to change the motion of a body. In the case of circular motion, the rock wants to travel in a straight line, and a constant application of force is necessary to constantly change the rock's direction (into what appears to be a circle, although any other path would be equally viable).
We know that force is being constantly applied because there is tension in the string, indicating that it is acting and reacting with the rock, and the fact that the rock must be continuously spun in order for the circular motion to be maintained - otherwise the rock will stop moving.
At any point in the rock's path, we could draw a free-body diagram and observe the elements acting upon it; for example, the centripetal force directed along the axis of the string, the gravitational force directed downward, as well as the rock's angular (circular) velocity and its tangential (instantaneous) velocity.
Velocity can only exist along a straight line at any given point. The rock must have a velocity that points beyond the path described by the circle. The tension, caused by the limited length of the string, adds another vector to the rock's motion, pulling its resultant vector continually inwards from the tangential component. If the string breaks, the centripetal force can no longer act on it, and the inward pull disappears. This leaves only the straight path, causing the rock to fly "away" from the circle.