The previous answer is not exactly correct. Here are some actual "points of sail" (terms that describe well-defined courses) for reference:
"close hauled" = a course with the bow as close to directly into the wind as possible (a boat cannot sail directly into the wind, but must sail ~30 degrees or more off the wind, depending on the boat and rig).
"beam reach" = sailing perpendicular to the wind.
"deep broad reach" = sailing almost directly downwind, with the wind coming from behind, but a number of degrees away for safety.
"dead run" = wind directly behind the boat. Caution must be taken not to jibe (gybe) unintentionally.
By and large:
"Large" = in generally a downwind direction. This was the easier direction for square-rigged ships to sail.
"By" = Close to. In this case, "as close to as possible."
So sailing "by and large" would typically mean a "deep broad reach", since that is the safest way for a boat with a boom to sail downwind (on a dead run there is the danger of an unintentional gybe, where the boom swings wildly across the boat and breaks rigging and people).
By and large has therefore come to mean "approximately," as mentioned in the previous comment.
Related Term: Full and By
This is the "into the wind" course that is closer to what the previous answer was getting at.
"By" = as close as possible in the current situation.
"Full" = a course generally into the wind.
"Full and by" would therefore typically mean "close-hauled" on a boat with no square sails. It is the closest-to-the-wind point of sail. However, in a square-rigged ship, with the squares up (the squares are currently being used), the "full and by" course might be very close to a beam reach. That is as close as the square-rigger can sail to the wind with the squares up. In that case the boat sailing "full and by" is *not* close-hauled. With the squares down, and only using the triangular sails, the "full and by" course would also be "close-hauled."
The "points of sail" above are well-defined. The commands/descriptions "sail full and by" or "sail by and large" depend on the situation but have more flexible meanings.
On a sailing ship or sailboat, new sailors at the helm (steering wheel) are usually ordered to sail "by and large." This means that they are to sail into the wind at an angle, as opposed to directly into the wind. Sailing directly (or almost directly) into the wind is most efficient, however doing so may cause the sail to flap back against the mast (the pole supporting the sail), resulting in loss of speed and control.
Sailing "by and large" thus means to set a course that is generally correct, if not exact. Over time, the phrase has become a synonym for "approximately."
Sources: Layton, Cryil W. T. Dictionary of Nautical Words and Terms, 3rd ed., rev., p. 62; Pittsburgh Post-Gazette, (July 23, 1994), p. C-6.