This depends on what aspect of freedom of speech and expression you are talking about.

If you are talking about the content (about what ideas we can express), our freedom only ends when we subject others to a "clear and present danger" of harm.  That means we can't slander others or threaten them, but that we can criticize them as much as we want, even if it hurts their feelings.

If you are talking about how we speak or express ourselves, the line is harder to draw.  We know that punching someone to express dislike is not protected expression.  However, other sorts of conduct like holding protests or burning the flag are protected.  Somewhere between those is the line between expression and conduct.

If you are talking about things like the time and place that we express ourselves, the restrictions are much tighter.  We can't express ourselves at times and places (late at night, in the middle of a busy street) or in ways (really loudly) that are too intrusive or cause too many problems.

All of these could be seen as places where freedom of expression and speech end.