The answer to this depends on the circumstances. Officers cannot simply go around looking into the stalls in public restrooms just because they want to.
People generally have an expectation of privacy when they enter a stall in a public restroom. However, there are situations in which this expectation is overridden. The general rule is that the expectation of privacy is greatly weakened if there is reason to suspect that criminal activity is going on. If there is sufficient reason to believe this (as, for example, in this case where a man and a woman entered a one-toilet bathroom together in an area where prostitution is common), then courts have ruled that there is no expectation of privacy and police may look into the stall.