The general answer is that this depends on the circumstances. There are times that the police may look into a restroom stall without violating the 4th Amendment's bans on unreasonable searches and seizures.
For example, in the case of United States v. Hill, the 8th Circuit Court of Appeals ruled that a couple that entered a one-toilet bathroom together did not have any reason to expect that they would have privacy in that bathroom. They did not have an expectation of privacy because they were acting in a suspicious manner (entering a one-toilet bathroom together in an area in which prostitution was common). Because they were acting suspiciously, it was okay for the police to look in on them.
In general, police cannot simply look into random restroom stalls. However, if there is some reason to be suspicious, they are allowed to act on that suspicion.