The disadvantages of such arrangements greatly outweigh the advantages, which is why cities in rich countries generally have zoning laws to prevent such things from happening.
There are two major benefits of building factories near to residential areas. First, the factories will typically create many job opportunities for the residents of the areas. Second, the fact that the people live near to the factory makes things easy on infrastructure. The city in which the factory is built will not have to worry about building extra roads or rapid transit lines to carry workers to the factory.
However, there are many disadvantages that are felt by those who live in the residential areas. Perhaps the greatest of these has to do with the annoyances brought by the factory. There might be loud noises all day and night. There could be bad smells. There is likely to be a great deal of traffic as trucks or trains bring raw materials to the factory and carry away the finished products. All of these sorts of annoyances can seriously reduce the quality of life enjoyed by the people of the residential area.
People in rich countries, at least, tend to feel that the negatives outweigh the positives. Therefore, they tend to enact laws preventing this sort of arrangement.