The reason why Yunior's dad takes his own children to eat at his mistress's house may have everything to do with the idea of hinting at his children that men can do such things because...they are men.
Yunior's dad is a very chauvinistic individual. He feels that he is king of the house and, unfortunately, his wife enables his behavior with her eternal submission to him.
Taking a mistress is, in the first place, a sign of machismo. He clearly does not care that he is married, and perhaps feels that he is completely entitled to get as many women as he wants.
Second, since his mistress is a woman and has her own home, Yunior's Dad's chauvinistic persona understands that if any place would be safe to go, it would be the mistress's house. Obviously the mistress does not complain, which means that she, too, is submissive to the man.
Finally, it is a known thing that Hispanic women are permissive with males when it comes to nourishing, supporting, and accepting them. The mistress probably wants to show Yunior's Dad that she, too, could be a mom type to his kids. She is making everything in her house sufficiently simple for Yunior's dad to always go and visit her.
Therefore, the real reason behind this behavior is a combination of the mistress's enabling behavior and Yunior's dad chauvinistic imposition as well as his idea that he deserves that much from women.