The way the characters are developed, it is difficult to understand the inner thoughts of Big Papa. He is a sexually greedy man himself. While he is proud of his marriage, perhaps most proud of how he eloped with his wife when she was but fourteen, he also confines his appetites within his marriage. So, while he wouldn't condemn an older man for desiring a very young girl as he himself did, it seems he wouldn't agree with inappropriate relations outside of appropriate marriage. This aspect of his character suggests he had not made a connection between Peck and Lil Bit.