Assuming that people interpret events differently, why do you think that the Grandfather thinks that Lil Bit flunked out of college her first semester and then didn't return home until a while after Peck's death?
The way the characters are developed, it is difficult to understand the inner thoughts of Big Papa. He is a sexually greedy man himself. While he is proud of his marriage, perhaps most proud of how he eloped with his wife when she was but fourteen, he also confines his appetites within his marriage. So, while he wouldn't condemn an older man for desiring a very young girl as he himself did, it seems he wouldn't agree with inappropriate relations outside of appropriate marriage. This aspect of his character suggests he had not made a connection between Peck and Lil Bit.
It is never specified whether the grandfather is aware of Li'l Bit's incestuous relationship with Peck or not, but we can probably assume that given the massive taboo surrounding incest, he would have avoided coming to this conclusion, even if he had suspected it. Perhaps he just felt that Li'l Bit needed to have her rebellion and her time away from home. I don't necessarily think he would have connected Li'l Bit's disappearance and reappearance with Peck's death.