Nick makes this observation when he talks with Gatsby the morning after Myrtle's death. Gatsby had stood vigil all night at the Buchanans' home to make sure that Daisy was safe. Nick had been there for a short time, also. While he was there, Nick had glimpsed Tom and Daisy inside the house, sitting together at their kitchen table:
[Tom] was talking intently across the table at her and in his earnestness his hand had fallen upon and covered her own. Once in a while she looked up at him and nodded in agreement . . . . There was an unmistakable air of natural intimacy about the picture and anybody would have said that they were conspiring together.
After witnessing this scene, Nick knows that Daisy will not be leaving her husband for Gatsby. This is what Nick remembers as he speaks to Gatsby the next day. Nick knows that Gatsby has founded his life upon the illusion that he can regain what he has lost and repeat his past with Daisy. Nick knows how much of himself Gatsby has invested emotionally in his dream. Nick cannot bring himself to be the one to destroy Gatsby's hope and crush his dream by telling him the hard truth.