The first sign that Grandfather could be ill from yellow fever is in chapter 11, page 78, when he begins coughing in the back of the carriage on the way out of Philadelphia. Initially, he waves off the coughing fit as nothing, but on page 81, they are stopped by guards making sure no one with yellow fever enters their town. The doctor insists the grandfather has contracted the illness and he, along with Mattie, is thrown off the carriage and told to walk back the many miles to Philadelphia.
They haven't gone far, when, at the beginning of chapter 12, page 84, Grandfather asks if he can rest under a chestnut tree. Mattie feels his forehead and says "it was hot and dripping." When she looks into his eyes, however, she sees they are bloodshot rather than yellow. A sign, she hopes, that he has nothing more than summer grippe.
They never find if Grandfather has yellow fever or not, as in the end, he dies at the hands of robbers.