Colonel Brandon invited a large party, including Elinor, Marianne, their young sister, Lady Middleton, Sir John, and Mrs. Jennings to tour the grounds at Whitwell. Whitwell is an estate owned by his brother-in-law who was abroad and left strict orders about allowing visitors on to the estate. Colonel Brandon had permission to visit the estate, but his party could only visit if he was present. He intended to give them a picnic and take them sailing on the lake at Whitwell (Ch. 12). However, plans fell through and no one, not even Elinor, was able to visit Whitwell.
The morning of the party, Colonel Brandon received letter from London. He was evasive about the nature of letter, saying that it was not bad news and "merely a letter of business," even though the letter made him "change colour," in other words grow either pale of flushed (Ch. 13). Though he claimed it was just a business letter, he said that it urgently requested his attention in London. Despite all protests, he had to cancel the party and leave for London immediately. He could not "afford to lose ONE hour."
Hence, no one in the story made it to Whitwell that day.