Chapter XIX, which follows the chapter in which the newly deceased Judge Pyncheon is mocked mercilessly (though, admittedly, deservedly), is entitled "Alice's Posies," and therein lies the answer to your question. Though the narrator spends some time describing the Pyncheon elm which is itself an important symbol in the book, he alights on the flowers that grow out from the little valley between two of the house's seven gables. He says that if an observer were looking upon the house,

One object, above all others, would take root in [their] memory. It was the great tuft of flowers—weeds, you would have called them, only a week ago—the tuft of crimson-spotted flowers, in the angle between the two front gables. . . . They were flaunting in rich beauty and full bloom, to-day, and seemed, as it were, a mystic expression that something within the house was consummated.

There you have it. The narrator tells us directly that the flowers seem to have bloomed with a new vigor, coming alive in their fullest expression, as if to celebrate the end of something. Jaffrey's death marks the end of an era, a bleak one for the Pyncheon family, and—with the removal of his darkening and sinful presence—the house and the family are free to move on with life.