This expression is from The Critic by Sheridan, but it is spoken by Tilburina, a character in Puff's play within a play The Spanish Armada. She is "a parody of the tragic heroine, torn between love and duty. She eventually goes mad after Don Whiskerandos’s death and throws herself into the sea." Sounds like Ophelia to me. Here is a bit of the scene:

Tilb.: ...--But O, to me can they no joy afford!Nor rose, nor wallflower….…nor all the finches of the grove!

Puff: Your white handkerchief, madam!

Tilb.: I thought, sir, I wasn’ to use that till ‘heart rending woe.’

Puff: Oh, yes, madam, at ‘the finches of the grove,’ if you please.

Tilb.: Nor larkLinnet, nor all the finches of the grove! (weeps)

Puff: Vastly well, madam.

Dan.: Vastly well, indeed!

Tilb.: For, O, too sure, heart-rending woe is nowThe lot of wretched Tilburina!

Dan.: O!—‘tis too much!

Sneer: O!—it is indeed!

(Act II, scene 1)

The Critic was a popular play. The eNotes study guide cites F. O'Toole as noting that "of the twelve most often staged plays in England between 1776 and 1800, four were by Shakespeare and two (The Duenna and The School for Scandal) were by Sheridan." He further states: ‘‘With The Critic holding its place as one of the most frequently performed afterpieces, Sheridan the playwright continued to occupy a central place in British cultural life.’’

Dickens's audience would have been familiar with the allusion to this play.

I don't know about The Critic, but when I teach Great Expectations, we discuss how the name of the snotty gentlemen's club fits with the adage "Birds of a feather flock together".  It may have been Dickens' commentary about how the members of the club all do, act, and think the same way... much like a flock of birds.  When one bird flies off the telephone wire, the rest seem to fly right behind and in unison.  It seems appropriate for the type of young men who belong to "The Finches of the Grove".