The poem you're referring to is Dickinson's "The Way to know the Bobolink". Here are the first three stanzas:
The Way to know the Bobolink
From every other Bird
Precisely as the Joy of him --
Obliged to be inferred.
Of impudent Habiliment
Attired to defy,
At times to Majesty.
Of Sentiments seditious
Amenable to Law --
As Heresies of Transport
Or Puck's Apostacy.
It's an almost child-like poem, written in ballad meter (a line of iambic pentameter followed by a line of iambic trimeter) and, I suppose, to answer your question, there are two key things you need to know.
Firstly, it's that the "Bobolink" is a type of fast-moving, small blackbird, and that - as the first stanza states - Dickinson (in more than one poem) uses it as a symbol of joy and energy.
Secondly, "apostacy" is a term used when people turn away from or reject their former religion. Quite how Puck can be considered to forego religion - or, in some more metaphorical sense, turn away from something - is where it becomes interesting. Is that he gets Oberon's orders (to put the love-juice on Demetrius' eyes) wrong - and then enjoys his mistake? Is it perhaps, his turn to the audience at the end of the play?
It's not, I would argue, a question to do with Puck's role in the play, his potential for mischief, for trickery and trouble-causing and for glee in the face of pain and mayhem, but more to do with precisely what might be his "apostacy".
Puck plays a humorous role in A Midsummer Night's Dream. He is a fairy who causes mischeif in the forest. When he is asked to annoint the eyes of Demetrius with a flower so that he will fall in love with Helena he mistakingly annoints the eyes of Lysander. This is what causes the conflict throughout the rest of the play.