The elegiac atmosphere is established immediately in the poem with Gray's description of the countryside surrounding the cemetery and the time of day that makes up the poem's setting. The day is ending:
The curfew tolls the knell of parting day,
The lowing herd winds slowly o'er the lea,
The plowman homeward plods his weary way,
And leaves the world to darkness and to me.
The reference to "tolls the knell" suggests the slow and deep ringing of a funeral church bell. As the cattle return from pasture, their "lowing" creates a sense of softness and peace. The plowman is tired, moving slowly homeward. Darkness is falling. Everything in the opening stanza has connotations of fading light as quiet and stillness slowly descend upon the scene.
The second stanza continues the tone:
Now fades the glimmering landscape on the sight.
And all the air a solemn stillness holds,
Save where the beetle wheels his droning flight,
And drowsy tinklings lull the distant folds;
As the darkness gathers, the countryside fades from view. A "solemn stillness" reigns, except for the "droning" sound of a beetle in flight and the "drowsy tinklings" of herds with their cow bells moving in pastures far away and out of sight.
The only other sound that disturbs the stillness is introduced in the third stanza, that of a "moping owl" that can be heard as the moon shines down upon the "yonder ivy-mantled tower" where her secret nest can be found.
In these three stanzas, Gray establishes a somber atmosphere through rich sensory detail that perfectly suits the content and the tone of the elegy to follow.
To have an elegiac tone, it needs to adequately convey a feeling of expressing remorse and sadness for something that is past, or lost. In his poem, Thomas Gray mourns for the lost lives of all of the country people that are buried there. He does this rather successfully. He starts right off creating a sad, lonely, morose tone as the last light fades and he is left alone in the graveyard. The last plowman "leaves the world to darkness and me," which sets a melancholy tone right away. An elegiac tone would not be upbeat or cheerful, so Gray puts himself alone, at night, right off the bat to set the right mood for an elegy. The environment and setting itself are symbols of loss and death; the fact that "fades the glimmering landscape" and day dies out, just as these people's lives died out, sets the right mood.
Then, after having set the mood, Gray actually becomes openly elegiac. He goes on and on about how these people shall "no more" go about the different aspects of their lives; he mentions farmers, housewives, and children. He mourns the loss of their "jocund" work, their "sturdy" ways. He lectures anyone who would judge or mock these simple farmers and peasants. He says that they too deserve mourning.
It is possibly after this point that the poem gets a bit less elegiac and more contemplative; he ponders life and death in general, and not specifically in relation to these people that have passed on. And the ending is futuristically mournful, as he imagines his own life having passed on. So, in this poem the best tone of elegiac moods are in the beginning as he sets up the moods with the setting, and specifically relates the losses of the lives of the people in the churchyard. I hope that these thoughts help; good luck!