By definition, an "elegy" is a serious reflection for the dead. An elegy also tends to be a lament. Gray's elegy is exactly that; however, he isn't lamenting the death of any one, single individual. Most of the poem is a reflection on death in general. A central theme of the entire poem is that everybody is going to die eventually. It doesn't matter if you are poor or rich. You are still going to wind up dead. For example, the following line is directly pointing out that fame, fortune, and glory won't save you from death.
The paths of glory lead but to the grave.
As Gray continues to reflect on death in general, he begins reflecting on his own death, and he wonders what someone might say about him after his death. This occurs in stanzas 24 and 25.
For thee, who mindful of th' unhonour'd Dead
Dost in these lines their artless tale relate;
If chance, by lonely contemplation led,
Some kindred spirit shall inquire thy fate....
Gray then spends the next few stanzas having that "kindred spirit" ask somebody else about what the dead man (Gray) was like. In stanza 29, that person tells the "kindred spirit" to simply read the epitaph that is engraved on the tombstone.
"Approach and read (for thou canst read) the lay,
Grav'd on the stone beneath yon aged thorn."
The following stanza is the start of the epitaph. The significance of the epitaph is that Gray is writing his own epitaph. He's reflecting on his own death that obviously hasn't happened yet. That's a bit morbid in my opinion, but his lament is a fairly uplifting account of his life. He says that he had a humble birth, but he also says that he worked hard to gain knowledge.
A youth to Fortune and to Fame unknown.
Fair Science frown'd not on his humble birth,
Stanza 31 says that Gray more or less led a generous and "sincere" life, and Heaven rewarded him for that life. The epitaph ends by telling readers to not worry so much about other details because they don't matter. They don't matter because Gray is dead.
This long poem, a series of stanzas treating various natural phenomena at their ends, and set in a tranquil twilight setting (itself the ending of the day), memorializes the unheralded, unmemorialized lives of everyday beautiful things, unsung beauty that “Awaits alike the inevitable hour.” Much of the beauty of natural things, desert flowers or ocean gems, display their beauty to no audience, no receiver of their beauty. The same is true of the persons lying in a country graveyard, not famous or celebrated, but nonetheless beautiful in their simple rural setting; “Far from the madding crowd’s ignoble strife,/Their sober wishes never learned to stray”. The epitaph that ends the poem, putatively written on a simple gravestone and addressed to some unknown country person who lived his life, “A youth to Fortune and to Fame unknown” is really the Romantic poet’s oath: to be the audience, the receiver of nature’s beauty, and to give an epitaph to all the otherwise unacknowledged beauty, in the poems themselves.