By definition, an "elegy" is a serious reflection for the dead. An elegy also tends to be a lament. Gray's elegy is exactly that; however, he isn't lamenting the death of any one, single individual. Most of the poem is a reflection on death in general. A central theme of the entire poem is that everybody is going to die eventually. It doesn't matter if you are poor or rich. You are still going to wind up dead. For example, the following line is directly pointing out that fame, fortune, and glory won't save you from death.
The paths of glory lead but to the grave.
As Gray continues to reflect on death in general, he begins reflecting on his own death, and he wonders what someone might say about him after his death. This occurs in stanzas 24 and 25.
For thee, who mindful of th' unhonour'd Dead
Dost in these lines their artless tale relate;
If chance, by lonely contemplation led,
Some kindred spirit shall inquire thy fate....
Gray then spends the next few stanzas having that "kindred spirit" ask somebody else about what the dead man (Gray) was like. In stanza 29, that person tells the "kindred spirit" to simply read the epitaph that is engraved on the tombstone.
"Approach and read (for thou canst read) the lay,
Grav'd on the stone beneath yon aged thorn."
The following stanza is the start of the epitaph. The significance of the epitaph is that Gray is writing his own epitaph. He's reflecting on his own death that obviously hasn't happened yet. That's a bit morbid in my opinion, but his lament is a fairly uplifting account of his life. He says that he had a humble birth, but he also says that he worked hard to gain knowledge.
A youth to Fortune and to Fame unknown.
Fair Science frown'd not on his humble birth,
Stanza 31 says that Gray more or less led a generous and "sincere" life, and Heaven rewarded him for that life. The epitaph ends by telling readers to not worry so much about other details because they don't matter. They don't matter because Gray is dead.