The fact that the writing is being done with a finger suggests that it is the hand of God tracing the words in the sand. Once you have done something you will regret, there is no way to erase what has been written because it is also engraved in your own memory. Life only moves in one direction. Many writers have commented on this same human feeling of remorse over something done or something left undone which should have been done. Lady Macbeth tells her husband after the murder of King Duncan:

What's done cannot be undone.

Here are some others:

Droll thing life is--that mysterious arrangement of merciless logic for a futile purpose. The most you can hope from it is some knowledge of yourself--that comes too late--a crop of unextinguishable regrets.
Joseph Conrad, Heart of Darkness

What makes old age hard to bear is not the failing of one’s faculties, mental and physical, but the burden of one’s memories.
Somerset Maugham

Sonnet 30
When to the sessions of sweet silent thought
I summon up remembrance of things past,
I sigh the lack of many a thing I sought,
And with old woes new wail my dear time's waste:
Then can I drown an eye, unused to flow,
For precious friends hid in death's dateless night,
And weep afresh love's long since cancell'd woe,
And moan the expense of many a vanish'd sight:
Then can I grieve at grievances foregone,
And heavily from woe to woe tell o'er
The sad account of fore-bemoaned moan,
Which I new pay as if not paid before.
But if the while I think on thee, dear friend,
All losses are restor'd and sorrows end.
Shakespeare

Young people may find it a little difficult to understand what is meant by the line in question, because they may not have had time enough to build up much of a bad record. But they are certain to have their share of memories they would like to erase if they live long enough. We all make many mistakes in life, and we all have a store of unextinguishable regrets.

You need to include a couple words from the next line in order to complete the entire phrase and make sense of it. The full phrase should read, "The moving finger writes; and having writ, moves on." The complete quatrain containing the phrase is even more helpful in understanding its meaning.

The Moving Finger writes; and, having writ,
Moves on: nor all thy Piety nor Wit
Shall lure it back to cancel half a Line,
Nor all thy Tears wash out a Word of it.

The "moving finger" is being used as a symbol for Fate or for Time. The point is that once the moment is past, it's gone. There's no way to recapture it, regardless of your prayers or cleverness or anything else. Once the "moving finger writes," time marches on and nothing will ever be able to change whatever happened during that instant in time. The incident is over and done, recorded and unchangeable.