What is the difference between Greek Tragedy and Marlowe's concept of tragedy as seen in Doctor Faustus?
To begin, Aristotle defines tragedy (in terms of the Greek dramatic tradition) as...
...an enactment of a deed that is important and complete, and of [a certain] magnitude...it is enacted...and through pity and fear it effects relief (catharsis) to such [and similar] emotions.
Here we can assume that the response of the audience is what is critical to the effectiveness of the tragedy, by eliciting "pity and fear" in response to the deed of great magnitude that the protagonist commits, while also providing the audience a sense of relief ("catharsis").
Note examples of tragic heroes from many of Shakespeare's plays: including Brutus from Julius Caesar, Hamlet from the play of the same name, and Macbeth (also from the play of the same name). In each case, there is the hero's destruction, yet there is also a sense of taking a wrong step that one has struggled with, that has led to his demise. With Brutus, he truly believes that his actions are noble—for he would sacrifice him life to save Rome. Hamlet is desperate to avenge his father's death, but fears losing his immortal soul in the process, and is eventually killed because he delays killing his uncle—who ultimately kills Hamlet (and almost everyone else) at the end. Macbeth (who provides the most striking comparison with Faustus) struggles with doing the moral thing. He admits that his ambition (and the audience is aware of his wife's endless harping) drive him to turn his back on what he knows is the decent and moral thing to do.
I have no spur
To prick the sides of my intent, but only
Vaulting ambition… (I.vii.25-27)
Consider, then, Doctor Faustus, by Christopher Marlowe. Faustus is guilty of hubris and ennui—he believes he has learned all there is to learn, and discounts the importance of a man's immortal soul. He does not mistakenly choose to do wrong: he actively searches out Lucifer, and foolishly believes ("hubris") that he can control evil. His boredom with life leads him to look for something beyond knowledge (and he is very intelligent), turning to first magic and then a pact with the devil in order be obtain power. His eyes are open in doing so. He has no illusions. Faustus calls on Valdes and Cornelius to help him delve into the forbidden practices of magic:
And make me blest with your sage conference.
...Know that your words have won me at the last
To practice magic and concealed arts (I.99-100, 101-102)
He has many chances to ask for God's pardon, for there are those who appeal to his better sense, but he seems to have no desire to be saved.
Satan begins to sift me with his pride:
As in this furnace God shall try my faith,
My faith, vile hell, shall triumph over thee.
Ambitious fiends! see how the heavens smile
At your repulse, and laugh your state to scorn!Hence, hell! for hence I fly unto my God. (XV.4-9)
On God, whom Faustus hath abjured! on God,
whom Faustus hath blasphemed! Ah, my God, I would
weep, but the Devil draws in my tears. (XVI.29-31)