A condition of thinking about the future, Frank Kermode once wrote, is that we assume one's own time stands in an extraordinary relation to it. “We think of our crisis as pre-eminent, more worrying, more interesting than other crises.” Everyone who is anyone in the world of letters certainly scrambled to offer their interpretation of the apparently world-changing events of 11 September, a crisis more eminent, more worrying and more interesting than past crises, if Martin Amis and the well-known thriller writer Robert Harris, among others, were to be believed. The result was an inevitable overreaction, not to the event itself, which was desolating, but to its world-historical implications. But then, hysteria is something in which we have come to specialise in this country, as exemplified by the disturbingly intemperate media response to the recent death of George Harrison, a reclusive paranoid who spent much of his time hidden behind the high walls of his fortified mansion. A sweet lord indeed.

“The imagination,” Wallace Stevens said, “is always at the end of an era”, and the predominant tone of much literary reflection on 11 September, and its dislocating aftermath, was catastrophist—eschatological anxiety and an unconvincing sudden seriousness, as if human nature itself changed the day the towers collapsed. Or perhaps it was merely that we in the relatively benign, affluent west had forgotten that the world has always been a spectacular carnival of suffering. Arthur Schopenhauer, in his essay “On the Suffering of the World”, asks us to imagine an animal being torn apart and eaten by another animal, and suggests that we contrast the pleasure of the predator with the pain of its dying victim: this, he seems to say, is an incarnation of the relentless, pitiless struggle for survival that is life on the planet. “History,” he continues, “shows us the life of nations and finds nothing to narrate but war and tumults … [Man] discovers adversaries everywhere, lives in continual conflict and dies with sword in his hand.” So it was ever thus.

Yet there seemed to be general agreement among most writers that a certain kind of literature was no longer possible after 11 September, as if, as Andrew O'Hagan put it, “language is something else now, and so is imagery, and so is originality”. For James Wood, writing in the Guardian, the great age of the novel of social hyper-realism—as represented by Don DeLillo's 900-page epic Underworld and its numerous imitators—is at an end. Rather, he called for a return to the novel of stylised inwardness, the work that can show us that “human consciousness is the truest Stendhalian mirror”. He wrote: “The idea that the novelist's task is to go on to the street and figure out social reality may well have been altered by the events of 11 September, merely through the reminder that, whatever the novel gets up to, the ‘culture’ can always get up to something bigger … If topicality, relevance, reportage, social comment, preachy presentism, and sidewalk-smarts—in short, the contemporary American novel in its … triumphalist form—are novelists' chosen sport, then they will sooner or later be outrun by their own streaking material.”

Implicit in much of the commentary written by novelists is the anxiety that, in the present circumstances, fiction, the make-believe, is either irrelevant or incapable of offering a convincing representation of a world rendered dramatically unstable by religious terrorists; that our invention will always be defeated by the fantastic nature of contemporary reality. Philip Roth expressed a similar concern when, in 1960, he wrote: “The American writer in the middle of the 20th century has his hands full in trying to understand, describe and then make credible much of American reality. It stupefies, it sickens, it infuriates, and finally it is even a kind of embarrassment to one's own meagre imagination. The actuality is continually outdoing our talents, and culture tosses up figures almost daily that are the envy of any novelist.”

There is a prevailing feeling, certainly in Britain, that Roth was right: that long ago many British writers turned away from the defining particulars of their age, retreating instead into the costume dramas and ready-made stories of history. “If an English novelist writes realistically about the present,” Sebastian Faulks once told me, “the result is usually banal, uninteresting or reads like a style piece.” Which perhaps explains the popularity of those such as Faulks himself, Louis de Bernières and Pat Barker, who have found in history, and the pathos of two world wars in particular, a resonant subject, one far removed from the political realities and dilemmas of our time.

It was no surprise to me that the winner of the Booker Prize this year—Peter Carey's The True History of the Kelly Gang (Faber and Faber)—was a work of archaeology and historical reclamation, in which the past is never past, but always reverberates strangely in the present. It is an accomplished book, a virtuoso exercise in pastiche, but it tells us little of what it is like to be alive today. It has no vision of contemporary crisis. Carey looks resolutely back, not forwards; he is engaged in a complicated process of reimagining Australian history, subverting the founding myths of his native land.

Those of our major writers, such as Martin Amis and Salman Rushdie, who have worked to document their times in fiction—the crush and frenzy of cities, the tyranny of the media image, the baleful effects of junk and celebrity culture, the crisis of faith, the end of ideology—have also become our most consistently reviled, perhaps because their brand of cartoonish social realism demands too much of us, as if it is somehow an affront to our culture of complacency.

No book was more traduced this year than Rushdie's Fury (Jonathan Cape), his portrayal of New York during the money-madness of the recent new technology boom; and yet no book was more alert to the crisis of modern urban experience. Fury is saturated in news, in the hard, fast events of the here and now. It has the lustre of today's newspaper—and much of its irrelevance and trivia, too. Above all else, it is shadowed by the sense of an ending, not merely the end of a stock-market boom, but of something far stranger and more menacingly opaque—the end of an entire way of life, perhaps? In the light of recent events, Fury, for all its technical flaws, can be read as a prescient commentary on what was bad about life in modern America, and soon to become much worse.

Reflecting on the history of the 20th century, it is hard not to conclude that the most challenging literary works were produced at moments of the greatest political and social upheaval: that crisis nurtured creativity, as it may yet again today. There is much to be made from the similarities between the al-Qaeda terrorists—rootless, wandering nihilists—and the “superfluous man” of the fiction of Dostoevsky, Turgenev and Conrad, the morose, isolated fanatic who has no moral compass and who, in a corrupt society, can find no release for his restless energies. Because he is highly intelligent, has a certain courage and is motivated by hate, the superfluous man is also dangerous. But such figures have all but disappeared from modern fiction. Imagine, for a moment, what Joseph Conrad might have done with the story of John Walker, the American who converted to Islam at the age of 16: after changing his name to Abdul Hamid, he went to study the Koran in a madrasa in north-west Pakistan, and then, in an act of grand renunciation, crossed the border into Afghanistan. He ended up fighting for the Taliban, only to be captured and imprisoned in the northern fortress of Qala-i-Jangi, from where he emerged, hollow-eyed, as one of the few survivors of a four-day onslaught in which hundreds of non-Afghan Taliban fighters were massacred. Walker is a true Conradian grotesque, a figure tossed up by the culture that should be the envy of any novelist. But who among our leading literary writers could make great fiction from the rough contours of his life, in the way that Conrad used the true story of a failed attempt to blow up Greenwich Observatory as inspiration for The Secret Agent, his novel of anarchists and revolutionary exiles in London? Who would be capable of such a sustained, expansive imaginative engagement with the shifting complexities of his or her time? Perhaps only Ian McEwan, a writer who, before his superb Atonement excelled more as an exquisite miniaturist, but whose talents, you feel, could take him anywhere.

Perhaps, then, paradoxically, we are at a moment not only of great world-historical anxiety, but of rare opportunity, too: that heightened instability may inspire a new generation of writers to produce a new kind of fiction, rooted in the world yet imaginatively estranged from it, a fiction that breaks with existing conventions—the oppressive, low-grade tabloid pressure to entertain and to conform, which has so destroyed our television and is destroying our newspapers—through daring to make it new. As Henrik Ibsen wrote, at the end of the 19th century: “The great task of our time is to blow up all existing institutions—to destroy.” And out of that urge to destroy, he might have added, may come something dynamic and true. Certainly, the modern writer can no longer complain that he or she has no subject, that all is too quiet and orderly in the English garden.