"The Phantom Rickshaw is a story within a story. The first person narrator tells us he was given the manuscript of the tale which will be summarized below, written when the author was suffering from a high fever and shortly before he died. The frame casts doubt on the ghost story that follows, but the very doubt the narrator suggests tends to make us want to believe the narrative is true, especially as the second narrator is so vehement.

This second narrator, Theobald Jack Pansay, falls in love with a married woman, Agnes Keith-Wessington. She falls more fully and devotedly in love with him. When Pansay tells Agnes he is tired of her and wants to break off the affair, she expresses her continuing devotion. This makes him hate her.

Later, Jack falls in love with Kitty Mannering and they become engaged. Jack, who now hates Agnes even more, tells her of the engagement. She insists it is a mistake and that they will get back together some day. Jack says cruel things and flees her, Agnes's rickshaw following after him. A week later Agnes dies. Jack believes his cruelty killed her.

A few weeks later, Jack is riding in a carriage with Kitty when he sees Agnes's rickshaw, though Kitty does not. He is amazed when their carriage cuts through it, and then he realizes it is a phantom rickshaw.

Jack is haunted by Agnes's ghostly rickshaw and her golden head as she rides through him time after time like a ghost crying that his engagement is a mistake, a "hideous mistake."

Kitty breaks off the engagement over Jack's insistence he is seeing a ghost, Jack is eventually deemed insane, and yet he goes on meeting Agnes and her rickshaw every night, pulled to her by some need, until he too dies.