From the earliest moments of their meeting, the narrator finds some aspects of the signalman's manner and behaviour to be extremely unusual. The first of these occurs in the opening paragraph when the narrator shouts down to announce his presence. The signalman's response is to look down the train line, not directly above, a move which the narrator finds "remarkable."
Similarly, when the narrator makes his way down to the signal box, he interprets the signalman's attitude as one of "watchfulness" and "expectation." The narrator is unable to explain the reasoning behind this, though he stops for a moment to observe it.
Once the narrator has reached the signal box, he tries to engage the signalman in conversation, but, once again, his efforts are met with an unusual response. Instead of responding, the signalman looks back to the red light, "as if something were missing from it," and then asks if he has seen the narrator before.
The narrator is baffled by the actions and words of the signalman and begins to wonder if he might have a disease of the mind. At one point, he also wonders if the signalman is, in fact, a ghost:
The monstrous thought came into my mind, as I perused the fixed eyes and the saturnine face, that this was a spirit, not a man.
In truth, the signalman is neither supernatural nor mad. As the narrator is about to learn, the signalman is plagued by a ghost who, for reasons unknown, is able to predict future accidents on the line. Tragically, and unbeknownst to the signalman, the ghost has appeared for a third time to warn him of his own impending demise.