Things to remember when contemplating the speed of reading are the genre of fiction and the diction (vocabulary, register, style) of the author, characterization of characters, the techniques employed, the abstract concepts, and the structure of the novel.
If you are reading popular fiction, like Contagion by Robin Cook, reading will go by at a fast speed because the genre is uncomplicated, his diction is simple (though he throws in sophisticated vocabulary now and then), characters are simple and straightforward (though he does offer character background), his abstractions are minimal, his techniques are minimal, and his story structure is uncomplicated though it has layers of detail. In this sort of a product, there are no abstractions nor any conceptual intricacies to slow down reading: there is nothing compelling you to give thought and to engage in mental conversation or debate with the narrative or characters. For an idea of this "mental conversation or debate," think of watching one of the George Clooney's Ocean's movies: engaged viewers are in constant inner dialog with the film as the narrative unfolds because there are mysteries or motives or ironies that engage us.
Literary fiction, like that written by Thomas Hardy, Charles Dickens, Tolstoy, Dostoevsky, Austen is not read at a fast speed (indeed that is one of the chief complaints from inexperienced literary students) because of intricacies in diction, abstract concepts, characterization, and novel structure. In other words, literary fiction cannot be read at a fast speed because the wealth of detail and the intricacies of elements and techniques (metaphor, symbolism, metonymy, imagery) require time and consideration because of the full "flavor" of the work. Two contemporary writers who might be included in this description are John Le Carre and Jonathan Safran Foer.