He feels "a calm so deep" as the city is about to wake up for another day.
Wordsworth is often regarded as a nature poet, someone who derived his inspiration primarily from contemplating the natural world. But in "Westminster Bridge," he shows that this wasn't always the case—that he could be every bit as inspired by an urban environment.
That said, it's notable that Wordsworth's breathless admiration for London takes place at a time of day—dawn—when there's almost no one around. One gets the impression that he wouldn't feel quite so calm during the middle of the day, with all those people rushing about the place. Essentially, Wordsworth treats London like a deserted mountain, peaceful forestM or some other feature of the natural landscape. It is only because of the quiet of the dawn that he's able to detect that "mighty heart" beating just beneath the surface of the city.