If that were all to be seen on the map, then nothing would have happened. The fact of the matter is that the door was a magical, enchanted door, only to be opened with the special key carried by Thorin during sunset on Durin's Day. Elrond explains the significance of the moon-letters to Thorin while the company rests up in Rivendell:
"Stand by the grey stone when the thrush knocks," read Elrond, "and the setting sun with the last light of Durin's Day will shine upon the key-hole" (50).
Without the guidance of the moon-letters, Thorin and company might have continued sitting outside for a very long time, mostly because they would not have seen the key-hole appear. Even knowing the secret of the door, they would have missed it, except for Bilbo's sharp eyes and wit; after all, he is the one who hears the thrush's knocking at puts the clues together at the last minute, just barely in enough time for Thorin to secure the key-hole.