Looking at this novel clearly indicates that Bilbo as a character faces numerous challenges as he goes on his first adventure with the dwarves and leaves the comfortable confines of the Shire. There could be any number of different answers to this question. You could easily say that Bilbo is most challenged when he meets the trolls, or has to rescue the dwarves from the spiders, or from the elves, or when he steals the Arkenstone from Thorin. However, if we look at the novel carefully and what the narrator tells us, there is a specific event that apparently requires most courage from Bilbo. This comes in Chapter 12, when Bilbo goes to the dragon's lair alone to face Smaug:
It was at this point that Bilbo stopped. Going on from there was the bravest thing he ever did. The tremendous things that happened afterwards were as nothing compared to it. he fought the real battle in the tunnel alone, before he ever saw the vast danger that lay in wait.
Facing the dragon completely by himself, especially bearing in mind the kind of sounds that Bilbo can hear Smaug making, is undoubtedly a massive challenge for him, and he needs to summon up all of his bravery in order to go into Smaug's lair.