Bilbo's remarks shows he is an unlikely hero, one not interested in any type of adventure. Gandalf arrives and plainly says he is looking for someone willing to go on an adventure. However, Bilbo associates such things as almost scandalous. Bilbo Baggins and his family were considered respectable "because they never had any adventures or did anything unexpected."
Bilbo also is a creature of habit. He enjoys his orderly life and the unchanging comfort of his hobbit hole. He lives by the clock so the last thing he wants is something as terrible as an adventure to interfere with his routine.