In addition, bowls of milk is one of the items his sister and family put in the room to feed him with during his whole "change." Perhaps he associates milk with his unhappiness and inability to reach fulfillment.
Because literally everything about Gregor's former existence has been altered. He no longer likes milk, the food he associates with comfort and humanity. He is now a beetle and acts like a beetle, those scavengers who exist on the discards of other beings.