On one level, Holden likes the museum so much as a kid simply because there are so many interesting things to see there, like the "pretty" deer and birds and the Eskimo and the "spooky" witchdoctor in the canoe. He also seems to have liked the people that worked there; he remembers the attendants as being "nice" and he further recalls that the teacher who used to take them there "never got sore." All in all, Holden's experiences of the museum have been wholly positive.
As other answers have mentioned, Holden also likes the museum because of its stasis: nothing ever changes, and there is an air of security and stability there which he craves even more now that he is in the throes of confused adolescence. He reflects that, while the museum itself never changed, he and the other children who visited it did, simply because in real life nothing ever stays quite the same:
The only thing that'd be different would be you.
It is interesting that Holden is unable to re-enter the museum as an adolescent, even though he plans a visit and has "been looking forward to it and all." When he gets there, all of a sudden he feels that he couldn't "have gone inside for a million bucks." Although he doesn't admit it, this is probably because he realizes that there is really no going back to a place which he associates exclusively with the happiness and safety of childhood. The museum functions as an important and poignant image in the novel.
The direct quote from the book is:
"The best thing, though, in that museum was that everything always stayed right where it was. Nobody'd move."
Holden loves the museum because nothing changes. He's stresses over the ducks because the leave during the winter. He loves the ducks in the museum because they always stay there. This reveals a lack of stability in his mind and his desire for routine and stability. These are all symptoms of his depression.
Because he thinks that all the stuff in the museum never changed. He wishes his life could frozen in time like that. He doesn't want to grow up. Also he could remember memories of childhood when he got there.