There was no particular need for Montresor to keep the stones and mortar too carefully hidden. If Fortunato had noticed them, it wouldn't have occurred to him that they had any connection with himself. He would probably have assumed that Montresor was having some sort of necessary repair work done in the catacombs. After all, these tunnels were hundreds of years old. The only thing that might have troubled Fortunato would be the sight of those chains. But they are attached to the granite wall and cannot be seen from outside the dark niche.

When Fortunato finds himself bound by the padlocked chains, he begins to sober up quickly. He realizes that his only way out of the trap is to use psychology. When he laughs and says, "An excellent jest," he is revealing his own character. If it were a practical joke, or jest, it would be a hideously cruel one. Cruel jests are the kind a man like Fortunato would play on others, and thus they are the kind he would appreciate. He pretends to be enjoying Montresor's "jest," even though he is probably only hoping against hope that it might indeed be an elaborate practical joke.

In this story, Montresor lures Fortunato down into the catacombs beneath Montresor’s home.  He does so on the pretext that he wants Fortunato to appraise a pipe of amontillado (wine) that he has acquired.  But what he really wants to do is to kill Fortunato in a particularly gruesome way.  He plans to wall Fortunato up and allow him to die of thirst.

Montresor has planned this out carefully.  He has ensured that no servants will be around to witness what he is doing.  Just as importantly, he has already brought in and hidden a quantity of building materials that he will use to wall Fortunato up once he has chained him so he cannot escape.  Montresor has hidden these building materials (stone and mortar) under a heap of human bones.  The bones appeared to have fallen down from where they had been placed in the catacombs.  However, Montresor seems to have made the pile himself to conceal the stone and mortar.  As he says

I busied myself among the pile of bones of which I have before spoken. Throwing them aside, I soon uncovered a quantity of building stone and mortar. 

The answer, then, is that the stone and mortar were hidden under a pile of bones.