The short answer is that the danger lurks only within themselves. It is vital to Milton's theological argument that Adam and Eve are created by God 'sufficient to have stood/Though free to fall' (Book 3). Their key defence against threat is their God-given Reason and their steadfastness in the face of threat depends on their using Reason to keep Passion in check: 'Take heed lest passion sway/Thy judgment to do aught, which else free will/Would not admit' (Book 8 - Raphael warns Adam).
The longer answer is that danger lurks in the figure of Satan himself. And of course that very word 'lurk' suggests something of Satan's furtiveness as he approaches Eden in Book 9 - he's already been ejected once from Eden, having caused Eve her unsettling dream in Book 4. Of the pair, it is Adam who is more aware of this danger, as he tells Eve when she suggests that they work apart to tend the garden at the start of Book 9. Eve uses quasi-logical arguments to refute Adam's, including the idea that Paradise cannot be paradisal if beset by constant threat - and does he not trust her to withstand? Eve's only real apprehension of danger, if you could call it that, is when Satan has led her to the tree and she recognises it as the one forbidden by God, but it takes only Satan's seductive rhetoric to persuade her that eating from the tree, far from being a danger to her, will cause her to ascend to godliness.
Adam allows his Passion to overmaster Reason in choosing to follow Eve's course.