Why are "Lamb to the Slaughter" and "The Landlady" considered mysteries?
I have to take issue with your question. I don't actually see how "Lamb to the Slaughter" can be considered a mystery, as there is not much that we as readers are left to work out or to discern. We are given all the information we need. Of course, the rather foolish police who tuck in with great gusto to the murder weapon might look at it differently, but this is because of the dramatic irony that makes this short story so successful.
"The Landlady," on the other hand, can definitely be considered a mystery story. Although the rather grim ending of Billy Weaver is inferred (he is poisoned by cyanide) there is so much that is left unanswered and the whole text is dominated by mystery and suspense. We are never told for example why it is that Billy is so hypnotised by the sign of the boardinghouse, why the landlady is able to open the door before he presses the bell and so on and so forth. This is a much more mysterious tale than "Lamb to the Slaughter," which is more of a darkly comic short story.