The literary device used in the quote "There might also be a dragon with five legs in my house, but no one has ever seen it" is verbal irony. More specifically, the device is sarcasm.
Sarcasm is considered verbal irony. Verbal irony is not only known as when a person says or writes one thing and means another, but also (in regards to sarcasm) as an expression where sneering, scorning, or jesting is done in order to show personal disapproval in the guise of praise of a person, a situation or thing.
Therefore, Proctor (in Act III) is being sarcastic through his use of verbal irony. Readers know that there is no five legged dragon in his home. He is simply saying it to make a point.