The handkerchief is a gift that Othello has given to Desdemona. It has great sentimental value because it is a family heirloom. Othello considers the handkerchief to be a symbol of love freely given and freely received. Othello demands to see it because he is afraid that Desdemona is now freely giving her love to Cassio. In many ways, the handkerchief is a talisman in Othello's mind. If Desdemona is in possession of the handkerchief, then she must treasure Othello. If she does not have it, then (he believes) she does not.
So is he correct? That's a question that can be answered two ways. He is not correct in believing that Desdemona has been unfaithful--but his sentimental connection to the handkerchief certainly makes his demand understandable...and "understandable" could be construed as "correct" from his character's perspective.