Can a motel owner hold a guest's belongings for due payments?
Yes, most every state in the US has a law that allows a motel or hotel operator to keep a guest's personal property until payment is made. These are called innkeeper's laws and under the facts you describe the motel legally has what is called an innkeeper's lien. What I would suggest you do is locate the web site that has the laws for your state and search for "innkeeper" or for "inkeeper's lien". That way you will know exactly what are the requirements under the law for your state, and what you need to do to have your property released. The law will probably also include a provision that the motel will have to keep your property secure and safe. This is because you and the motel owner have what is called a "bailment" under the law.
One more thing. Normally the back of the door in guest room a motel/hotel will have a sign that lists the details of that particular state's innkeeper law.