The major argument behind this is that choosing not to wear a seat belt could have a serious impact on people other than you. It could end up burdening society and therefore should not be allowed. If you are in a car crash and you die, your action can have serious impacts on other people. If you have children, for example, their life prospects can be damaged. Without one of their parents, they will likely grow up with less money than they would have had. They will also be emotionally scarred to at least some degree. Society might have to spend more of its resources on them. School counselors might have to work with them more. They might be more likely to end up in the criminal justice system. Society might also lose the potential benefits your children could have provided. If your children do worse in school and therefore have worse jobs, society loses because their talents are not being fully realized.
In these ways and many others, your decision not to wear a seat belt can have serious impacts on society. Therefore, it can be argued that society has the right to intervene.