What rights grandparents have and under what circumstances they have those rights are a function of state law. Each state has its own statute regarding the rights of grandparents. You will have to review the statute in the state that has jurisdiction over the parties.
Generally, no matter which state's law governs, there are unlikely to be any absolute rights of a grandparent against a parent. The standard for all custody law is "the best interests of the child." This means that if a court were to determine that the child's best interests were to not be with a grandparent, the father would prevail. This could come up in any number of ways, depending upon the state. For example, if a grandparent were found to be interfering with the child's relationship with the surviving parent, the court would not see that as being in the best interests of the child. Or, if the child were old enough and preferred not to see the grandparent, that would have some weight in some state proceedings.
There are two distinct sets of possible rights for grandparents. One is custody and the other is visitation. Custody can be complete or partial. Visitation can be supervised or unsupervised. The standard is the same for either, the best interests of the child. In no state are the courts in the business of affording any party rights over another party without meeting that standard.
If this is a personal inquiry, seriously consider consulting with an attorney in your state. Some states have legal services organizations that offer income-eligible help, and some counties have bar associations where you can consult with someone who practices family law for a reasonably-priced initial consultation. And it is always best for children if the adults who love them can work things out amicably.