Women and African Americans have safeguards for their right to vote because they are the ones who have had the vote denied to them on a systematic basis in the past. Therefore, they are the ones who need a safeguard.
White males, as a group, have not had to worry about their voting rights. Early in the history of the US, men without property were ineligible to vote in most states, but that changed by the end of the 1820s. Since then, no group of white males has needed a safeguard.
By contrast, both African Americans and women have been prevented from voting. Women were not allowed to vote in national elections until 1920. African Americans' right to vote was not truly protected until 1965. Because of this systematic denial of the vote in the past, these groups got safeguards for their voting rights.