I've adapted this answer from a college paper I wrote back in 2007, and the numbers have adjusted slightly since then. I suggest you check through the Sentencing Project's website for updated information. But the numbers haven't changed very much, and my conclusion still holds: yes, we should definitely let felons who have paid their debt to society vote in subsequent elections.
It is literally written on the first page of the United States’ story that governments derive “their just powers from the consent of the governed.” Although it has never been true that all of “the governed” have been permitted to voice consent or dissent through the ballot, the arc of American history has been one of steadily broadening enfranchisement. Today, there remains only one major group of mentally competent citizens over eighteen years old who cannot vote: currently incarcerated and released prisoners, or “ex-felons.”
This is a crucial issue for America because it disproportionately disenfranches non-white voters. This a) robs minority voters of the chance to participate in the democratic process and b) unfairly advantages the Republican Party, which tends to draw many fewer minority votes. The heaviest burden falls on African-American men, among whom 1.4 million, or 13% of the population, are disenfranchised – a rate seven times the national average (Mauer & King, 2007). States are also disproportionately affected according to their minority population and rate of incarceration, and in some states, as many as 1 in 4 black men are prohibited from voting (The Sentencing Project, 2007). The potential for unfair ballot access increases when states with a similar ethnic or socioeconomic makeup have similar laws. For example, Maine and Vermont, the two states with the highest percentage of white citizens (95.8% and 96.3%, respectively, according to the 2006 US Census) are the only two states that allow convicted felons to vote while serving their sentences. In contrast, the states that permanently disenfranchise residents convicted of a felony, Virginia and Kentucky, have considerably larger minority populations (at 19.6% and 7.4 % black, respectively).
These laws, combined with Florida’s sizeable population and influence as a “swing state,” forced felon disenfranchisement back into the limelight in the 2000 election. In an election that came down to just over 500 votes, ex-felons would almost certainly have changed the outcome. In Florida alone, 200,000 otherwise eligible voters were barred from voting in the 2000 elections because of a conviction, not including citizens incarcerated at the time. To put it differently, if Florida ex-convicts could vote, they would represent a population larger than the total number of registered voters of Wyoming, Montana, or North Dakota (see 2006 US Census).