The answer to this question would probably have been different 25 or 30 years ago. In those days, Grant was seen as a relatively poor president, partly because his legacy was tarnished by the scandals. Today, Grant is seen in a much more positive light, implying that the scandals do not tarnish his reputation to a very great degree.
For a long time, Grant was seen as a very weak president who was, at the very least, careless in how he chose his associates. People did not necessarily believe that Grant himself was corrupt, but they felt that he must not have been a very good leader if he picked so many high officials who were corrupt. When this was the conventional wisdom about Grant, it is clear that he was harmed by the scandals in his administration.
Today, however, Grant is seen much more positively. Grant is generally seen as a man who had the courage of his convictions. He is now remembered as a decent man who tried hard to achieve justice. Historians are now more likely to see him as a man who did the best he could in a set of circumstances that did not allow him to achieve everything that he would have hoped.
Thus, I would say that the scandals once tarnished his legacy but that they no longer have much of an impact on how he is perceived.