The author is using the rivers as symbols. The Devon River, in Gene's mind, is fondly remembered. It is the river into which he and Finny jumped many times during the halcyon summer before they were drawn into the war. It is significant that the Devon is a freshwater river, as it represents for Gene a time of innocence and freedom, when he and his companions were still boys, too young to be considered as fodder for war. The Devon is the river of the carefree summer, and even after Finny's accident, it does not lose its appea.; it is symbolic of serenity, and a time of peace.
In contrast, the Naguamsett River is "an ugly, marshy, saline river." It is influenced by the forces of the world outside of the sanctuary of Devon School, like "the Gulf Stream, the Polar Ice Cap, and the moon." The Naguamsett does not have the pristine and untouched insulation of its counterpart, the Devon River. Dark and forbidding, it is a symbol of the war, and the outside world gone mad.
The Devon River may represent the pureness of Finny while the Naguamsett may represent the Gene's heart which is full of hatred.
Also, after having pushed Finny, Gene never sets foot in the Devon. it is almost like he is sinful, and simply can't do it.
But, without any intentions, he does end up in the Naguamsett after his fight with Quackenbush. This is almost like his baptism into his sinful and guiltful life.