Part of the reason why the loyalty oath was so controversial was because it almost seemed like an insult added to injury. Once the war had turned against the Japanese, Americans of Japanese descent were looked at as subversive and enemies of the state. The entire notion of internment was predicated upon the government's perception that all Japanese citizens need to be rounded up and placed in one area. The loyalty oath was a manner by which Japanese- Americans had to pledge their loyalty to a government which had prejudged them and commenced a bombing campaign against Japan which pushed the boundaries of humanity's treatment towards one another. All of this even before the dropping of the atomic bombs.
The loyalty oath that the US government asked of the Japanese-American internees was controversial for a number of reasons.
In general, it was controversial because it required the Japanese-Americans to swear loyalty to a government that had not treated them as it should have treated its citizens. It also put Japanese-Americans in a position of either (eventually) being sent back to Japan or, in essence, volunteering to go to war.
For the Wakatsuki family in specific, the loyalty oath was controversial because of what it might mean for the men in the family. The father was not happy with the idea of his son going off to fight for the government that had interned the family.