From the perspective of the President, the rules of engagement in Vietnam had to line up with the ideas of limited war. The US needed to conduct the war so that it could, hopefully, achieve all of the following goals. First, it had to win the “hearts and minds” of the Vietnamese. Second, it had to prevent any foreign countries from entering the war. Finally, it had to retain the sympathy and approval of its allies.
For these reasons, President Johnson imposed somewhat limited rules of engagement. He had to ensure, for example, that American soldiers would be restricted in the types of targets they could engage. They had to (ideally, at least) be sure not to kill civilians, thus pushing more people to sympathize with the communists. President Johnson also had to ensure that China, in particular, would not feel threatened. This was one reason that he would not agree to do things like invading the North. The US had made China feel threatened in the Korean War with negative consequences. Therefore, in this war, the rules of engagement did not allow incursions into North Vietnam. This was also, in part, due to the need to look good in the eyes of foreign observers. The US could not afford to look aggressive in this war.
Thus, from Johnson’s perspective, rules of engagement needed to be rather restrictive so as to ensure that the goals of limited warfare could (hopefully) be accomplished.