Explain the context "everywhere we went I had trouble of making which was I,the one walking at my side,the one walking in my pants"
This line is taken from White's essay "Once More to the Lake." In this essay, he is talking about how he, as an adult, feels when he brings his son to a lake where he had really enjoyed going on vacation. This had been his favorite place when he was a kid.
As he brings his son around the various places, White has a hard time separating himself from his son -- he keeps seeing thing through the eyes of his son. He can feel the emotions that he himself had as a kid. When he does that, he feels like he is part of his son.
This is the context for the line you cite -- White is walking along, thinking about the way things had been when he was a child, and he has a hard time remembering which person is really him -- is he the adult walking along or is he the child?