The three men let the boy and his father pass because the father is armed.
The father realizes that he has to protect his son at all costs. The post-disaster world is surrounded with danger, and most of it is from sub-human cannibals, looters, and roving bands. The father carries the gun because it is his only form of protection, and the only civilization.
The boy and his father encounter the three men as they enter a small town.
Emaciated, clothed in rags. Holding lengths of pipe. What have you got in the basket? He leveled the pistol at them. (p. 185)
The men let them pass because a gun trumps a pipe. If they were armed, they would have robbed and possibly killed the man and boy since they outnumbered them. But since they are emaciated and have only pipes, they are forced to let them pass.