The village elders are somewhat uneasy about the arrival of Song-sam. This is the aftermath of a brutal war, and so they're naturally apprehensive about what will happen next. As a figure of civil authority, Song-sam has, to some extent, usurped the traditional authority previously enjoyed by the village elders. And because he, unlike them, has the power of armed force to back him up, the old men of the village wisely decide that it's best to stay out of his way for the time being.

We know this because the elders only occasionally pass by Song-sam as he enters the village. Their authority has now gone, and so it would seem that they've reluctantly accepted that the future of the village and its people is now completely in the hands of the civil authorities, as represented by Song-sam. One should also note that the village elders' faces, like those of the children, are etched with fear.

Furthermore, it's notable that Song-sam hasn't met with many village elders. This could indicate a quiet refusal on their part to accept the new order of things as it will diminish their power and influence within the village. Those few elders who have met Song-sam could've done so in order to curry favor with the new administration, and trying not to be completely marginalized.