Though there are some reasons to think the poem might take place in winter (Warren says "In winter he comes back to us," pointing to Silas's penchant for missing the important harvest season and showing up in the winter when there is little work), other factors could lead us to...
Though there are some reasons to think the poem might take place in winter (Warren says "In winter he comes back to us," pointing to Silas's penchant for missing the important harvest season and showing up in the winter when there is little work), other factors could lead us to believe this isn't the case.
At the beginning of the poem, Warren comes home, and Mary runs to meet him at the door. She pushes him out the door and they sit down on the steps of the house. Since they are sitting outside, and this is likely taking place in New England, this wouldn't be at all comfortable if it was winter. Warren says Silas deserted them "last haying." This is ambiguous. It could mean the previous planting season, so this could be in the springtime, or it could mean the recently finished planting season, so this could be late fall. As Mary sits on the steps, she plays with the "harp-like morning glory strings." Morning glories are annual vines. If the events of the poem happen in spring, the morning glory vines would not have grown up enough yet for Mary to be playing with them. The jobs Silas is offering to do are off-season jobs: ditching the pasture and clearing land. These jobs would be done in late fall or winter when the jobs of planting, plowing, and harvesting don't consume all of a farmer's attention. Another clue is that Warren returns from a shopping trip after dark. Sun sets earlier in the fall and winter, but in New England the daylight hours would extend to 9:00 at night, so Warren might have returned from such a trip before the moon and stars were out if the poem were set in summertime.
We can say from the clues in the poem that Silas probably doesn't return in winter or summer. It seems likely that the events of the poem take place in the fall after the final harvest--perhaps in October.