Yes, and no. First, when we say Congress members, for the House of Representatives we're talking about two year terms, which means they more or less perpetually are running for re-election. That being said, the vast majority of these seats are safely Republican or Democrat, and very little campaigning often needs to be done in order to maintain the seat. Senators have a longer term and are therefore able to focus on the business of government for a longer period.
I'll say "no" to this. Yes, they do spend a lot of time in their districts. However, is that really a bad thing? After all, these people are supposed to be representing the people in their districts. Therefore, they need to go and spend time in the districts meeting people and getting a feeling for the attitudes of their constituents. So I would say that spending a lot of time in their districts actually makes them more able to represent their constituents.