We are not told a whole lot about Mr. Jagger's house, but Pip does go there for dinner in Chapter 26. At that point, we are told that Jagger's house is large and pretty nice on the inside in terms of what possessions the man has. On the outside, it is not terribly well kept up. It badly needs to be painted and the windows have not been washed in a while.
On the inside, Jagger only uses three of the rooms on the first floor. The whole insde of the house is pretty gloomy. The rooms are dark brown.
To prove any of this, just look at Chapter 26, starting with the 3rd paragraph.