In a symbolic sense, the crumbling decay of the Usher home seems to represent the ever-crumbling home life suffered by the author. The unnamed narrator says of the house,
I looked upon the scene before me—upon the mere house, and the simple landscape features of the domain — upon the bleak walls—upon the vacant eye-like windows — upon a few rank sedges—and upon a few white trunks of decayed trees — with an utter depression of soul which I can compare to no earthly sensation more properly than to the after-dream of the reveller upon opium — the bitter lapse into every-day life — the hideous dropping off of the veil.
The house is in absolute decay and produces within the viewer a terrible sense of depression, hopelessness, and even confusion. This may bear some similarities to Poe's own life. Poe's father first abandoned his family, leaving Poe's mother all alone with three children. Then, by the time Poe was three, his mother died of tuberculosis while her very young son watched. After this, the Poe children were split up, and Edgar was sent to live with the family of his mother's good friend, Frances Allan; Poe loved Frances a great deal. However, she soon died of tuberculosis as well, leaving Poe alone with his foster father, a hard man who always thought Poe was an ungrateful burden. Poe's home life crumbled over and over again, and the crumbling of the literal house of the Ushers could be seen to symbolize this.
Further, the unnamed narrator of the story tells us "that the entire [Usher] family lay in the direct line of descent." In other words, then, the Usher family seems to have routinely practiced incest, and it is, perhaps for this reason that the Ushers suffer from such strange illnesses. Poe, in real life, married his cousin, Virginia, when she was only thirteen years old. He even referred to her as "sissy" as often as he referred to her as "wifey," appearing to combine several female roles into one and ascribing them all to Virginia. By all accounts, their relationship was loving, but she was a child and his relative, and this is quite troubling as well.