The most important way in which they were distinct from Rome was in the identity of their rulers. Though they kept some of the same governmental structures, the people at the top of the pyramid were not Roman. Instead, they were people (except in the Byzantine Empire) from Germanic tribes.
The Germanic rulers also brought in some legal traditions that differed from Roman law. The Frankish Kingdom, for example, brought in Germanic law, as seen in such ideas as that of wergeld. These kingdoms also started the move towards feudalism. In the Frankish Kingdom, the Frankish ruling class intermarried with the remaining Roman senatorial class. These families then came to have enough power that they could compete with the kings to some degree. This was very different than how the Roman Empire had been.
While these new kingdoms kept many aspects of Roman rule, they had Germanic rulers and were gradually moving away from Roman law.