The Byzantine Empire began as a part of the Roman Empire. Then, over time, the Byzantine Empire gradually split away from the Roman Empire, developing its own culture and institutions.
The Byzantine Empire can be said to have started in 286 AD when the Emperor Diocletian split the Roman Empire up into four regions for administrative purposes. This sort of started the idea of a divided empire.
Later, in 330, Constantine founded the city of Constantinople (later to be called Byzantium) and set it up as the capital of the whole Roman Empire. After the death of the Emperor Theodosius in 395 AD, the empire was formally split between his two sons -- one ruling from Rome and one from Constainople.
Over time, the Eastern Empire became more Greek and therefore more distinct from the Western Empire. When the Western Roman Empire fell, only the Eastern Empire was left and it came to be known as the Byzantine Empire.