This really is a great question. There were many fundamental differences between the Greeks and Romans. First, the Greek poleis were much smaller and hence they did not have the infrastructure in place to rule a vast empire. For instance, many Greek thinkers like Plato believed that the ideal city should only have around five thousand male citizens. Second, the Greeks prized citizenship so much that they rarely integrated people. In other words, they did not want to make new citizens. Third, a city like Athens was a radical democracy, where all people had a say. Hence, it was not that effective.

The Romans, on the other hand, were imperialistic. First, they integrated people rather easily. They even gave citizenship to many people with little qualms. Second, their political structure gave power to individuals to rule large areas. The Romans called this imperium. So, consuls (the highest office in Rome) had great power over decision such as warfare. This made governance much easier. Third, the Romans had officials (again with imperium) to govern provinces. All of these officials would report back to Rome. Hence, there was a centralized government, but also local magistrates that would rule on behalf of Rome. Finally, the Romans had massive armies all over the place to enforce their rule.