This is a great question. There are several reasons why the Roman army became great in numbers and not the Greek city-states.
First, as you ask in your question, the Greeks did not develop an empire. They had city-states. So, these cities were much smaller than the Roman Empire. Let me give you a sense of scale. Plato believed that the perfect sized Greek city was comprised of five thousand men. This is tiny compared the the Roman Empire. So, the most important point is that we dealing with a different scale.
Second, the Romans assimlated people, whereas the Greeks did not. If you read even the first chapters of Livy, one of the great Roman historians, you will see that the Romans assimilated the Sabines, the Etruscans, and other people fairly easily. This swelled their numbers and land in time.
These two reasons are the most important and make all the difference.
Perhaps the biggest reason for the difference lies in the Roman and Greek mentalities. For Romans, they were all citizens of and fighting for the same place (Rome). While there may have been class distinctions, at the end of the day they were all Romans, all favored the same deities, etc. The Greeks, on the other hand, were divided because of their city-states. Often, the city-states didn't get along with each other and each had its own patron deity. As a result, uniting Greeks beneath an arching banner was difficult because they viewed themselves more as an Athenian, Spartan, etc. than as a Greek. Additionally, not all of the city-states excelled at standard combat. The Spartans were known for their prowess in hand-to-hand combat, whereas the Athenians excelled at naval combat. Romans were taught how to battle from a young age, and Roman soldiers were generally very well received by the populace.