In the rise of the Greek polis, what were the strict barriers to the extension of citizenship that prevented any one city state from becoming dominant?
It may be important here to examine the assumptions underlying the question. The first assumption is that poleis all had the same political systems and rules concerning who might be a citizen. While it is true across all poleis that women and slaves were excluded, and that there was a minimum age for citizenship, there was significant variation otherwise. Moreover, since the limitation of citizenship to adult males applied across all poleis, it would not have affected their relative strength. Athens had among the most restrictive citizenship laws of the Greek poleis, requiring grandparents to have been citizens, and yet was one of the most powerful cities of Greece.
That brings up the second point, that restrictive citizenship laws would not necessarily hinder the dominance of one polis over the others. Rome had quite restrictive citizenship laws until the Severan reforms, and nonetheless managed to run a large empire quite efficiently.
As to why the Greek city states did not unify into one nation, I doubt one can pinpoint a single cause. Unless the poleis were being attacked by an external enemy (Persia) and needed to band together for defense, was there any reason why they should unite to form a single political entity? Other than when threatened by Persia or Macedonia, panhellenism didn't seem particularly popular an ideology until the Hellenistic period (after the Greek poleis were forcibly united by Philip and then Rome).
Moreover, when one powerful city (Athens, Sparta, Thebes) attempted to dominate the rest, that would merely ally other powerful cities against the one with imperial pretensions.