To the extent that the Populists were considered to be a viable third party, it was because they had a large constituency whose needs were not really being addressed by either of the major political parties. The Populists' power was based in the agricultural vote. At that point in American history, farmers made up a much larger percentage of the population, particularly in the South. Even so, their concerns were not being met by either party before the Populists arose. The Republicans were generally the party of big business and the Democrats were not receptive to Populist ideas such as the idea of "subtreasuries."
Since the Populists could draw on a relatively large and unserved constituency, they were somewhat viable as a third party.