As the other answers say, internal divisions were not the biggest obstacle to trade unionism. Corporate hostility to unions and the election of pro-corporate US presidents have created bigger obstacles to union progress.

While there were internal divisions between labor unions, and women and minorities certainly had issues with the male dominance of some of the larger unions, the various labor groups were all united around the ideas of collective bargaining and supporting workers' rights. The unions shared a similar view of the world and, despite quibbles and quarrels, would have coalesced around moving forward with the basic agenda of a better deal for the working person, such as higher wages, a minimum wage, more job security, and so on.

One of the largest obstacles, after the Civil War and through to today, were wealthy owners of corporations having a good deal of political clout. For example, during the Pullman Strike of 1894—caused when, during a financial depression, Mr. Pullman lowered wages but not the rent he charged workers for their housing—half a million workers across the US struck in solidarity with the Pullman employees. This was very frightening to the social classes in power, who couldn't make money if people didn't go to work. President Grover Cleveland actually brought in the US Army to break the strike. Actions such as these on the part of government had a dampening effect, to say the least, on the union movement.

These kinds of union-breaking activities have gone on regularly since then. A more recent example would be President Reagan firing more than 11,000 striking air traffic control workers in 1981 when their union called a strike. That also was a crushing blow.

Business owners have very good reason to oppose unions, as unions curtail their power. One can hardly blame them for protecting their interests. Therefore, another way these very wealthy and powerful people make union activity difficult is by creating a social climate through the media in which ordinary people are made to doubt and feel uncomfortable about supporting unions. They are portrayed as groups that shield shirkers by insisting on the importance of seniority, destroy jobs, and undermine the work ethic. People need to think and do research on their own to decide if unions will help them or hurt them.

The time frame of the question is rather large. In the early days there was a considerable rift between skilled labor and unskilled labor. Many of the craftsmen worried that letting in unskilled labor into unions would diminish their power. Also, whenever union strikes were violent meant bad publicity. A railroad strike may obtain sympathy in the press until there is violence, at which point management has the upper hand. Many skilled labor unions also were somewhat xenophobic towards letting in workers from Southern Europe.

The largest obstacle to union membership during this time frame was that a union of workers seems too closely related to communism for several Americans. People have less power as individuals in a union and they lose their own ability to negotiate on their own. During periods of social turmoil such as the late 1800's and the 1920's unions were perceived to be hotbeds of communist thought. People believed that unions would undo the American free enterprise system and, especially in the 1920's, replace it with a Soviet-style governing system. Even as late as 1992 some people thought that unions were too leftist in principle and that they had outlived their usefulness. Since the end of the Great Depression, the United States has banned child labor, promoted safe working conditions, and guaranteed a minimum wage. Of course, all of this would not be possible if not for unions.

Internal divisions were partly responsible for the struggles of the American labor movement, but they were not the most important obstacle.

Internal divisions did matter.  There were divisions between unions that wanted to fix capitalism and those that wanted to do away with capitalism.  There were divisions between unions that wanted to represent all workers and those who wanted to represent only skilled workers.  There were divisions between those who would have represented women and African Americans and those who would not.  All of these divisions helped to weaken the labor movement.

But the biggest obstacle to unions was and is the general suspicion of unions in the American mind.  Americans tend to believe more in the idea of individualism than in the idea of class solidarity.  They tend to see unions as in some way anti-American because they represent a move away from the ideal of the individual worker making his or her own way in the world.

It is this distrust of unions, more than a lack of unity in the labor movement, that was and is an obstacle to labor power in the US.