Anyone who is different is automatically on the other team--as in "us" and "them." Until these differences are somehow minimized, they are difficult for most of us to deal with. Learning to see beyond a missing limb, different skin color, size, foods, traditions, behaviors, beliefs, or whatever other separator might exist is easier for some than others, but it's ultimately a choice.
In my opinion, it is the fact that different cultures have very different ideas about what is normal and what is not. Because of that, you can end up making big mistakes without having any idea you are doing so.
For example, in the culture where I grew up, it is considered very impolite to look other people in the eye. This is especially true if they are your "superior." Here in the US, however, not looking a superior in the eye is seen as disrespectful or evasive. Therefore, a person who grew up where I did can give offense to a superior in the US without meaning to. This kind of thing makes dealing with people from other cultures tricky.