The main reason why we sometimes find it hard to practice cultural relativism is that we are innately somewhat ethnocentric. We have a hard time getting beyond the prejudices that we have developed through our years of being socialized.
Over the years, we are socialized to believe in what our societies believe. We are so immersed in our own society that we come to think that the way that it is set up is the most natural way for societies to be ordered. This is particularly true if we grow up without being exposed to other cultures.
Because of this, practicing cultural relativism can be difficult for us. Cultural relativism asks us to, in essence, forget about what our own culture tells us. Instead, we are told to evaluate the things that other people do on the basis of their own cultural assumptions. This is very hard because we have grown up to believe that our own ways are most natural and best.
Thus, we have difficulty practicing cultural relativism because we get too socialized into the ways and values of our own society and culture.