Discuss the measurement process and differentiate the levels of measurement, a classification system to help assess what is measured, and the two primary qualities that any assessment tool possess: reliability and validity. How can you improve reliability? Describe three commonly used scales researchers use: the Likert, Social Distance, and Guttmann scales.
When we measure things in the social sciences, we can use different levels of measurement. The levels of measurement are, in order, nominal, ordinal, interval, and ratio.
- Nominal is the lowest level of measurement. At this level, the values that we get simply “name” the things we are measuring. For example, if I have a survey and I ask people to circle their native country, I can assign numbers to each country, but that does not mean that there is any order or hierarchy in which the countries belong.
- Ordinal data are in order, but that is all the meaning that they have. A typical example of this is data on educational achievement. We might assign 0 to “less than high school,” 1 to “high school graduate,” 2 to “some college,” 3 to “college degree” and so on. These are in order of achievement, but they are the same distance apart. There is, for example, more difference between 0 and 1 than between 2 and 3.
- Interval data do have the same distance between values. That is, the difference between 1 and 2 is the same as the difference between 5 and 6. However, they are not as good as ratio data. For example, we cannot really say that a day that is 50 degrees Fahrenheit is 10 times warmer than a day that is 5 degrees.
- Ratio data is the best data of all. This is data that has all of the attributes of the previous three levels but can also be made into ratios. For example, we can say that a person who makes $80,000 per year makes twice as much money as someone who makes $40,000.
When we attempt to measure anything, we want to be sure that our test is both reliable and valid. Reliability in testing means that we would get the same result (or nearly the same result) every time that we conduct a test. Validity means that our test actually measures the thing that we mean for it to measure. For instance, let us think of IQ testing. A reliable IQ test would not say that a person has an IQ of 80 today and then yield a score of 120 for that same person (assuming they were taking the test seriously and that all variables that could affect their score were constant) the next day. A valid IQ test would be one that actually measures intelligence rather than measuring how much the person has learned.
There are many things that can be done to improve reliability. One thing is to increase the number of test items. If you give me one multiple choice problem, I might get it right by guessing. If you give me 10 questions, it will be hard for me to get a good score without actually knowing the subject. Another thing to do is to make the test questions clearer. For example, if I want to do opinion testing, I might not want to ask “do you approve of immigration” because respondents might not be sure whether I am asking about legal or illegal immigration. Responses from two people who have the same opinion might not be the same because they might interpret the question differently. The more specific the question, the more reliable the data.
Finally, we turn to the issue of the scales. A Likert scale is one that asks a group of questions, each of which offers an ordered range of answers. The classic example of this is a set of questions that each ask if you strongly disagree, somewhat disagree, are neutral, somewhat agree, or strongly agree with a particular statement about a topic. This is ordinal data and a set of such questions can gauge your opinions (and the intensity of your opinions) on a given issue. A Guttman scale is a series of questions, each of which has a yes or no answer these questions are ordered so that any person who says “no” to the first would automatically say “no” to the others. For example, you might be asked the following questions:
- Would you be willing to work with a person of a different race?
- Would you be willing to have lunch in the same room with a person of a different race?
- Would you be willing to have a person of a different race in your neighborhood?
- Would you be willing to socialize with a person of a different race?
If the questions are written properly, no one who said “no” to Question 1 would ever say “yes” to Question 4. A social distance scale is simply one example of a Guttman scale. It measures the degree to which people from different groups feel close to one another. The questions in my example would be examples of a social distance scale that measures feelings of closeness between races.