The Difference Between Reliability and Validity

In research design, two important concepts are reliability and validity. During this political campaign season, these two constructs are being mangled constantly. Here is a simplified look at these two variables.

A concept is deemed reliable when one person can use the methodology and then communicate it well enough to another person to follow the same procedure. It means that there is commonality in procedure. For example, one laboratory is using the same steps as another lab. Therefore, if they were doing the same experiment, the results should be comparable statistically.

Validity is a separate concept from reliability. Simply, validity means that the concept being examined really is what is being assessed. For example, if one is measuring air pollution, then it is necessary to define what the pollutants are that are being measured and the procedure actually measures it. With something that is defined easily, such as pollution, then there is some universal agreement that a variable is something that is an unwanted contaminant. There is agreement on what is a pollutant. Other concepts, like racism for example, are far more difficult to assess and measure.

Now, here is some confusion when using the concepts of reliability and validity. - Something can be reliable but not valid.

Here is a simple example to illustrate this. Suppose that you have bathroom weight scales and these weight scales are broken. The weight scales will represent the methodology. One person weighs you with these scales and obtains a result. Then, the weight scales are passed along to another person. The second person follows the same procedure, uses the same weight scales and weighs you. The same broken weigh scales are used. The two people, using the same broken weight scales, come to similar measures. The results are reliable. The results are obtained by two (or perhaps more) people using the faulty scale. Although the results are reliable, they may not be valid. That is, by using the faulty scales, the results are not a true indicator of the real weight.

In this example, validity would be measuring your weight compared to accepted weights, such as on a balance scale. Then the measure of weight is certain to be actually what the person weighs.

The goal of research design is to have a procedure (and its results) be both reliable and valid.

*** I hope that clear that matter somewhat. Thanks for reading this far. Please feel free to re-read this posting if you have any bouts of insomnia.

Catherine Forsythe