Idea of Construct Validity

Construct validity refers to the degree to which inferences can legitimately be made from the operationalizations in your study to the theoretical constructs on which those operationalizations were based. I find that it helps me to divide the issues into two broad territories that I call the “land of theory” and the “land of observation.” The land of theory is what goes on inside your mind, and your attempt to explain or articulate this to others. It is all of the ideas, theories, hunches and hypotheses that you have about the world. In the land of theory you will find your idea of the program or treatment as it should be. You will find the idea or construct of the outcomes or measures that you believe you are trying to affect. The land of observation consists of what you see happening in the world around you and the public manifestations of that world. In the land of observation you will find your actual program or treatment, and your actual measures or observational procedures. Presumably, you have constructed the land of observation based on your theories. You developed the program to reflect the kind of program you had in mind. You created the measures to get at what you wanted to get at.

Construct validity is an assessment of how well you translated your ideas or theories into actual programs or measures. Why is this important? Because when you think about the world or talk about it with others (land of theory) you are using words that represent concepts. If you tell someone that a special type of math tutoring will help their child do better in math, you are communicating at the level of concepts or constructs. You aren’t describing in operational detail the specific things that the tutor will do with their child. You aren’t describing the specific questions that will be on the math test that their child will do better on. You are talking in general terms, using constructs. If you based your recommendation on research that showed that the special type of tutoring improved children’ math scores, you would want to be sure that the type of tutoring you are referring to is the same as what that study implemented and that the type of outcome you’re saying should occur was the type they measured in their study. Otherwise, you would be mislabeling or misrepresenting the research. In this sense, construct validity can be viewed as a “truth in labeling” kind of issue.

There really are two broad ways of looking at the idea of construct validity. I’ll call the first the “definitionalist” perspective because it essentially holds that the way to assure construct validity is to define the construct so precisely that you can operationalize it in a straightforward manner. In a definitionalist view, you have either operationalized the construct correctly or you haven’t – it’s an either/or type of thinking. Either this program is a “Type A Tutoring Program” or it isn’t. Either you’re measuring self esteem or you aren’t.

The other perspective I’d call “relationalist.” To a relationalist, things are not either/or or black-and-white – concepts are more or less related to each other. The meaning of terms or constructs differs relatively, not absolutely. The program in your study might be a “Type A Tutoring Program” in some ways, while in others it is not. It might be more that type of program than another program. Your measure might be capturing a lot of the construct of self esteem, but it may not capture all of it. There may be another measure that is closer to the construct of self esteem than yours is. Relationalism suggests that meaning changes gradually. It rejects the idea that we can rely on operational definitions as the basis for construct definition.

To get a clearer idea of this distinction, you might think about how the law approaches the construct of “truth.” Most of you have heard the standard oath that a witness in a U.S. court is expected to swear. They are to tell “the truth, the whole truth and nothing but the truth.” What does this mean? If we only had them swear to tell the truth, they might choose to interpret that as “make sure that what you say is true.” But that wouldn’t guarantee that they would tell everything they knew to be true. They might leave some important things out. They would still be telling the truth. They just wouldn’t be telling everything. On the other hand, they are asked to tell “nothing but the truth.” This suggests that we can say simply that Statement X is true and Statement Y is not true.

Now, let’s see how this oath translates into a measurement and construct validity context. For instance, we might want our measure to reflect “the construct, the whole construct, and nothing but the construct.” What does this mean? Let’s assume that we have five distinct concepts that are all conceptually related to each other – self esteem, self worth, self disclosure, self confidence, and openness. Most people would say that these concepts are similar, although they can be distinguished from each other. If we were trying to develop a measure of self esteem, what would it mean to measure “self esteem, all of self esteem, and nothing but self esteem?” If the concept of self esteem overlaps with the others, how could we possibly measure all of it (that would presumably include the part that overlaps with others) and nothing but it? We couldn’t! If you believe that meaning is relational in nature – that some concepts are “closer” in meaning than others – then the legal model discussed here does not work well as a model for construct validity.

In fact, we will see that most social research methodologists have (whether they’ve thought about it or not!) rejected the definitionalist perspective in favor of a relationalist one. In order to establish construct validity you have to meet the following conditions:

• You have to set the construct you want to operationalize (e.g. self esteem) within a semantic net (or “net of meaning”). This means that you have to tell us what your construct is more or less similar to in meaning.
• You need to be able to provide direct evidence that you control the operationalization of the construct – that your operationalizations look like what they should theoretically look like. If you are trying to measure self esteem, you have to be able to explain why you operationalized the questions the way you did. If all of your questions are addition problems, how can you argue that your measure reflects self esteem and not adding ability?
• You have to provide evidence that your data support your theoretical view of the relations among constructs. If you believe that self esteem is closer in meaning to self worth than it is to anxiety, you should be able to show that measures of self esteem are more highly correlated with measures of self worth than with ones of anxiety.