Validity

Validity in scientific investigation means measuring what you claim to be measuring.

Validity is difficult to assess and has many dimensions. The following general categories of validity can help structure its assessment:

Internal validity

This is about the validity of results within, or internal to, a study. It usually concerns causality, i.e. the strength of assigning causes to outcomes. For laboratory experiments with tightly controlled conditions, it is usually easy to achieve high internal validity. For studies in difficult to control environments, e.g. health services research, it can be difficult to claim high internal validity. When you claim high internal validity you are saying that in your study, you can assign causes to effects unambiguously. Randomisation is a powerful tool for increasing internal validity - see confounding.

In the context of questionnaires the term criterion validity is used to mean the extent to which items on a questionnaire are actually measuring the real-world states or events that they are intended to measure. This type of internal validity could be assessed by comparing questionnaire responses with objective measures of the states or events to which they refer; for example comparing the self-reported amount of cigarette smoking with some objective measure such as cotinine levels in breath.

External validity

This is about the validity of applying your study conclusions outside, or external to, the setting of your study. Another term for this is generalisability. Sometimes this is obvious, for example a public opinion poll taken at the entrance to a football match would not be properly representative of the general population. Often it is less obvious, for example a study in medical settings on a Monday morning will not be representative of the pattern of illnesses seen at other times of the week. A key to improving external validity is to understand the setting thoroughly before you embark upon the study.

Construct validity

A construct is a concept. A clearly specified research question should lead to a definition of study aim and objectives that set out the construct and how it will be measured. Increasing the number of different measures in a study will increase construct validity provided that the measures are measuring the same construct

In the context of questionnaires the term content validity is used to mean the extent to which items on a questionnaire adequately cover the construct being studied. A related, but somewhat confusing, term in questionnaire methodology is factorial validity, which refers to the clustering of correlations of responses by groupings of items in the questionnaire. Factor analysis can be used for this purpose. Basically, the groupings must make intuitive sense to the investigator otherwise the questionnaire has poor factorial validity.