Garson G.D. Validity and Reliability

New York: Statistical Associates Publishing, 2013. — 111 p.A study is valid if its measures actually measure what they claim to, and if there are no logical errors in drawing conclusions from the data. There are a great many labels for different types of validity, but they all have to do with threats and biases which would undermine the meaningfulness of research. Researchers disagree on the definitions and types, which overlap. The typology is much less important than understanding the types of questions the researcher should ask about the validity of research.
The question of validity arises in three contexts.1. At the level of the item or measure, the researcher is concerned with the content itself (face validity), unidimensionality (ex., avoiding double-headed items like the agree/disagree item, “The head of my organization is inspiring and effective.”), and whether the item correlates as expected
(criterion validity)
2. At the level of the construct, the researcher is concerned with whether the indicator measures cohere well (convergent validity) and are differentiated from indicator measures for other constructs in the model (divergent validity). Also of concern is whether the construct correlates as expected (criterion validity) and does not reflect an instrumentation artifact (common method bias).3. At the level of the study, the researcher is concerned with whether statistical procedures have been applied properly (statistical validity) and
whether research design is sound (internal validity) and whether generalizations are appropriate (external validity).
Reliability, discussed further below, is the correlation of an item, scale, or instrument with a hypothetical one which truly measures what it is supposed to.