Key ideas
What makes a measure trustworthy
Psychometrics is the science of measuring psychological qualities such as anxiety, belonging or conscientiousness. Because these cannot be seen directly, researchers build instruments and then gather evidence that the scores are consistent and mean what they claim to mean.
Step 1Constructs and operational definitions
A construct is the idea being measured, such as "academic stress." An operational definition states exactly how it will be measured, such as total scores on the Perceived Stress Scale. A clear construct definition guides every later choice.
ConsistencyReliability
- Internal consistency: do the items hang together? (Cronbach's alpha, McDonald's omega)
- Test-retest: are scores stable over time when the trait should be stable?
- Inter-rater: do two trained observers agree? (Cohen's kappa, ICC)
Reliability is a property of scores in a particular sample, so report it for your own data.
MeaningValidity
- Content: do the items cover the whole construct?
- Construct: do scores relate to similar measures (convergent) and not to unrelated ones (discriminant)? Does the factor structure fit?
- Criterion: do scores predict a real outcome now (concurrent) or later (predictive)?
Validity is an argument built from many kinds of evidence (Messick, 1995).
ScoresScoring, norms and cutoffs
Follow the developer's scoring rules exactly, including reverse-scored items (on a 1–5 scale, a reversed 2 becomes 4). Norms compare a score with a reference group; cutoffs flag scores worth a closer look. A cutoff from one population may not fit another.
FairnessMeasurement across cultures
A test can work differently across languages and cultures. Good practice includes careful translation and back-translation with community review (International Test Commission, 2017), checking measurement invariance before comparing group means, and looking for items with differential item functioning. Also ask whether the construct itself means the same thing in each culture.
EthicsUsing tests responsibly
- A screening score is not a diagnosis; it signals who may need a full evaluation.
- Get informed consent, protect confidentiality and store data securely.
- Use instruments within your training; some require a licensed professional.
- Have a referral and safety plan before asking about suicide, trauma or abuse.
See the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 2014).