Doc Lobe Research Methods

Research starts with a clear question and precise terms. The same words (variable, validity, significance) come up in every course, proposal and defense. Search the glossary or filter it by topic. Each term has a plain definition and a student-life example.

Glossary of research terms

Levels of measurement decide your options

Level	What it means	Example	Typical statistics
Nominal	Categories with no order	Major, first-generation status	Counts, percentages, mode, chi-square
Ordinal	Ordered categories, unequal gaps	Class rank; “never / sometimes / often”	Median, percentiles, Mann–Whitney, Spearman
Interval	Equal gaps, no true zero	Temperature in °F; many scale scores	Mean, SD, t-tests, ANOVA, correlation
Ratio	Equal gaps and a true zero	Hours of sleep, reaction time	All of the above, plus ratios (“twice as long”)

A design is the plan for collecting data so that it can answer your question. Quantitative designs measure and count to test hypotheses. Qualitative designs study meaning, experience and process in depth. Mixed methods designs combine both on purpose. The right design follows from the question, not the other way around.

Designs, and why each one works

Match the question to the design

If your question starts with…	Consider
“Does X cause Y?”	True experiment; quasi-experiment if you can't randomize
“Is X related to Y?” / “Does X predict Y?”	Correlational or survey design; regression
“How much / how many / how common?”	Descriptive survey with a probability sample
“What is it like to…?”	Phenomenology
“How does this process unfold?”	Grounded theory
“How does this group make meaning?”	Ethnography
“Why did the numbers come out this way?”	Explanatory sequential mixed methods

Three things decide the test: the type of outcome (continuous, yes/no, or categories), your goal (compare groups or relate variables), and the structure of the data (how many groups, and whether the same people are measured more than once). Answer the questions below to get a recommendation.

Test chooser

Reference table

Test	Use it when	Example	Key assumptions	APA-style report

APA examples in rows 1–4 and 9 come from the worked examples on this page. Other rows show the reporting format only.

A test of significance asks: if there were truly no effect (H₀), how surprising would data like mine be? The p-value is that probability. If p is at or below your chosen α (usually .05), you reject H₀. A small p-value does not tell you the effect is large or important, so you also report an effect size and a confidence interval.

Six steps to check for significance

Type I and Type II errors

H₀ is actually true

H₀ is actually false

You reject H₀

Type I error (α)
A false alarm: you claim an effect that isn't there.

Correct (power = 1 − β)
You detect a real effect.

You fail to reject H₀

Correct
No effect, and you don't claim one.

Type II error (β)
A miss: a real effect goes undetected.

Power explorer

Two groups, two-tailed test (normal approximation). The grey curve is where the test statistic falls if H₀ is true; the teal curve is where it falls if the true effect is d. Power is the teal area past the critical line.

Effect size d 0.50 n per group 30 α (two-tailed)

H₀ distributionα (Type I error)β (Type II error)Power

Effect sizes: how big is it?

Measure	Small	Medium	Large
Cohen's d (two means)	0.20	0.50	0.80
Pearson r	.10	.30	.50
η² (ANOVA)	.01	.06	.14
Cohen's f²(regression)	.02	.15	.35

Benchmarks from Cohen (1988). They are rough guides; judge size against prior studies in your field (Lakens, 2013).

Confidence intervals

A 95% confidence interval gives a range of plausible values for the population effect: estimate ± critical value × SE.

If a 95% CI for a mean difference excludes 0, the two-tailed test at α = .05 is significant. The width shows precision: narrow intervals come from large samples and low variability.

Example: In the z-test example below, 22.1 ± 1.96 × 1.00 gives a 95% CI of [20.14, 24.06]. The value 20 lies outside it, which agrees with p = .036.

What p-values do and don't mean (ASA statement)

Principles from Wasserstein and Lazar (2016). See also Greenland et al. (2016) on common misinterpretations.

Use a z-test when…

You know the population standard deviation (σ) from a large prior source, such as national norms.
The population is normal, or the sample is large enough (n ≥ 30) for the Central Limit Theorem.
You are testing proportions with a large sample (one- or two-proportion z-test).

z = (x̄ − μ₀) / (σ / √n)

Use a t-test when…

σ is unknown and you estimate it with the sample SD (s). This is almost always the case in real studies.
Data are roughly normal, especially with small samples. With larger n, t is robust to moderate skew.
Pick the version by structure: one-sample, independent samples (two separate groups) or paired (same people twice, or matched pairs).

t = (x̄ − μ₀) / (s / √n), df = n − 1

Why t has fatter tails

Estimating σ with s adds uncertainty, so the t-distribution spreads more weight into the tails. As df grows, t gets closer to the normal (z) curve. That is why t critical values are larger than 1.96 for small samples.

Degrees of freedom 3

Normal (z)t with chosen df

Four worked examples, step by step

Correlation (r)

Measures how strongly two continuous variables move together, from −1 to +1. Use it when you only want to describe the association. It does not show cause, and it only captures straight-line relationships.

Simple linear regression

Fits the line ŷ = a + bx to predict an outcome from one predictor. The slope b is the expected change in y for each one-unit change in x. Use it when you want to predict, or to express the relationship in the outcome's units.

Multiple regression

ŷ = a + b₁x₁ + b₂x₂ + … Each slope is the effect of one predictor while holding the others constant. Use it to control for confounds and to compare predictors. Watch for multicollinearity (predictors that overlap heavily).

Logistic regression

Used when the outcome is yes/no (retained or not, passed or not). It models the log-odds of the outcome; results are reported as odds ratios. Linear regression would predict impossible probabilities below 0 or above 1.

Why linear regression works, and when it doesn't: check LINE

L

Linearity

The relationship is a straight line. Check a scatterplot and a residuals-vs-fitted plot.

I

Independence

Each observation is separate. Repeated measures or students nested in classes break this; use multilevel models.

N

Normal residuals

Residuals are roughly normal. Check a histogram or Q–Q plot. Matters most with small samples.

E

Equal variance

Residual spread is similar at every level of x (homoscedasticity). A funnel shape signals a problem.

Ordinary least squares chooses the line that makes the sum of squared residuals as small as possible. When LINE holds, the slope's standard error, t-test and confidence interval are trustworthy.

Worked example: do study hours predict exam scores?

Predict for study hours 7 Show residuals

hours (x): 2, 4, 5, 6, 8, 9, 10, 12, 13, 15
score (y): 60, 70, 64, 72, 69, 80, 74, 85, 78, 88

Means
x̄ = 8.4, ȳ = 74.0
Sums of products and squares
Sxy = Σ(x − x̄)(y − ȳ) = 302.0
Sxx = Σ(x − x̄)² = 158.4
Slope and intercept
b = Sxy / Sxx = 302 / 158.4 = 1.907
a = ȳ − b·x̄ = 74 − 1.907(8.4) = 57.98
Fit
r = .90, R² = .81 (81% of the variance in scores is explained by hours)
Test the slope (H₀: β = 0)
SE_b = 0.325, t = b / SE_b = 5.86, df = n − 2 = 8
critical t(.05, two-tailed) = 2.306 → p < .001
Predict
ŷ(7 hours) = 57.98 + 1.907(7) = 71.3

Conclusion. Study hours significantly predicted exam scores. Each extra hour was linked to about 1.9 more points.

b = 1.91, t(8) = 5.86, p < .001, R² = .81

3D model: multiple regression plane

With two predictors, the regression “line” becomes a plane. Each dot is a simulated student. Set the true effects, then watch least squares fit the plane. Vertical lines are residuals. Drag the model to rotate it.

Drag to rotate

True effect of study hours (b₁) 3.0 True effect of sleep hours (b₂) 2.0 Noise (SD) 6 Sample size 60

3D model: the t-distribution family

Each ribbon is a t-distribution with a different df, from 1 (front) to 30 (back). The dashed curve at the far back is the normal (z) curve. The highlighted curve shows your df, with the rejection region in red.

Drag to rotate

Highlight df 5 α Tails

p-value and critical-value calculator

Distribution df Test statistic Tails α

p-value areaCritical value(s)Your statistic

Standard normal (z) table

Each cell is the area to the left of z, Φ(z). Row gives z to one decimal; column adds the second decimal. For negative z use Φ(−z) = 1 − Φ(z). Click a cell to send it to the calculator.

t critical-value table

Each cell is the value of t that cuts off α in the upper tail. For a two-tailed test, use the column whose two-tailed α matches, and compare |t|. Click a cell to send it to the calculator.

How to use the tables, step by step

Finding a p-value from z = 2.10

Go to row 2.1 and column .00: Φ(2.10) = .9821.
Upper-tail area: 1 − .9821 = .0179.
Two-tailed: double it, p = .0357.
Compare with α = .05: p < α, so reject H₀.

Finding a critical t for df = 18

Two-tailed α = .05 puts .025 in each tail.
Go to row df = 18 and the column “one-tail .025 / two-tail .05”: 2.101.
Reject H₀ if |t| ≥ 2.101. Our t = −4.12, so reject.
If your df isn't listed, use the next smaller df (a slightly stricter cutoff).

Thesis (usually master's)

Shows you can carry out a focused study and apply existing methods. Often 40–100 pages; may replicate or extend prior work. Your program sets the exact format.

Dissertation (doctoral)

Makes an original contribution to knowledge. Usually longer, with a proposal defense, a committee and a final oral defense. Some programs allow a three-article format.

Twelve steps from topic to defense

The five-chapter model

Chapter	What goes in it

Ethics: the Belmont principles and the IRB

Respect for persons

Informed, voluntary consent. Extra protections for people with reduced autonomy.

Beneficence

Do no harm, and maximize benefits while minimizing risks.

Justice

Share the burdens and benefits of research fairly; don't select people just because they are easy to reach.

In the United States, research with human participants is governed by the Common Rule (45 CFR 46). You must get IRB approval before recruiting anyone or collecting data, including pilot data and surveys of classmates.

Reporting results in APA style (7th ed.)

Analysis	How to write it
Independent t-test	Workshop participants reported lower stress (M = 18.5, SD = 2.17) than the control group (M = 22.5, SD = 2.17), t(18) = −4.12, p < .001, d = −1.84.
Paired t-test	Anxiety dropped from pretest to posttest, t(7) = 5.29, p = .001, d_z = 1.87.
Regression	Study hours predicted exam scores, b = 1.91, t(8) = 5.86, p < .001, R² = .81.
General rules	Italicize statistical symbols. Report exact p-values to two or three decimals (use p < .001 for smaller). Drop the leading zero for values that can't exceed 1 (p, r, R²). Always include an effect size. Follow APA JARS for what to report in each section.

Source for this module. The twelve-week plan below follows the structure of Wendy Laura Belcher's Writing Your Journal Article in Twelve Weeks: A Guide to Academic Publishing Success (2nd ed., University of Chicago Press, 2019).

Week titles are quoted from the publisher's table of contents. The goals and tasks are short summaries written by Doc Lobe for teaching; they are not text from the book. © 2019 Wendy Laura Belcher. All rights reserved to the author and to the University of Chicago Press. The free workbook forms linked below are © Wendy Laura Belcher, all rights reserved, and are linked from her website rather than copied here. For the full method, examples and exercises, use the book: buy it from the University of Chicago Press or borrow it from your library.

Most first articles start as something you have already written: a seminar paper, a thesis or dissertation chapter, or a conference talk. Belcher's approach treats publishing as revising that draft on a steady schedule, a little each day, rather than waiting for a perfect idea. The plan runs twelve weeks, with extra chapters for starting from nothing (Week 0) and for handling the journal's decision (Week X).

The twelve-week plan

From thesis chapter to journal article

Thesis or dissertation	Journal article	What changes
Chapter 1: Introduction (15–30 pages)	Introduction (2–4 pages)	Open with the problem and your argument. Cut background the journal's readers already know.
Chapter 2: Literature review (30–60 pages)	Focused review within the introduction or a short section	Keep only the studies that set up your question and the gap you fill. Cite recent work from the target journal.
Chapter 3: Methodology	Method	Be concise but complete enough to replicate. Follow APA JARS for your design.
Chapter 4: Results (all research questions)	Results (often one or two questions)	One dissertation can become several articles, each built around one clear finding.
Chapter 5: Discussion	Discussion and conclusion	Lead with what the finding means for the field. Keep limitations honest and brief.

Page ranges are typical, not rules. Always check the target journal's word limit and author guidelines. Many programs require you to acknowledge that an article was adapted from your thesis.

Is this journal trustworthy? A quick check

Predatory journals charge fees without real peer review or editing. Answer these questions about a journal you are considering. The checklist is adapted from Think. Check. Submit.; also look the journal up in DOAJ if it is open access.

What happens after you submit

SubmitManuscript, cover letter, ORCID iD and required statements, uploaded through the journal's system.
Editor's screenThe editor checks scope, fit and basic quality. A desk rejection can happen here, often within days or weeks.
Peer reviewUsually at least two independent reviewers. This often takes one to three months or more.
DecisionAccept (rare the first time), minor revisions, major revisions (revise and resubmit) or reject.
Revise and respondMake the changes and write a point-by-point response letter. It may go back to the reviewers.
Accept, proofs, publishSign the author agreement, check the proofs carefully, and your article gets a DOI.

Rejection is common, even for experienced scholars. Use the reviews to improve the paper and send it to the next journal on your list.

What goes in a cover letter

Addressed to the editor by name, about one page
The manuscript title and article type
A few sentences on what you found and why it fits this journal's readers
A statement that the work is original and not under review elsewhere
Conflict-of-interest and ethics (IRB) statements, if the journal asks for them here
Your contact details

Writing the response to reviewers

Thank the editor and reviewers, briefly
Copy each comment, then answer it directly below
Say exactly what you changed and where (page and line)
If you disagree, explain why politely and with evidence
Make the reviewers' job easy; don't make them hunt for changes

Based on Noble (2017), Ten simple rules for writing a response to reviewers.

Example: one row of a response letter

Reviewer comment	Response	Change made
R2.3: The sample of 20 students seems small for the conclusions drawn.	Thank you. We agree the sample limits generalization. We now report the effect size with its 95% confidence interval and have softened the claims in the Discussion.	Results, p. 12, lines 240–246; Discussion, p. 16, lines 330–338; new limitation added on p. 18.

A made-up example for teaching.

Authorship, identity and your rights

Who counts as an author

Under ICMJE guidelines, each author must make a substantial contribution, draft or critically revise the work, approve the final version, and agree to be accountable for it. Agree on author order early, especially with your advisor.

Credit every role

The CRediT taxonomy names 14 contributor roles, such as conceptualization, formal analysis and writing (review and editing). Many journals ask you to list them.

Get an ORCID iD

A free, permanent ID that links you to all your publications, even if your name or school changes. Many journals require one at submission.

Know your rights

Read the copyright or license agreement before signing. The SPARC Author Addendum can help you keep rights such as posting your accepted manuscript. Check a journal's preprint and sharing policy in Jisc Open Policy Finder.

Research Methods Lab