Home
Doc LobeTeaching, Learning & Research Firm
Lesson · Research and program evaluation

Program Evaluation

Is our program working, for whom, and why? Program evaluation helps counselors, schools and agencies answer these questions with data. This lesson covers needs assessment, logic models, formative and summative evaluation, outcome measures, implementation frameworks such as RE-AIM, and culturally responsive evaluation that is done with communities, not to them.

Types of evaluationLogic model builderRE-AIMCulturally responsive evaluationLesson plan20-question check
Start here

Why evaluate counseling programs?

Research asks what is generally true. Program evaluation asks whether a specific program is meeting its goals in a specific place, so leaders can improve it, report to funders and make fair decisions about resources.

Improve

Find what works, what does not and what to change while the program is running.

Prove

Show stakeholders and funders the results, including for accreditation such as CACREP or ASCA's RAMP.

Protect

Catch harm and gaps early, such as a group that helps most students but not English learners.

Share power

Give participants a voice in what counts as success.

Know the terms

Types of evaluation

Practice

Which evaluation is it?

Sort each question into the type of evaluation that answers it.

Build it

Logic model builder

A logic model is a one-page map of how a program is supposed to work: resources lead to activities, which lead to outputs and then to short-, medium- and long-term outcomes. Build one for a program you know, or load the example.

What to measure

Outputs, outcomes and good measures

Outputs vs. outcomes

  • Outputs count what the program did: 12 sessions held, 40 students served.
  • Outcomes describe what changed for people: attendance rose, anxiety scores fell.
  • Funders increasingly ask for outcomes, not just outputs.

Choosing measures

  • Use valid, reliable measures normed for your population (see the Psychometrics Lab).
  • Mix quantitative data (scores, attendance, grades) with qualitative data (interviews, focus groups).
  • Collect baseline data before the program starts.

Designs

  • Pre–post designs show change but not cause.
  • Comparison groups and randomized designs strengthen causal claims.
  • Single-case (AB, ABAB) designs suit individual clients. See the Research Methods Lab.
Implementation

RE-AIM: will it work in the real world?

RE-AIM (Glasgow and colleagues) asks five questions about whether a program reaches people and lasts. Tap each card.

Equity

Culturally responsive evaluation

Culturally responsive evaluation (Hood, Hopson and Frierson) centers the culture and context of the people a program serves. It treats community members as partners in deciding questions, methods and what counts as success.

Create with, not for

Build the theory of change and logic model with community members so outcomes reflect what matters to them (Meyer, Louder and Nicolas, 2022).

Disaggregate data

Look at results by race, language, disability, gender and income. An average can hide groups the program is not reaching.

Share findings back

Report results to participants in accessible language and formats, and act on what they say.

Respect sovereignty and consent

Tribal nations and communities may own their data. Follow community review processes in addition to IRB requirements.

Watch and read

Videos and recent research

Videos and peer-reviewed articles published in 2016 or later. Links need an internet connection.

Videos

Social experiments to fight povertyEsther Duflo · TED2010

How randomized evaluations show which programs actually help.
The best stats you've ever seenHans Rosling · TED2006

Making data clear and memorable for the people who use it.
What doctors don't know about the drugs they prescribeBen Goldacre · TEDMED 2012

Why reporting every result, including negative ones, matters.

Peer-reviewed articles

  • Meyer, M. L., Louder, C. N., & Nicolas, G. (2022). Creating with, not for people: Theory of change and logic models for culturally responsive community-based intervention. American Journal of Evaluation, 43(3), 378–393.A step-by-step process for building logic models with diverse and immigrant communities.https://doi.org/10.1177/10982140211016059
  • Glasgow, R. E., Harden, S. M., Gaglio, B., Rabin, B., Smith, M. L., Porter, G. C., Ory, M. G., & Estabrooks, P. A. (2019). RE-AIM planning and evaluation framework: Adapting to new science and practice with a 20-year review. Frontiers in Public Health, 7, 64.How to use RE-AIM to plan and evaluate programs.https://doi.org/10.3389/fpubh.2019.00064
  • Damschroder, L. J., Reardon, C. M., Opra Widerquist, M. A., & Lowery, J. (2022). The updated Consolidated Framework for Implementation Research based on user feedback. Implementation Science, 17, 75.A widely used framework for understanding why programs succeed or fail in real settings.https://doi.org/10.1186/s13012-022-01245-0
  • Lewis, C. C., Boyd, M., Puspitasari, A., Navarro, E., Howard, J., Kassab, H., Hoffman, M., Scott, K., Lyon, A., Douglas, S., Simon, G., & Kroenke, K. (2019). Implementing measurement-based care in behavioral health: A review. JAMA Psychiatry, 76(3), 324–335.Using routine outcome data to improve services.https://doi.org/10.1001/jamapsychiatry.2018.3329
For instructors

Lesson plan preview: Prove it and improve it

A ready-to-teach lesson where teams build a logic model and evaluation plan for a real program.

Quick check

Test your understanding

Twenty questions. More in Counselor Exam Prep.

Sources

References

  1. American School Counselor Association. (2019). The ASCA national model: A framework for school counseling programs (4th ed.).
  2. Centers for Disease Control and Prevention. (2024). CDC program evaluation framework. U.S. Department of Health and Human Services.
  3. Damschroder, L. J., et al. (2022). The updated Consolidated Framework for Implementation Research based on user feedback. Implementation Science, 17, 75.
  4. Glasgow, R. E., et al. (2019). RE-AIM planning and evaluation framework. Frontiers in Public Health, 7, 64.
  5. Hood, S., Hopson, R. K., & Frierson, H. T. (Eds.). (2015). Continuing the journey to reposition culture and cultural context in evaluation theory and practice. Information Age Publishing.
  6. Meyer, M. L., Louder, C. N., & Nicolas, G. (2022). Creating with, not for people. American Journal of Evaluation, 43(3), 378–393.
  7. Royse, D., Thyer, B. A., & Padgett, D. K. (2016). Program evaluation: An introduction to an evidence-based approach (6th ed.). Cengage.
  8. W. K. Kellogg Foundation. (2004). Logic model development guide.