For most of the time that I’ve been working in libraries, I’ve been teaching. At first, I was creating research guides, leading building tours, demonstrating online searching. Later, both in the classroom and as a trainer, I’ve taught a ton of in-person classes, facilitated workshops, and created online tutorials. And so it may come as a surprise that I didn’t have a model for evaluating if all my effort had any impact on learning. In fact, as I review the various books about teaching in higher education and in libraries that I happen to have right now in my office, I see very little mention of ways to evaluate instruction.
When I decided to get serious about my own skills as a trainer and teacher, I finally came across the 4 level model of evaluation created by Donald Kirkpatrick. The model provides strategies for evaluating your instruction at four levels: reaction, learning, behavior, and results. I thought I’d share it here with you in the hopes that you will find that it helps you in your own library instruction.
Will Kirkpatrick’s Four Levels of Evaluation Model Work For Me in My Library?
The answer is yes! If you find yourself wondering if your instruction or program is actually having an impact, Kirkpatrick’s Four Levels of Evaluation Model is for you. It is a standard model used by trainers in all environments. It is relatively simple and gets you beyond the basics so that you can really start to understand the impact of your efforts. You can apply it in-person and online. You can apply it if teaching students, community members, and your colleagues. So, yes, the Kirkpatrick Model will definitely work for you in your library.
The model was created by Donald Kirkpatrick and, James and Wendy Kirkpatrick (the son and daughter-in-law of Donald) published an updated version in a book called Kirkpatrick’s Four Levels of Training Evaluation in 2016. If you are interested in reading one book on this topic, I recommend that book. If you follow that link and make a purchase, I will receive a small amount of money through the Amazon Affiliates Program to help me to support this website. I’m also a fan of you checking the book out from your own library, of course!
In this post, I’ll share some tips on getting started with evaluating your library instruction or program by focusing Level 1, or reaction. Watch for the remaining levels—for learning, behavior, and results—in future articles.
How Do I Get Started with Level 1?
When teaching, you are probably already collecting information on your learners’ reaction. You might do so informally by noticing that everyone is attentive and smiling. You might do so a bit more formally by asking your learners to share their reactions in a survey (also known as a “smile sheet”). That’s Level 1 which measures Reaction.
Here is a reaction question that I usually ask after large networking events:
What was your favorite part of the event?
This open-ended style is an easy question to ask and an easy question to answer, and I usually get great feedback on what went well–particularly about the physical space and food. I also ask for more general feedback in another question, and learn about things that could be improved upon next time.
Is That All There Is to Level 1?
That can be all there is to measuring Level 1 Reaction. But you don’t have to stop there. You might also ask about specific features of a course. Here is a summary of responses to the following question:
To what extent did each of the following activities help you to develop a career goal and next steps within a career development plan?
This question was part of a survey sent to learners who had completed a multiple-week immersive program focused on career development. See the results of that question below to get a sense of how useful this particular Level 1 Reaction question was.
Interested in learning more about the career development program? Chandra Lane and I co-presented about it at the 2018 Annual Conference of the American Library Association. Check out this handout for lots of details.
You can see that this style of question provides much more detailed (and useful) information than the open-ended question. It’s still a Level 1 Reaction question.
Industry reports from the Association of Talent Development have found that most organizations collection information on Level 1 Reaction, and that many organizations don’t go any further. You might wonder why so many trainers collect this data point and nothing else. Perhaps it’s because it’s so easy! At the end or following training–in person or online–it’s easy to ask the learner if they liked the experience. The information you receive can be incredibly helpful, too, especially when deciding what to change if you will be offering the same or similar instruction in the future.
Whether or not your learners liked the experience is just baseline data, though, when you are evaluating the instruction you’ve provided. It doesn’t tell you anything about what your learners actually learned. Learning is the 2nd level in the Kirkpatrick model…and that is the topic for a new post.
How about You?
Kirkpatrick’s 4 levels of evaluation is something that I use regularly in my own work–particularly to measure Level 1 Reaction. How do you measure reaction? I’d love to hear your story in the comments.