Extreme Make-over: Smile Sheet Edition

Eval Form Cover Image

A few weeks ago I finished reading Will Thalheimer’s book, Performance-focused Smile Sheets: A Radical Rethinking of a Dangerous Artform (here’s my brief review of the book).

A colleague recently made fun of me, suggesting that I read “geeky books” in my spare time. That would be true if I just read books about smile sheets for fun. And while I did have fun reading this book (so I guess I am kind of geeky), I’ve been attempting to integrate lessons learned from the book into my work.

Following are two examples of improvements I’ve made on existing smile sheets, and the logic behind the changes (based upon my interpretation of the book): 

Example 1: Was the training a good use of your time?

01 Good Use of Time (Old)

Intent: I wanted a bottom-line metric: what percent of participants felt investing their precious time in this training session was worthwhile? Over the past several years, I’ve been able to smile while sharing post-training evaluation feedback with presenters and my supervisor by telling them: 100% of training participants responded that this was a good use of their time!

The problem: If my supervisor (or anyone else) ever asked: “What does that mean? Was it a good use of their time because they felt entertained? Because they were able to escape their office for a day or two? Because they’re able to do something better? Why was it a good use of their time?” I wouldn’t have a good answer.

Based on what I learned in the book, I’ve adjusted the question to read like this:

01 Good Use of Time (New)

Why I like this better: True, it’s not as clean (on the surface, anyway) as a question that simply asks whether the training was a good use of the participants’ time. However, this question allows me to identify specific ways that this training session may have been a good use of participants’ time.

If the training is designed to make sure participants are able to carry out tasks appropriately, then it’s really not a productive use of their time if the participants don’t feel they’re able to put these concepts into practice.

Example 2: Was the training engaging?

02 Engaging (Old)

Intent: I think that lecture-style presentations are super boring, and I wanted a way to show the world that people who design lecture-based presentations will pay for it on end-of-session evaluations when I ask a question about participants’ level of engagement (while those awesomely designed presentations will be rewarded with high scoring responses on this question).

The problem: In the end, presentations with activities designed to engage learners and get them to practice new skills sometimes scored higher on this question… marginally. Average scores of activity-based sessions often came in around a 4.2 or 4.3 while lecture-based sessions would come in around 4.0 or 4.2. Of course, what does a 4.2 even mean?!

Desperately seeking a better way to evaluate the level of engagement that the design of a training session would yield, I borrowed (stole?) this question from Will’s book:

02 Engaging (New)

Why I like this better: Instead of having to choose between a 3 or a 4 or a 5, now there is a continuum of choices to describe how engaging a training program turned out to be. Anything less than “mostly engaged” is probably unacceptable.

One key point in Will Thalheimer’s book is that Smile Sheet-level feedback should be actionable. It’s tough to take action on a collection of numbers. After all, at what point do you take action? When something averages out to be a 3.9? 4.2?

These are two examples of how spending a weekend reading a “geeky book” can change habits or practices in the workplace. I’d love to hear what you’ve read recently… and what you’ve done with that new knowledge.

 

4 thoughts on “Extreme Make-over: Smile Sheet Edition

  1. Thanks for sharing how Will’s book is influencing your smile sheets. I’m going through a very similar exercise now and am finding Will’s guidance both enlightening (why are we collecting this data in the first place) and practical (how do we ask good questions that help us truly understand the effectiveness of our learning events and take action to further improve them). Good stuff!

    • Yes! I’ve found it both helpful in re-vamping existing evaluation forms AND it’s given me food for thought on creating evaluation forms for sessions that we traditionally haven’t spent too much time evaluating (like new staff onboarding/orientation sessions).

  2. Just got my copy of Will’s book yesterday and am excited to start (geeky) reading. I will likely have more thoughts on this after reading, but your question regarding “How engaging was the course?” made me think. How do we reconcile option a) I like to learn by myself with f) I was regularly engaged? What if a learning preference is to process information later by oneself instead of out loud with others? Where the real benefit in learning is the gift of time with the material and with oneself? I feel like it is possible to be regularly engaged and to still feel that I would have preferred to learn things on my own. Is option a) fundamentally different from options b)-f)? I really have no answer to this yet, but thought I would throw it out.

    I hope you are well Brian! Thanks for all of the great posts.

    –Scott

    • The intent behind a (and your comment makes me wonder if there’s a better phrasing for it) is to ask: Would I have been better off at home (or in my office) just reading about this topic as opposed to wasting my time (and money) going to an in-person session? The specific audience being targeted with that particular evaluation form is a group of surgeons, so the training program darn well better be worth the opportunity cost (they could have been in their clinics, restoring sight to people, instead of attending this training session).

      As an introvert, I definitely need time to process and think about materials on my own. This question is less about needing time to process information and is more about: was being here a good use of your time?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s