Measure for effect
Isn't that right in all our organisations, small just as large ones? We measure everything and look at the ROI, which is certainly important for the business to continue to grow. Sales reps are measured on number of deals they close. Customer support has often a NPS to keep up or count complaints from customers that has been solved. We measure number of deliveries who gets delayed in the logistics department and number of invoices who gets paid at accounting. We measure as much as we can in the organisation. How else can we know that we are on the right path from achieving our goals?
"If you can't measure it, you can't manage it" - Peter Drucker.
Oh, just one more example, the Learning department. They get measured by the impact of the training, right? ROE, return on education. We measure the transfer of learning from the classroom back to the workplace. And how do we do that? Clearly transfer is the goal here so to continue serving the same training program to new trainees, we want to be sure of transfer success from last time. Feedback forms are apparently insanely popular among organizations to tackle the ROE and to get some ammunition for the training programs to continue. Research shows that 92% of companies use feedback forms to evaluate their trainings (American society for training & development). If you have not yet come across a training who hand out feedback forms I will give you a short explanations of what it is. Standardized scale based forms that measure nothing but a trainees satisfaction of the training. Common questions to include in a feedback form are "how much did you enjoy the training?" or "How eager are you to recommend this training to a colleague?". These forms also goes under names such as "evaluation forms" or "happy sheets".
How can such scale based questions measure our learning transfer goal?
It would be similar to a sales rep gets measured by asking the rep how satisfied he/she were with their customers from a scale 1 to 10. I am myself a sales person and I know for sure that my sales performance get measured by number of deals I successfully close at the end of the month. Being super satisfied with my first initial sales call with a prospect, says nothing about the actual result of the potential sale which happens further down the sales process.
I might be wrong here as in the field of training, trainees use much more of their learnings when they are satisfied with the training. Meaning that if there is a happy smile on that happy sheet, new learnings will be used back at the workplace. Even some research tells me I am wrong to say that feedback forms are not a good measurement to measure learning transfer within the field of training. Bringing up Donald Kickpatrick's evaluation model we can see that in the lowest level "reaction" as represents trainees' satisfaction with the training acquisition of skills is happening. Learning level is where the knowledge is determined and the following level refers to behaviors in the workplace. The last level is the results which refers to the ROI, the return on investment for the training. The levels are assumed to be built on one another. Just like Maslov's Hierachy of needs. I will not present the Hierchy of needs here as it is too off topic, but google it if you are not following. Anyways, the more positive reactions the happy sheet gets, the more knowledge is acquired. The more knowledge acquired, the more learning gets transferred to their workplace. And following the logic we should then get better business results for the organization.
Okay, wait one moment. Usually a happy sheet/evaluation form/feedback paper is filled in at the end of the class. Or sometimes sent by email a couple of days after. How is it then possible to secure that a trainee has transferred learnings from the training back to workplace by given a 10/10 on the question "How much did you enjoy the training?". There is clearly no follow-up later in time to confirm this. Happily I'm not alone in this corner of confusion as a bunch of transfer researchers decided to perform a meta-analysis of Kirkpatrick's evaluation model. And you know what? They they did not find a link between training satisfaction and transfer success. Meaning that the answer is NO to if trainees actually apply more of what they've learned by giving a 10/10 on the sheet. We cannot confirm transfer success by only looking at the feedback form.
Learn to meet company objectives
When we hand out feedback forms to our trainees we measure nothing but the satisfaction of the training program given. We cannot draw any conclusion at that point how much learning will be transferred back to the workplace. Which makes it complicated to measure business impact. I know many training departments are struggling with this but if we keep on relying on our happy sheets and continue to perform training programs based on satisfaction we can be heading to a total different path than the company objectives. Coming back to Peter Drucker's quote "What gets measured gets done" or "If you can't measure it, you can't manage it.".
How does your training department measure transfer of training?
Send me an email if you like to chat more about this topic!
Gustav@knowly.com
