If you are in training, you CARE.
- Have you ever felt that your training department and operations were on different pages, opposite sides, even in a battle?
- Do you feel that your training department is more vulnerable to downsizing than other departments?
- In your nightly dreams, is Nirvana a place where leaders value the training efforts in which you have put so much brain power?
Enter……Dr. Donald Kirkpatrick, founder of the Four Level Training Evaluation Model.
I was recently privileged to hear Dr. Donald Kirkpatrick speak at an ASTD (American Society for Training & Development) conference. He is 80 years old and his work is as relevant today as it was when it was first published in 1959. (Don’t stop reading just because it’s old – Einstein’s work is still relevant.)
One of the concepts that Kirkpatrick talks about is identifying a “chain of evidence” that provides compelling proof of the value of training to the organization. He encourages us to supply a preponderance of proof, rather than proving the value beyond a shadow of doubt.
This is an important differentiation for trainers and training leaders who have beat their heads against the wall to prove empirically that their training was specifically what impacted results in a positive way.
Why might empirical proof be difficult? Trainers know that the temperature of the room, the attitude of the individual’s supervisor and personal distractions can affect a participant’s evaluation of training. Additionally, individuals may score high on what they learned in training, then be discouraged to employ the skills or procedures on the job. Conversely, outside influences, such as a thriving market or a new product introduction may have impacted results in a positive way independent of training.
Rather than focus on irrefutable proof, build a chain of evidence, compelling proof that the training has value by evaluating training in as many of the levels as it makes sense.
The levels that Kirkpatrick developed are summarized below in reverse order. Why in reverse order? Because so often training professionals do not think beyond Level 1 or 2. Our companies are called upon to bring in results in very competitive world. Trainers need to partner with the organization to achieve those results as actively as any operations person. This focus positions the training department as a value added to the operation.
Level 4 – Results
Results measures the impact of the training on the organization or company. The evaluation views sustained results, usually from 30, 60, 90 days after the intended audience has completed training. It often measures even beyond this timeframe. Level 4 evaluation may include a % increase in performance results, a cost/benefit analysis and/or an ROI (return on investment).
- In “Implementing the Four Levels,” Kirkpatrick states that this is the most important because it is the most important to our stakeholders.
- Kirkpatrick recommends that you “start with the end in mind.” In the design phase of training, trainers must agree with stakeholders on the goal the training will achieve and design metrics to measure that goal along with training design. From Kirkpatrick’s viewpoint, this level measures “Return on Expectations.”
- Consider using a control group and experimental groups to measure results before and after training.
Level 3 – Behavior
This level of evaluation is done after the participant returns to work. Its goal is to determine if he/she is using the information delivered in training and to what level it is being used. (e.g. If the individual attained a satisfactory score on the end of course test and then does not apply the skill on the job, it may indicate a lack of willingness, coaching or a difference in the direction the person is receiving on the job. These are all good bytes of information to determine how to make the course most effective.)
- Engage the operational supervisors and managers in evaluating behavior on job. Make sure to get buy-in for the process during the design phase of the training.
- Either employ evaluation tools already in use in the operations or create tools that supervisors and managers can use routinely to develop people. The tools then become part of the operation, rather than another task to be done for the training department. (e.g. In a call center operation, trainers can use the observation checklist that is used by the QA (Quality Assurance) department. Trainers for a trucking company can use a checklist developed by the truck manufacturer to ensure quality repair is demonstrated on job.)
- Often the same tool used to evaluate learning can be used to evaluate behavior on job. (e.g. If training uses a simulation that times how long it takes to complete a transaction, the Level 3 evaluation may be the report from the Automatic Call Distributor that averages the time for each agent’s cumulative transactions.
Level 2 – Learning
Evaluation of learning is performed at the end of the training session. Some of the methods which can be used are tests, graded simulations or observations rated using a checklist.
- Ensure that the evaluation gages the level of learning as described by the objectives of the training. (e.g. knowledge, comprehension, application, etc.) Use Bloom’s Taxonomy to develop objectives to the appropriate level of learning.
- Design the evaluation so that it can be easily scored and tabulated.
- Avoid making tests too easy. Challenging multiple choice tests can be created by using scenario based questions and scenario answers that have complex details to consider before answering.
Level 1 – Reaction
Reaction gages how participants feel about or view the training. It is the quintessential “smile sheet” that can be as valuable or destructive as you make it.
- Provide the evaluation at the end of training class as possible. Online evaluations sent by email after class will not be completed by as many people and will not capture the immediate reaction of individuals as well as end of class evals.
- Ask only the questions that will provide valuable information that actually will be used.
- Avoid using the evaluations as the only or the major assessment of instructor performance. (Oh, yes, instructors have been known to throw out less than perfect evaluations in order to ensure a superior rating and a substantial annual raise.)
Use these levels – all of them – to get your training department and operations on the same page, the same side, to jointly win the battle. You will feel less vulnerable to downsizing and you might even achieve Nirvana!
What do you think?
© Linda M. Farley www.LMFarley.com LMF@onr.com
Follow Me on Twitter: http://twitter.com/LindaFarley
Comments on this entry are closed.