Donald L Kirkpatrick, Professor Emeritus, University Of Wisconsin (where he achieved his BBA, MBA and PhD), first published his ideas in 1959, in a series of articles in the Journal of American Society of Training Directors. The articles were subsequently included in Kirkpatrick's book Evaluating Training Programs (originally published in 1994; now in its 3rd edition - Berrett-Koehler Publishers).
Donald Kirkpatrick was president of the American Society for Training and Development (ASTD) in 1975. Kirkpatrick has written several other significant books about training and evaluation, more recently with his similarly inclined son James, and has consulted with some of the world's largest corporations.
Donald Kirkpatrick's 1994 book Evaluating Training Programs defined his originally published ideas of 1959, thereby further increasing awareness of them, so that his theory has now become arguably the most widely used and popular model for the evaluation of training and learning. Kirkpatrick's four-level model is now considered an industry standard across the HR and training communities...
More recently Don Kirkpatrick formed his own company, Kirkpatrick Partners, whose website provides information about their services and methods, etc.
kirkpatrick's four levels of evaluation model
The four levels of Kirkpatrick's evaluation model essentially measure:
* reaction of student - what they thought and felt about the training
* learning - the resulting increase in knowledge or capability
* behaviour - extent of behaviour and capability improvement and implementation/application
* results - the effects on the business or environment resulting from the trainee's performance
All these measures are recommended for full and meaningful evaluation of learning in organizations, although their application broadly increases in complexity, and usually cost, through the levels from level 1-4.
Quick Training Evaluation and Feedback Form, based on Kirkpatrick's Learning Evaluation Model - (Excel file)
kirkpatrick's four levels of training evaluation
This grid illustrates the basic Kirkpatrick structure at a glance. The second grid, beneath this one, is the same thing with more detail.
level evaluation type (what is measured) evaluation description and characteristics examples of evaluation tools and methods relevance and practicability
1Reaction
Reaction evaluation is how the delegates felt about the training or learning experience.
'Happy sheets', feedback forms.
Verbal reaction, post-training surveys or questionnaires.
Quick and very easy to obtain.
Not expensive to gather or to analyse.
2Learning
Learning evaluation is the measurement of the increase in knowledge - before and after.
Typically assessments or tests before and after the training.
Interview or observation can also be used.
Relatively simple to set up; clear-cut for quantifiable skills.
Less easy for complex learning.
3Behaviour
Behaviour evaluation is the extent of applied learning back on the job - implementation.
Observation and interview over time are required to assess change, relevance of change, and sustainability of change.
Measurement of behaviour change typically requires cooperation and skill of line-managers.
4Results
Results evaluation is the effect on the business or environment by the trainee.
Measures are already in place via normal management systems and reporting - the challenge is to relate to the trainee.
Individually not difficult; unlike whole organisation.
Process must attribute clear accountabilities.
kirkpatrick's four levels of training evaluation in detail
This grid illustrates the Kirkpatrick's structure detail, and particularly the modern-day interpretation of the Kirkpatrick learning evaluation model, usage, implications, and examples of tools and methods. This diagram is the same format as the one above but with more detail and explanation:
evaluation level and type evaluation description and characteristics examples of evaluation tools and methods relevance and practicability
1. Reaction
Reaction evaluation is how the delegates felt, and their personal reactions to the training or learning experience, for example:
Did the trainees like and enjoy the training?
Did they consider the training relevant?
Was it a good use of their time?
Did they like the venue, the style, timing, domestics, etc?
Level of participation.
Ease and comfort of experience.
Level of effort required to make the most of the learning.
Perceived practicability and potential for applying the learning.
Typically 'happy sheets'.
Feedback forms based on subjective personal reaction to the training experience.
Verbal reaction which can be noted and analysed.
Post-training surveys or questionnaires.
Online evaluation or grading by delegates.
Subsequent verbal or written reports given by delegates to managers back at their jobs.
Can be done immediately the training ends.
Very easy to obtain reaction feedback
Feedback is not expensive to gather or to analyse for groups.
Important to know that people were not upset or disappointed.
Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same.
2. Learning
Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before to after the learning experience:
Did the trainees learn what what intended to be taught?
Did the trainee experience what was intended for them to experience?
What is the extent of advancement or change in the trainees after the training, in the direction or area that was intended?
Typically assessments or tests before and after the training.
Interview or observation can be used before and after although this is time-consuming and can be inconsistent.
Methods of assessment need to be closely related to the aims of the learning.
Measurement and analysis is possible and easy on a group scale.
Reliable, clear scoring and measurements need to be established, so as to limit the risk of inconsistent assessment.
Hard-copy, electronic, online or interview style assessments are all possible.
Relatively simple to set up, but more investment and thought required than reaction evaluation.
Highly relevant and clear-cut for certain training such as quantifiable or technical skills.
Less easy for more complex learning such as attitudinal development, which is famously difficult to assess.
Cost escalates if systems are poorly designed, which increases work required to measure and analyse.
3. Behaviour
Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour, and this can be immediately and several months after the training, depending on the situation:
Did the trainees put their learning into effect when back on the job?
Were the relevant skills and knowledge used
Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles?
Was the change in behaviour and new level of knowledge sustained?
Would the trainee be able to transfer their learning to another person?
Is the trainee aware of their change in behaviour, knowledge, skill level?
Observation and interview over time are required to assess change, relevance of change, and sustainability of change.
Arbitrary snapshot assessments are not reliable because people change in different ways at different times.
Assessments need to be subtle and ongoing, and then transferred to a suitable analysis tool.
Assessments need to be designed to reduce subjective judgement of the observer or interviewer, which is a variable factor that can affect reliability and consistency of measurements.
The opinion of the trainee, which is a relevant indicator, is also subjective and unreliable, and so needs to be measured in a consistent defined way.
360-degree feedback is useful method and need not be used before training, because respondents can make a judgement as to change after training, and this can be analysed for groups of respondents and trainees.
Assessments can be designed around relevant performance scenarios, and specific key performance indicators or criteria.
Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols.
Self-assessment can be useful, using carefully designed criteria and measurements.
Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation.
Simple quick response systems unlikely to be adequate.
Cooperation and skill of observers, typically line-managers, are important factors, and difficult to control.
Management and analysis of ongoing subtle assessments are difficult, and virtually impossible without a well-designed system from the beginning.
Evaluation of implementation and application is an extremely important assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job, therefore evaluation in this area is vital, albeit challenging.
Behaviour change evaluation is possible given good support and involvement from line managers or trainees, so it is helpful to involve them from the start, and to identify benefits for them, which links to the level 4 evaluation below.
4. Results
Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test.
Measures would typically be business or organisational key performance indicators, such as:
Volumes, values, percentages, timescales, return on investment, and other quantifiable aspects of organisational performance, for instance; numbers of complaints, staff turnover, attrition, failures, wastage, non-compliance, quality ratings, achievement of standards and accreditations, growth, retention, etc.
It is possible that many of these measures are already in place via normal management systems and reporting.
The challenge is to identify which and how relate to to the trainee's input and influence.
Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training, so they understand what is to be measured.
This process overlays normal good management practice - it simply needs linking to the training input.
Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training.
For senior people particularly, annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training.
Individually, results evaluation is not particularly difficult; across an entire organisation it becomes very much more challenging, not least because of the reliance on line-management, and the frequency and scale of changing structures, responsibilities and roles, which complicates the process of attributing clear accountability.
Also, external factors greatly affect organisational and business performance, which cloud the true cause of good or poor results.
Since Kirkpatrick established his original model, other theorists (for example Jack Phillips), and indeed Kirkpatrick himself, have referred to a possible fifth level, namely ROI (Return On Investment). In my view ROI can easily be included in Kirkpatrick's original fourth level 'Results'. The inclusion and relevance of a fifth level is therefore arguably only relevant if the assessment of Return On Investment might otherwise be ignored or forgotten when referring simply to the 'Results' level.
Learning evaluation is a widely researched area. This is understandable since the subject is fundamental to the existence and performance of education around the world, not least universities, which of course contain most of the researchers and writers.
While Kirkpatrick's model is not the only one of its type, for most industrial and commercial applications it suffices; indeed most organisations would be absolutely thrilled if their training and learning evaluation, and thereby their ongoing people-development, were planned and managed according to Kirkpatrick's model.
For reference, should you be keen to look at more ideas, there are many to choose from...
* Jack Phillips' Five Level ROI Model
* Daniel Stufflebeam's CIPP Model (Context, Input, Process, Product)
* Robert Stake's Responsive Evaluation Model
* Robert Stake's Congruence-Contingency Model
* Kaufman's Five Levels of Evaluation
* CIRO (Context, Input, Reaction, Outcome)
* PERT (Program Evaluation and Review Technique)
* Alkins' UCLA Model
* Michael Scriven's Goal-Free Evaluation Approach
* Provus's Discrepancy Model
* Eisner's Connoisseurship Evaluation Models
* Illuminative Evaluation Model
* Portraiture Model
* and also the American Evaluation Association
Also look at Leslie Rae's excellent Training Evaluation and tools available on this site, which, given Leslie's experience and knowledge, will save you the job of researching and designing your own tools.
evaluation of HRD function performance
If you are responsible for HR functions and services to internal and/or external customers, you might find it useful to go beyond Kirkpatrick's evaluation of training and learning, and to evaluate also satisfaction among staff/customers with HR department's overall performance. The parameters for such an evaluation ultimately depend on what your HR function is responsible for - in other words, evaluate according to expectations.
Like anything else, evaluating customer satisfaction must first begin with a clear appreciation of (internal) customers' expectations. Expectations - agreed, stated, published or otherwise - provide the basis for evaluating all types of customer satisfaction.
If people have expectations which go beyond HR department's stated and actual responsibilities, then the matter must be pursued because it will almost certainly offer an opportunity to add value to HR's activities, and to add value and competitive advantage to your organisation as a whole. In this fast changing world, HR is increasingly the department which is most likely to see and respond to new opportunities for the support and development of the your people - so respond, understand, and do what you can to meet new demands when you see them.
If you are keen to know how well HR department is meeting people's expectations, a questionnaire, and/or some group discussions will shed light on the situation.
Here are some example questions. Effectively you should be asking people to say how well HR or HRD department has done the following:
* helped me to identify, understand, identify and prioritise my personal development needs and wishes, in terms of: skills, knowledge, experience and attitude (or personal well-being, or emotional maturity, or mood, or mind-set, or any other suitable term meaning mental approach, which people will respond to)
* helped me to understand my own preferred learning style and learning methods for acquiring new skills, knowledge and attitudinal capabilities
* helped me to identify and obtain effective learning and development that suits my preferred style and circumstances
* helped me to measure my development, and for the measurement to be clear to my boss and others in the organisation who should know about my capabilities
* provided tools and systems to encourage and facilitate my personal development
* and particularly helped to optimise the relationship between me and my boss relating to assisting my own personal development and well-being
* provided a working environment that protects me from discrimination and harassment of any sort
* provided the opportunity for me to voice my grievances if I have any, (in private, to a suitably trained person in the company whom I trust) and then if I so wish for proper consideration and response to be given to them by the company
* provided the opportunity for me to receive counselling and advice in the event that I need private and supportive help of this type, again from a suitably trained person in the company whom I trust
* ensured that disciplinary processes are clear and fair, and include the right of appeal
* ensured that recruitment and promotion of staff are managed fairly and transparently
* ensuring that systems and activities exist to keep all staff informed of company plans, performance, etc., (as normally included in a Team Briefing system)
* (if you dare...) ensuring that people are paid and rewarded fairly in relation to other company employees, and separately, paid and rewarded fairly when compared to market norms (your CEO will not like this question, but if you have a problem in this area it's best to know about it...)
* (and for managers) helped me to ensure the development needs of my staff are identified and supported
This is not an exhaustive list - just some examples. Many of the examples contain elements which should under typical large company circumstances be broken down to create more and smaller questions about more specific aspects of HR support and services.
If you work in HR, or run an HR department, and consider that some of these issues and expectations fall outside your remit, then consider who else is responsible for them.
I repeat, in this fast changing world, HR is increasingly the department which is most likely to see and respond to new opportunities for the support and development of the your people - so respond, understand, and do what you can to meet new demands when you see them. In doing so you will add value to your people and your organisation - and your department
Πηγή: businessballs
