Outcomes Measurement Simplified
Outcomes measurement, in relationship to CME, is the measured
effectiveness of a CME activity as it relates to outcomes of patient
care. New regulations from the Accreditation Council for Continuing
Medical Education (ACCME) require a shift from more traditional
evaluation methods to outcomes-based evaluations. Moving to this type of
evaluation model will impact the learning cycle of healthcare
professionals and the healthcare of patients. It will also help to
account for the effectiveness, cost, and impact, while setting goals for
CE program professionals.
Levels of Evaluation
In “Evaluating Your Evaluations: A Practical Guide,” the authors
explain that the 5 levels of evaluation for CME are a widely accepted
concept of Kirkpatrick and Philips which measure learning outcomes. [1]
This system is used as a means of categorizing evaluations and ranking
them for effectiveness. Level one being the least effective to level
five which is the most effective. As the levels progress so does the
level of difficulty in evaluation which is why planning your outcomes
measurement should be a vital part of every activity. In the past, many
evaluations were designed to achieve levels 1-2; new requirements will
see a shift to attain levels 3-5.
 Level 1
This level of evaluation has been the long standard of CME
evaluation. It is no longer the accepted way of evaluating an activity
though program planners may still find this information valuable in
accessing topic selection and speaker ratings. Level 1 = “reaction
(learners’ participation and satisfaction).” [1]
Level 2
This level is also a long standing evaluation type which focused on
the learners’ knowledge after the completion of an activity. This type
of evaluation determined if the learner planned to make changes due to
the information they learned. Level 2 = “learning (enhanced knowledge
and skills).” [1]
Level 3
This level measures the actual change or improvement made by a
learner. While it is self-reported, and it may be only a slice of the
full range of clinician behavior, it allows the learner to consider the
result of the learning in regards to patients. Level 3 = “behavior
(changes in clinical behavior).” [1]
Level 4
Both levels 4 and 5 present greater challenges for measurement. For
data Owens suggests “measures of quality, utilization, screening,
diagnoses, prescribing patterns, imaging, hospitalization, patient
satisfaction, and even public health.”[1] Level 4 = “results (improved
performance that produces changes in patient outcomes).” [1]
Level 5
Suggested data for level 5 measurement by Owens is “measures of
treatment outcomes and health status, such as morbidity and mortality,
secondary complications, adequate control of underlying disease,
hospitalization and rehospitalization, and community public health
statistics.” [1] Level 5 = “changes in population health outcomes.” [1]
Information from each level of evaluation can be useful to learners
as well as planners as they implement higher levels of the evaluation
model. With budgets stretched tight, taking on new process such as
outcomes measurement can be daunting. It is important to understand the
requirements but foremost how outcomes measurement can benefit your CE
program, your learners, and patient care. CME program planners need to
work closely with their CME joint sponsor to ensure outcomes measurement
is being performed on each of the activities in their program in order
to meet new ACCME requirements.
Resources:
- Owen, John A., et al. "Evaluating Your Evaluations: A Practical
Guide." CE Measure, Vol. 2 Issue 4. 2008. Carden Jennings Publishing
Co., Ltd. 14 Jan. 2009. <http://www.cemeasure.com/media/journal/2-4/074-80.08108.Owen.pdf>
- Accreditation Council for Continuing Medical Education Web site.
Available at: http://www.accme.org. Accessed January 15, 2009.
Variety in Professional Credit
CFMC is a national accredited provider of CME and CNE. We also
partner with other organizations on a regular basis to bring you
professional credits for NAHQ, AAFP, AORN, Social Worker, Psychology and
Pharmacy to name a few. For more information, please call Lorraine
Pickrell at 800-950-8250 ext 3372 or visit us on the web at
www.yourCEsource.org.
CFMC Upcoming Educational Activities
Visit www.yourCEsource.org for a complete list of
upcoming educational activities.
|
