Using the EBP Model to Solve a Problem

A group of nursing professional development professionals on the education committee for the Association of Nursing Professional Development (ANPD) formed a subcommittee specific to return on investment (ROI). They were challenged with providing the membership with education on ROI and the financial value of professional development activities.

The group decided to use the seven-step Melnyk and Fineout-Overholt Evidence-Based Practice (EBP) model to guide their exploration of the topic. Debra Liebig, M.L.A., B.S.N., RN-BC, program manager in the accreditation and readiness department at Children’s Mercy Kansas City, walks through the process.

Cultivate a spirit of inquiry

This first step involves gathering a group of people who are interested in and excited to explore the topic at hand. The ANPD sub-committee had the initial interest to look into as many aspects of their question as possible.

Ask clinical questions in the PICOT format

Develop a clinical question to guide your search. The team used the PICOT formula for writing a research question.

• Population. Identifies the subjects of the study.
• Intervention of interest. The main intervention/treatment of interest.
• Comparison intervention. An intervention/treatment to compare to the intervention of interest.
• Outcome(s). The result that will be measured.
• Time. Duration of the data collection process.

Their resulting question was: “In healthcare organizations (P), how does calculation of ROI for professional development activities (I), compared to nonfinancial evaluation of programs (C), affect decisions about resource allocation to professional development activities (O)?”

Search for the best evidence

Using keywords and reliable databases, in this step, gather what has already been written on your topic and sift through it.

They began to look for the answer to this question with a review of existing literature. They searched for keywords in Medline, CINAHL and ERIC using three groupings, including:

• Professional development, staff development, nurse educator, education, training.
• Return on investment, benefit-cost analysis, financial impact.
• Program evaluation, educational outcome, measurement, impact, resource allocation.

“We struggled with this because there is no consistency in the way the concepts are reported in the literature,” Liebig said. “We found ourselves doing more literature searching as we read the articles.”

Out of 140 articles that the team read, which had been published in the last 10 years (2004-2014) and included pre- and post-data or a financial impact, they found 12 studies that met their criteria.

Those 12 were referenced in their first two award-winning articles. They found another four that had “We identified that people were not publishing cost of implementing education programs, which would lead to practice changes, so we knew that we had a gap,” Liebig said.

Critically appraise the evidence 

This step includes evaluating the validity, reliability, and applicability of the study to solve the problem or answer the question at hand.

Outcomes of the studies found in existing literature ranged vastly, including attrition, lost workdays, injury rates, and job satisfaction. The investigators were monitoring as many as eight outcomes per study.

“When you think of the economic impact of your practice change, consider a wide range of outcomes,” Liebig said. “The change at least in part could affect multiple outcomes. We highly encourage you to watch them all.”

Integrate evidence with expertise and patient preferences and values

Taking action with the newly gained understanding of your topic comes in here. In clinical settings, the patient’s preferences, health, and demographics should be considered before implementation. The team put their knowledge into action in several ways:

• Writing their three-article series.
• Presenting webinars with ANPD, CHA, the Center of Transdisciplinary Evidence-based Practice, and The FULD Institute for EBP.
• Cathleen Opperman, DNP, RN, NEA-BC, CPN, nursing professional development specialist at Nationwide Children’s Hospital, writing chapter 18 in the ANPD CORE Curriculum on “Measuring Impact at the Organizational Level.”
• Hosting half-day workshops at ANPD Conventions.
• Presenting at the 2018 ANA Quality and Innovation Conference.

Throughout the process, they were also nursing another idea. “Our group dreamed about an app that we could enter the data and it would spit out all the calculations,” Liebig said.

“Through this app, you could make expense adjustments and outcome goals to create more viable economic impacts of your programs. We created it, and now access to it is available to ANPD Members for a nominal fee.”

Evaluate outcomes of practice decisions or changes based on evidence

This step calls for evaluating the outcomes of the interventions that you put into place. The team considered more publications with economic impact reported and increased comfort calculating financial impact for professional development activities to be indicators of the outcome of their work but have not gathered formal results yet.

“Maybe the measure of impact with this project will be another search of the literature in a few years to find many more studies including ROI and economic assessment,” Liebig said. “Maybe [it] will be incorporating economic or financial assessment as a competency for all Educators and Leadership.”

Disseminate EBP results

Getting the word out can greatly increase the impact of your study. The articles, webinars, workshops, Core Curriculum, and the program to do the calculations were a combination of the group’s Step 4 implementation and Step 6 dissemination efforts. “We think we are doing a rather thorough job with the disseminating step,” Liebig said.