To reduce missed appointments, a team at Children’s Hospital of Philadelphia (CHOP) developed and implemented a model to predict which patients are at risk for missing their outpatient visits.
The PATTERNS (Predictive Analytic Technology to Eliminate Repeat No Shows) project included a team of 20 individuals from medicine and nursing, to social work, informatics and patient experience. Technical and operations teams developed a predictive model to leverage data from the electronic health record (EHR) and identify interventions based on the drivers of missed appointments.
Interviews of patients and families in waiting rooms identified inconsistent and unclear appointment reminders as a key driver of missed appointments. In a follow-up survey to 200 families, the team identified multiple modes of appointment reminders, including: automated phone calls, text messages, mailed letters, EHR messages and personal calls from staff.
These reminders had varying frequencies, with some seven days prior to the appointment and others less than 24 hours before the visit. Survey feedback indicated families were confused by the lack of consistency in appointment reminders and the inability to get a reminder in their preferred mechanism, with 50% of families preferring text messages and less than 10% receiving reminders through that mode.
Reasons for missed appointments
The team of family consultants, practice managers, clinicians, schedulers and front desk staff developed workflows, job aids and scripts to improve the use of text message reminders. The team removed the burden of signing up for text message reminders from the family and integrated the process into the scheduling and registration process within the EHR.
Patient and family preferences were key so staff could note in the EHR the desire to continue to receive a phone call if preferred. However, to improve the poor performance of automated phone reminders, the team worked with family consultants to reduce the length of the appointment reminder message from over two minutes to less than 45 seconds, which reduced hang-ups and increased the confirmation rate.
To ensure patients with limited English proficiency could receive and act on appointment reminders, Hallam Hurt, M.D., neonatologist and project champion, recommended offering appointment reminders in other languages. As a result of a collaborative effort with the hospital’s language and cultural services team, appointment reminders are now available in Cantonese, Mandarin, Spanish and Arabic.
A side benefit of increased text messages, the project improved pre-visit communication with families regarding directions, parking and other questions to reduce no-shows and address late arrivals and unnecessary phone calls to the office. The team moved from static text messages to interactive texting. This allows families to ask and receive responses to questions about their upcoming appointment after receiving a detailed appointment itinerary.
Questions range from inquiries about parking to detailed questions about insurance referrals. The interactive texting system is fueled by a chatbot, an artificial intelligence program. The chatbot answers 95% of questions with the remaining 5% delegated to hospital staff to respond via text or phone call.
Marie Gleason, M.D., a cardiologist, helped to champion the efforts within her division. “The interactive texts have been well received,” she says. “Not only did families who received the text reminders appreciate them, they showed better clinic attendance rates, and scheduling staff reported improved ability to act on cancelled appointments and increase appointment slot utilization.”
While these changes address families who forgot their appointments due to inefficient reminders, those patients were at the lowest risk for no-show. The bigger challenge was families who chronically missed appointments—that’s where the predictive model came in.
Finding an opportunity
Robert Grundmeier, M.D., primary care physician and informaticist, led the development of a predictive model to determine missed appointment risk with Justine Shults, Ph.D., statistician and Luis Ahumada, Ph.D., computer scientist, as well as a team of developers and programmers who facilitated integration of the model into the EHR. The model used two years of data from primary and specialty care, analyzing over 4 million visits to identify variables that contribute to missed appointments, including appointment history, type of appointment, time of day, distance to appointment and insurance coverage.
The model was integrated within the scheduling and clinician workflows in the EHR. Patients identified as a risk for a missed appointment are prompted to respond to additional questions when scheduling based on drivers for missed appointments. These questions identify transportation, financial and other barriers the family may be experiencing. Responses are sent to a work queue in the EHR for the patient’s care team and social worker to address prior to the appointment.
Within weeks of implementing the model, the team realized its first success—a patient who had missed his last four appointments was flagged for intervention. The care team contacted the family, determined the patient’s barriers to attendance and addressed them prior to the visit. The patient attended his next appointment, got on track with his treatment plan and met with social workers to develop a plan for attending future appointments.
The effect on patients, families and providers
With 65% of patients opting into texting from a baseline of 10%, the hospital has seen improvements in appointment confirmation rates and adherence. Now, over 70% of patients confirm appointments compared to less than 20% previously.
With all primary care and 90% of specialty practices using the interactive texting system, and the implementation of the predictive model in 70% of departments, missed appointments have been reduced by a combined 9.2% in specialty and primary care. Using a team of clinicians, social workers, administrators and data analysts while engaging patients was the key to developing a solution that incorporated family preferences and staff workflows to lead to success.
