Leveraging Visual Feedback from Social Signal Processing to Enhance Clinicians’ Nonverbal Skills

Nonverbal communication between patients and clinicians affects the delivery of empathic patient-centered care and patient outcomes. To be effective communicators, clinicians must appropriately encode, decode, and regulate nonverbal cues, such as speech rate, pitch, facial expression, and body language. Yet, few efforts to develop tools for enhancing clinician communication have focused on nonverbal aspects of the clinical encounter. To address this gap, we describe a novel solution that both uses social signal processing technology (SSP) to capture nonverbal cues in real time and displays instant visual feedback. In this paper, we examine the theoretical underpinnings of nonverbal cues and their critical role in clinical encounters. We then describe opportunities for capturing nonverbal cues with SSP and explore visual designs for feeding back those social signals to enhance clinicians’ nonverbal communication.