AI Health Virtual Seminar Series: Artificial intelligence and the ethics of use: Patient and provider perspectives on utilizing prediction models in medical decision-making

Clinical predictions are a critical component of delivering quality healthcare. As clinical data grows rapidly in volume and quality, a large amount of research has been dedicated to developing machine learning (ML)-based clinical predictive models (CPMs), which can leverage large amounts of patient information to predict life expectancy, disease progression, and other health outcomes. In conjunction with attention to these models' development, we must also consider ethics of how these models can be best utilized and understood by patients and providers, including considering interpretability and implications for medical decision-making. In this seminar, we discuss to a current qualitative study examining these factors as understood by four key end-user groups: clinical providers; support providers such as dialysis staff and social workers; patients; and patients' caregivers (e.g., family members). As a use case, this study specifically examines a mortality prediction model for patients undergoing hemodialysis. We identify key factors related to trustworthiness, interpretability and use, and we provide suggestions of use-focused considerations to examine and prioritize ongoing.