Sharing some brief lessons learned about deploying clinical prediction models, based on our experience with a home-grown AI mortality prediction model for cancer patients. This model has been in production use for four years, so we’ve experienced both the highs and the lows.
For more details, you can check out our paper in JNCI on the randomized rollout of the AI model, as well as the accompanying editorial.
1. This Work is New, Sensitive, and Important
When you’re predicting serious health events, there’s a potential to cause harm, and people will have different attitudes toward it. Don’t embark on this unless you’re fully committed, willing to go to bat to explain to others why this is worthwhile, and ready to stay engaged for the long haul.
2. Stakeholder Involvement is Critical
We involved key stakeholders early on, including doctors, other providers, patients, the IT team, and the quality improvement team. Our Patient and Family Advisory Council was particularly invaluable, providing feedback from the most important perspective: the patients themselves. A high-up member of the hospital IT team had had a personal experience with a relative's suboptimal end of life care which made them receptive to our explanations of the value of the work. This resulted in IT allocating significant resources to build and support the needed infrastructure, which I was super grateful for.
3. Plan Ahead for Validation
Validation makes you confident that your model is performing well in practice. Models can stop working well due to gradual shift in input data distribution (imagine that treatments for a certain type of cancer get more effective over time; this could cause the model to under-predict survival time), or other issues like some of the predictor variables being coded differently in the medical record system.
You will need to do multiple kinds of validation. One simple thing is to monitor the distribution of the predictions to ensure that there are no rapid shifts. For instance, if the model has typically predicted that most patients with metastatic cancer will live for a few years, but then suddenly the median prediction changes to a few months, there is a bug or other issue to fix.
Validating the quality of predictions is also important. It can be challenging to get high-quality validation data. Setting up data sources early is crucial because working with hospital IT can be slow--they’re often overworked and juggling many projects. We validate our model's predictions using data from Epic follow-up/death information and the Social Security Death Index.
4. Get Feedback From Users
Engaging clinician users is challenging because clinicians are busy, and not all are equally invested in advance care planning (the primary use case for our model). We did a Qualtrics survey to clinicians, but it received few responses. We found more success with direct conversations and by monitoring real-world usage (actions often speak louder than words). You should make sure that clinicians know who is organizing the use of the model and who to give feedback to; we have gotten lots of useful feedback emailed by individual doctors.
That’s it! I hope this was helpful for others working on similar projects.