“Data ages like fish, not wine. It gets worse as it gets older, not better.”

Gregg Thaler, data quality expert


This editorial in the New England Journal of Medicine earlier this year is worth reading and understanding as it relates to an important issue of dataset shift. This phenomenon occurs when a machine learning system has a malfunction or at least underperforms due to a mismatch between the original data set that trained the model and the data set that the model is deployed on.

This issue was widely publicized in the sepsis-alerting model developed by Epic Systems that underperformed due to a patients’ demographic change due to the coronavirus epidemic. The authors describe many causes of dataset shifts that are more subtle but clinically important. In the excellent table that accompanies this letter to the editor, the three categories of dataset shift include: changes in technology (e.g., software vendors); changes in population and setting (e.g., new demographics as in the aforementioned case); and changes in behavior (e.g., new reimbursement incentives).

The high likelihood of these dataset shifts and the myriad of possible causes lead to discussion of an AI governance team at institutions that are interested in such AI tools in clinical practice. This is an excellent example of the absolute need for continual human-machine synergy. The future of AI tools in health should incorporate a continuous model retraining and monitoring strategy based on all the possible changes in condition, all the while being in close collaboration with clinicians. Both clinicians and AI will then learn from each other.

Read the full paper here.