I made a demo showing how changes in a patient population can affect the performance of a biomarker or classifier, specifically measured by AUC (Area Under the Receiver Operating Characteristic Curve). Using a simulated dataset, I first showed that hemoglobin A1c had an AUC of 0.77 for predicting foot ulcers in a population of diabetic patients. But when I added 1,000 non-diabetic patients who didn’t develop ulcers, the AUC jumped to 0.93. This illustrates how a more diverse or broader patient population can artificially inflate performance metrics, highlighting the need for careful evaluation of AUC when comparing models across different datasets.
See the full details on Google Colab: link