Course Calendar

The planned course schedule is provided below.

Week 1

Mar 31

BackgroundCourse Overview & Introduction to Foundation Models

Lecture Material:

• Course Overview

Apr 02

BackgroundIntroduction to LLMs

Lecture Material:

• Intro to LLMs

Suggested Reading:

• Monitoring AI-Modified Content at Scale
• Scaling Instruction-Finetuned Language Models
• Training Language Models to Follow Instructions
• Using ChatGPT to Facilitate Truly Informed Medical Consent

Week 2

Apr 07

ApplicationsAI Scientist Agents

Lecture Material:

• Advanced LLM Topics

Suggested Reading:

• A Visual-Language Foundation Model for Pathology Image Analysis
• OctoTools: An Agentic Framework
• FrugalGPT

Apr 09

ApplicationsCurrent Advances in LLMs and Agents

Lecture Material:

• MedArena

Suggested Reading:

• Physician Sentiments on AI in Healthcare

Week 3

Apr 14

BackgroundEvaluations of LLMs and agents

Lecture Material:

• Virtual Lab

Suggested Reading:

• The Virtual Lab
• TextGrad

Apr 16

ApplicationsGenerative AI for Drug Discovery

Lecture Material:

• Generative AI for Antibiotic Design
• Language-Guided Biology

Suggsted Reading:

• Generative AI for Antibiotic Design
• Embedding Models for Single-Cell Biology

Week 4

Apr 21

BackgroundSelf-Supervised Learning for Vision

Homework #1 Due

Lecture Material:

• Self-Supervised Learning for Vision

Suggested Reading:

• SimCLR
• Masked Autoencoders

Apr 23

BackgroundSelf-Supervised Vision-Language Models

Lecture Material:

• Intro to Vision-Language Models

Suggested Reading:

• Contrastive Language-Image Pretraining (CLIP)
• SigLIP
• ConVIRT
• 3D VLMs for Abdominal CTs
• ImageBind
• Discovering Systematic Errors with VLMs

Week 5

Apr 28

ApplicationsGenerative VLMs in Health

Project Proposal Due

Lecture Material:

• Generative VLMs

Suggested Reading:

• Denoising Diffusion Probabilistic Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• Learning Vision from Models Rivals Learning Vision from Data
• MedVAE: Efficient Automated Interpretation of Medical Images with Large-Scale Generalizable Autoencoders

Apr 30

ApplicationsFoundation Models for Segmentation

Lecture Material:

• FMs for segmentation

Suggested Reading:

• CLIPSeg
• U-Net
• RegionCLIP
• MedSAM

Week 6

May 05

BackgroundBuilding and Deploying Foundation Models

Homework #2 Due

Lecture Material:

• Building and Deploying FMs

Suggested Reading:

• IBM Model Infrastructure

May 07

BackgroundImproving Performance I: Retrieval-Augmented Generation

Lecture Material:

• Improving LLM Performance

Suggested Reading:

• Survey of LLM Prompting Methods
• Chain-of-Thought Prompting
• Sparse Lexical and Expansion Model for Information Retrieval
• Retrieval-Augmented Language Model Pre-Training
• Multimodal RAG

Week 7

May 12

BackgroundImproving Performance II: Controlled Forgetting and Fact-Checking

Lecture Material:

• Improving LLM Performance

Suggested Reading:

• Direct Preference Optimization
• RL with AI Feedback
• Hallucination Detection

May 14

BackgroundInference Scaling and Reasoning

Lecture Material:

• Advanced Topics

Suggested Reading:

• Hallucinations in Medical FMs
• MAIRA-2

Week 8

May 19

OpportunitiesFoundation Model Adaptation & Evaluation

Lecture Material:

• Adaptation and Evaluation

Suggested Reading:

• Clinical Text Summarization
• In-Context Learning
• Low-Rank Adaptation

May 21

OpportunitiesFoundation Model Alignment & Regulation

Homework #3 Due

Lecture Material:

• Evaluation and Regulation

Suggested Reading:

• Merlin: A Vision Language Foundation Model for 3D Computed Tomography
• HealthBench

Week 9

May 28

ApplicationsEMR Foundation Models (Guest Lecture: Jason Fries)

Lecture Material:

• FMs for Electronic Health Records

Suggested Reading:

• The Shaky Foundations of LLMs and FMs for EHR

Week 10

Jun 2

OpportunitiesEthics of Foundation Models (Guest Lecture: Roxana Daneshjou)

Lecture Material:

• Uncovering Bias in FMs

Suggested Reading:

• Large language models propagate race-based medicine
• Red teaming ChatGPT

Jun 4

Project Presentations DueFinal Presentations

Week 11

Jun 11

Project Reports Due