| Date | Topic | Description | Events | Deadlines |
|---|---|---|---|---|
| 9/23 | Introduction |
LLMs hallucinate. How to turn LLMs into useful, dependable agents? Concept of computational thinking. Carefully designed LLM-based algorithms for reading, writing, retrieving data; scientific research—curation, insight discovery. Accountable AI. Course design and outline. | ||
| 9/28 | How to RAG without hallucination |
How to create a hallucination-free conversational bot grounded on free text? RAG text retrieval; Summarization; Verifying generation; Response generation; Detecting inconsistencies in corpora. Evaluation methodology. System: WikiChat | Homework 1 out | |
| 9/30 | Research Agents | How to use LLMs to curate knowledge in an open domain? Research in the pre-writing stage by iteratively searching and reading from different perspectives; adding interactivity to allow personalization. System: STORM / Co-STORM | Student intro form due | |
| 10/5 | Research project ideas | What are the ongoing research projects that students can participate in? The major themes: creating a deep researcher; acquisition of new data; data analysis on databases, long documents, literature. Formal semantic representations: extracting Satisfiability Modulo Theories (SMT), Knowledge Graphs (KG), rule-based conversational policies. Real-life applications from cancer drug resistance research, sustainability modeling, clinical trial matching, journalism. | ||
| 10/7 | Introduction to Agents for Structured and Hybrid Data Monica |
How to create a hallucination-free conversational bot grounded on structured data? Semantic parsing; Databases; Expressiveness of database queries; Few-shot prompting on small schemas; Handling enumerated types; Comparison with human annotations. | Project proposal assignment out | Homework 1 due |
| 10/12 | Student project ideas | Students pitching preliminary project ideas | ||
| 10/14 | A Structured / Unstructured Query Language |
How to answer questions combining structured and unstructured data? SUQL: the first query language that combines information retrieval (NL) with database queries (SQL); Language design; Evaluation methodology. | ||
| 10/19 | Project Proposals | Groups present their proposals | Project proposals due | |
| 10/21 | Project Proposals | Groups present their proposals | ||
| 10/26 | Reactive Agents for Parsing Knowledge Graph Queries |
Knowledge graphs succinctly represent facts from many different sources—e.g. Wikidata has 15B facts; Inaccessibility due to the complexity of writing queries in SPARQL; Reactive agent workflow. System: SPINACH, deployed at Wikimedia. | ||
| 10/28 | Deep Researcher | How to answer questions that require reading, analyzing, and comparing across large documents in a set? LLM contexts are never large enough. Automatic schematization, database construction, and data reconciliation. System: SLIDERS | ||
| 11/2 | Minimizing hallucination in task-oriented agents |
How to create a task-oriented agent that does not hallucinate? Genie Worksheet is the first high-level task-oriented agent specification language that lets users focus on the task to be done; low-level dialogue implementation details are left to the implementation of the language. How to scale the creation of effective and reliable agents across different domains easily? Implementation of the Genie Worksheet; formal dialogue state representation; semantic parsing; dialogue state tracking; response generation. | ||
| 11/4 | Insight discovery | How to derive new insights from large tables of data? Principles from Exploratory Data Analysis and Data Storytelling. Reframes deep research over structured data as a thesis-driven analytical process. System: DataSTORM | ||
| 11/9 | Accountable AI: SMT | Accountability means that LLMs can provide a faithful rationale that is sound and complete. High-stakes decision making such as clinical trial matching requires better recall and precision than conventional embedding-similarity based retrieval. Constraint-satisfaction-based retrieval. Formulating constraints as SMT (satisfiability modulo theories) permits the use of a theorem prover as an accountable decision maker. Systems: SatIR, VERDICT | ||
| 11/11 | Coding Agents | Coding agents such as Claude Code and OpenAI Codex can perform complex tasks on their own. Anatomy of coding agents: an LLM, tools, and a harness; meta-optimization techniques of harnesses. | ||
| 11/16 | VLMs for Transcription |
Transcription of historical manuscripts and prints is challenging due to the use of old languages, unusual layouts, and the quality of the digitization. Data sets and fine-tuning of vision-language models to make historical documents accessible. | ||
| 11/18 | Training LLMs | How do we create LLMs? Instruction-following models; training data. | ||
| Thanksgiving Break | ||||
| 11/30 | No class | |||
| 12/2 | Project Presentations | Groups present their final projects. | Final project presentation |