| 1 |
Jan 6 Monday |
Class Introduction
[slides]
|
Optional Readings:
- U.S. Department of Education, Office of Educational Technology. Artificial Intelligence and Future of Teaching and Learning: Insights and Recommendations, Washington, DC, 2023.
- Litman, D. (2016, March). Natural language processing for enhancing teaching and learning. In Proceedings of the AAAI conference on artificial intelligence (Vol. 30, No. 1).
- CRPE's Think Forward: AI Learning Forum. Wicked Opportunities: Leveraging AI to Transform Education, 2024.
- Common Sense Media. Generative AI in Kβ12 Education: Challenges and Opportunities, 2024.
- Digital Promise. An Ethical and Equitable Vision of AI in Education: Learning Across 28 Exploratory Projects, 2024.
|
| 1 |
Jan 8 Wednesday |
Discovery & Exploration in Educational Language Data
[slides]
|
Required Reading:
- π Nguyen, D., Liakata, M., DeDeo, S., Eisenstein, J., Mimno, D., Tromble, R., & Winters, J. (2020). How We Do Things With Words: Analyzing Text as Social and Cultural Data. Frontiers in Artificial Intelligence, 3.
- π Liu, J., & Cohen, J. (2021). Measuring teaching practices at scale: A novel application of text-as-data methods. Educational Evaluation and Policy Analysis, 43(4), 587-614
Optional Reading:
|
| 2 |
Jan 13 Monday |
Discovery & Exploration in Educational Language Data Parsing, Lexical Analyses
[slides]
|
Required Reading:
- π π Lucy, L., Demszky, D., Bromley, P., & Jurafsky, D. (2020). Content analysis of textbooks via natural language processing: Findings on gender, race, and ethnicity in Texas US history textbooks. AERA Open, 6(3), 2332858420940312.
Optional Reading:
- Markowitz, D. M., Kittelman, A., Girvan, E. J., Santiago-Rosario, M. R., & McIntosh, K. (2023). Taking Note of Our Biases: How Language Patterns Reveal Bias Underlying the Use of Office Discipline Referrals in Exclusionary Discipline. Educational Researcher, 0(0).
- Monroe, B. L., Colaresi, M. P., & Quinn, K. M. (2008).Fightinβ Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict. Political Analysis, 16(04), 372β403.
-
|
| 2 |
Jan 15 Wednesday |
Centering Teachers in the Design & Development of Tools
Guest Visit by Dan Meyer
The Difference Between Great AI and Great Teaching
|
ASU GSV-keynote: The Difference Between Great AI and Great Teaching
Required Reading:
Optional Reading:
|
| 3 |
Jan 20 Monday |
MLK day - No class
HW1 due on Tuesday at 11:59pm
|
|
| 3 |
Jan 22 Wednesday |
An Evaluation-Centered Approach to Developing LLMs for Education
Guest Lecture by Sara Wiltberger and Shubham Milind Phal from Google DeepMind Towards Responsible Development of Generative AI for Education
|
Required Reading:
- π LearnLM Team (2024). LearnLM: Improving Gemini for Learning. arXiv preprint arXiv:2412.16429.
- π Jurenka, I., Kunesch, M., McKee, K. R., Gillick, D., Zhu, S., Wiltberger, S., ... & Ibrahim, L. (2024). Towards responsible development of generative AI for education: An evaluation-driven approach. arXiv preprint arXiv:2407.12687.
|
| 4 |
Jan 27 Monday |
Discovery & Exploration in Educational Language Data Topic Modeling, Clustering, Grounded Exploration
Project Rationale due on Tuesday at 11:59pm
[slides]
|
Required Reading:
Optional Reading:
- Nelson, L. K. (2020). Computational Grounded Theory: A Methodological Framework. Sociological Methods & Research, 49(1), 3β42.
- π Chang, J., Gerrish, S., Wang, C., Boyd-graber, J. L., & Blei, D. M. (2009). Reading Tea Leaves: How Humans Interpret Topic Models. Advances in Neural Information Processing Systems, 288β296.
- Chew, R., Bollenbacher, J., Wenger, M., Speer, J., & Kim, A. (2023). LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Coding. arXiv.
- Chen, N.-C., Drouhard, M., Kocielnik, R., Suh, J., & Aragon, C. R. (2018). Using Machine Learning to Support Qualitative Coding in Social Science: Shifting the Focus to Ambiguity. ACM Transactions on Interactive Intelligent Systems, 8(2), 1β20.
- McCarthy, A. D., & Dore, G. M. D. (2023, July). Theory-grounded computational text analysis. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (pp. 1586-1594).
- Alvero, A. J., Giebel, S., Gebre-Medhin, B., Antonio, A. L., Stevens, M. L., & Domingue, B. W. (2021). Essay content and style are strongly related to household income and SAT scores: Evidence from 60,000 undergraduate applications. Science advances, 7(42), eabi9031.
|
| 4 |
Jan 29 Wednesday |
Using NLP for Educational Measurement
[slides]
|
Required Reading:
- ππ Lee, G.-G., Latif, E., Wu, X., Liu, N., & Zhai, X. (2024). Applying large language models and chain-of-thought for automatic scoring. Computers and Education: Artificial Intelligence, 6, 100213.
- πLatif, E., & Zhai, X. (2024). Fine-tuning ChatGPT for automatic scoring. Computers and Education: Artificial Intelligence, 6, 100210.
Optional Reading:
- Henkel, O., Hills, L., Boxer, A., Roberts, B., & Levonian, Z. (2024). Can Large Language Models Make the Grade? An Empirical Study Evaluating LLMs Ability To Mark Short Answer Questions in K-12 Education. Proceedings of the Eleventh ACM Conference on Learning @ Scale, 300β304.
- Demszky, D., Liu, J., Mancenido, Z., Cohen, J., Hill, H., Jurafsky, D., & Hashimoto, T. B. (2021, August). Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 1638-1653).
- Suresh, A., Jacobs, J., Perkoff, M., Martin, J. H., & Sumner, T. (2022). Fine-tuning Transformers with Additional Context to Classify Discursive Moves in Mathematics Classrooms. Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022), 71β81.
- Hunkins, N., Kelly, S., & D'Mello, S. (2022, March). βBeautiful work, you're rock stars!β: Teacher Analytics to Uncover Discourse that Supports or Undermines Student Motivation, Identity, and Belonging in Classrooms. In LAK22: 12th International Learning Analytics and Knowledge Conference (pp. 230-238).
- Crossley, S. A., Kyle, K., & McNamara, D. S. (2016). The tool for the automatic analysis of text cohesion (TAACO): Automatic assessment of local, global, and text cohesion. Behavior research methods, 48, 1227-1237.
- Hills, O. H. L. (2023). Leveraging Human Feedback to Scale Educational Datasets: Combining Crowdworkers and Comparative Judgement (arXiv:2305.12894). arXiv.
|
| 5 |
Feb 3 Monday |
Round 1 Practice Pitches
HW2 due on Wednesday at 11:59pm |
No Reading
|
| 5 |
Feb 5 Wednesday |
Generative Language Models for Education Simulating Students
[slides]
|
Required Reading:
- ππ He-Yueya, J., Ma, W. A., Gandhi, K., Domingue, B. W., Brunskill, E., & Goodman, N. D. (2024). Psychometric Alignment: Capturing Human Knowledge Distributions via Language Models . arXiv.
- Lu, X., & Wang, X. (2024). Generative Students: Using LLM-Simulated Student Profiles to Support Question Item Evaluation. Proceedings of the Eleventh ACM Conference on Learning @ Scale, 16β27.
Optional Readings:
- Markel, J. M., Opferman, S. G., Landay, J. A., & Piech, C. (2023).GPTeach: Interactive TA Training with GPT Based Students.
- Liu, N., Sonkar, S., Wang, Z., Woodhead, S., & Baraniuk, R. G. (2023). Novice Learner and Expert Tutor: Evaluating Math Reasoning Abilities of Large Language Models with Misconceptions . arXiv.
- Shaikh, O., Chai, V., Gelfand, M. J., Yang, D., & Bernstein, M. S. (2024). Rehearsal: Simulating Conflict to Teach Conflict Resolution (arXiv:2309.12309). arXiv.
- Park, J. S., Popowski, L., Cai, C., Morris, M. R., Liang, P., & Bernstein, M. S. (2022). Social Simulacra: Creating Populated Prototypes for Social Computing Systems. Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, 1β18.
- Saha, S., Hase, P., & Bansal, M. (2023). Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Theory of Mind. arXiv.
- Yue, M., Lyu, W., Mifdal, W., Suh, J., Zhang, Y., & Yao, Z. (2025). MathVC: An LLM-Simulated Multi-Character Virtual Classroom for Mathematics Education (arXiv:2404.06711). arXiv.
- Zhang, Z., Zhang-Li, D., Yu, J., Gong, L., Zhou, J., Hao, Z., Jiang, J., Cao, J., Liu, H., Liu, Z., Hou, L., & Li, J. (2024). Simulating Classroom Education with LLM-Empowered Agents (arXiv:2406.19226). arXiv.
|
| 6 |
Feb 10 Monday |
Generative Language Models for Education
Guest Lecture by Scott Crossley Intelligent Texts in the Classroom
|
Required Reading:
- ππ Morris, W., Crossley, S., Holmes, L., Ou, Chaohua, Dascalu, M., & McNamara, D. (2024). Formative Feedback on Student-Authored Summaries in Intelligent Textbooks using Large Language Models. Journal of Artificial Intelligence in Education.
Optional Reading:
- Crossley, S. A., Choi, J. S., Morris, W., Holmes, L., Joyner, D. & Gupta, V. (2024). Using Intelligent Texts in A Computer Science Classroom: Findings from an iTELL Deployment. Proceedings of 8th Educational Data Mining in Computer Science Education Workshop (CSEDM 2024). Atlanta, GA.
|
| 6 |
Feb 12 Wednesday |
Generative Language Models for Education
LLMs for Student Assessment and Feedback
Guest Lecture by Mei Tan
|
Required Reading for Commentary (π pick one):
- Li, S., & Ng, V. (2024). Automated Essay Scoring: A Reflection on the State of the Art. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (pp. 17876-17888).
- Stahl, M., Biermann, L., Nehring, A., & Wachsmuth, H. (2024). Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generation. arXiv preprint arXiv:2404.15845.
- Mizumoto, A., & Eguchi, M. (2023). Exploring the potential of using an AI language model for automated essay scoring. Research Methods in Applied Linguistics, 2(2), 100050.
Recommended (for Guest Lecture):
Optional Reading:
- Baffour, P., Saxberg, T., & Crossley, S. (2023, July). Analyzing Bias in Large Language Model Solutions for Assisted Writing Feedback Tools: Lessons from the Feedback Prize Competition Series. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023) (pp. 242-246).
- Paulson Gjerde, Kathy, Margaret Y. Padgett, and Deborah Skinner. "The Impact of Process vs. Outcome Feedback on Student Performance and Perceptions." Journal of Learning in Higher Education 13.1 (2017): 73-82.
- Pardos, Zachary A., and Shreya Bhandari. "Learning gain differences between ChatGPT and human tutor generated algebra hints." arXiv preprint arXiv:2302.06871 (2023).
- Matelsky, J. K., Parodi, F., Liu, T., Lange, R. D., & Kording, K. P. (2023). A large language model-assisted education tool to provide feedback on open-ended responses (arXiv:2308.02439). arXiv. http://arxiv.org/abs/2308.02439
|
| 7 |
Feb 17 Monday |
Presidents' Day - No class |
|
| 7 |
Feb 19 Wednesday |
Designing NLP Tools for Empowering Teachers in the Real World
Q&A with Rakiya Brown from TeachFX
[slides]
Experimental Protocol due Wednesday at 11:59pm
|
Required Reading:
- π Nicholson, R., Bartindale, T., Kharrufa, A., Kirk, D., & Walker-Gleaves, C. (2022). Participatory Design Goes to School: Co-Teaching as a Form of Co-Design for Educational Technology. CHI Conference on Human Factors in Computing Systems, 1β17.
Optional Reading:
- Wang, D., Bian, C., & Chen, G. (2024).Using explainable AI to unravel classroom dialogue analysis: Effects of explanations on teachersβ trust, technology acceptance and cognitive load. British Journal of Educational Technology, 55(6), 2530β2556.
- Jacobs, J., Scornavacco, K., Harty, C., Suresh, A., Lai, V., & Sumner, T. (2022). Promoting rich discussions in mathematics classrooms: Using personalized, automated feedback to support reflection and instructional change. Teaching and Teacher Education, 112, 103631.
- Lee, V. R., Clarke-Midura, J., Shumway, J., & Recker, M. (2022). βDesign for Co-Designβ in a Computer Science Curriculum Research-Practice Partnership.
- Kulkarni, C. (2019). Design Perspectives of Learning at Scale: Scaling Efficiency and Empowerment. Proceedings of the Sixth (2019) ACM Conference on Learning @ Scale, 1β11.
|
| 8 |
Feb 24 Monday |
Deploying NLP Tools To Empower Teachers
Guest Visit by Sarah Johnson
Round 2 Practice Pitches due Monday at 11:59pm
|
Required Reading:
Optional Reading:
|
| 8 |
Feb 26 Wednesday |
Deploying NLP Tools To Empower Teachers Experimental Design & Evaluation
Guest visit by Mariah Olson
[slides]
HW3 due Friday at 11:59pm
| Required Reading:
- π π Wang, R. E., Ribeiro, A. T., Robinson, C. D., Loeb, S., & Demszky, D. (2024). Tutor copilot: A human-ai approach for scaling real-time expertise. arXiv preprint arXiv:2410.03017.
- Yun, J., Hicke, Y., Olson, M., & Demszky, D. (2024, July).Enhancing Tutoring Effectiveness Through Automated Feedback: Preliminary Findings from a Pilot Randomized Controlled Trial on SAT Tutoring . In Proceedings of the Eleventh ACM Conference on Learning@ Scale (pp. 422-426).
Optional Reading:
- Wang, R. E., Zhang, Q., Robinson, C., Loeb, S., & Demszky, D. (2024). Bridging the Novice-Expert Gap via Models of Decision-Making: A Case Study on Remediating Math Mistakes. NAACL.
- Demszky, D., Wang, R., Geraghty, S., & Yu, C. (2024, March). Does feedback on talk time increase student engagement? evidence from a randomized controlled trial on a math tutoring platform. In Proceedings of the 14th Learning Analytics and Knowledge Conference (pp. 632-644).
- Demszky, D., Liu, J., Hill, H. C., Jurafsky, D., & Piech, C. (2023). Can automated feedback improve teachersβ uptake of student ideas? Evidence from a randomized controlled trial in a large-scale online course. Educational Evaluation and Policy Analysis, 01623737231169270.
|
| 9 |
March 5 Monday |
Frontiers and Open Questions
|
No reading, besides revisiting reading commentaries, past readings, discussions, homeworks
|
| 9 |
March 7 Wednesday |
Frontiers and Open Questions |
No reading, besides revisiting reading commentaries, past readings, discussions, homeworks
|
| 10 |
March 10 Monday |
Final Pitches |
No Reading |
| 10 |
March 12 Wednesday
|
Final Pitches
Final paper due on Thursday, March 13 at 11:59pm
|
No Reading |
| | | |