Assn 3, Design the Future of Student Work

Due: Monday, Dec 7th 11:59PM
Grace: Tuesday, Dec 8th 11:59PM

Context Every assignment you've done so far in CS398, you've had access to a coding agent. That changes what an assignment is even for. If an agent can write the code, the point of the assignment can't just be "did you produce working code" — it has to be something an agent can't do for you: making a judgment call, designing an evaluation, deciding what "good" means. This assignment asks you to think explicitly about that shift, and then do something about it.

Question 0: Autopsy an Assignment

Pick one existing homework assignment — it can be one of ours from earlier this quarter, one from another class you've taken, or one you find online. Give it to a coding agent with no other help from you, and see how far it gets.

Include the transcript (or a representative excerpt) in your writeup. Then answer: which parts of the assignment did the agent complete trivially? Which parts, if any, required a decision that you don't think the agent made well, or that it made silently without flagging that a decision was being made? Be specific — point to an exact step, prompt, or line of output.

Question 1: Design a New Assignment

Design and write the full handout for a brand new homework assignment, for any real course (CS398 itself is a fine choice, but doesn't have to be). Your assignment must:

Have a clear, defensible learning objective — what should a student understand after finishing it that they didn't before?
Resist being "solved" by simply handing your handout to an agent. Explain specifically what part of your assignment you expect an agent to fail at, or to do in a way a student still has to catch and correct.
Include a grading rubric or evaluation plan an instructor could actually use, not just "the code should work."

Pilot your assignment on at least one real person (a classmate, a friend, anyone willing). Report what happened: where did they get stuck, where did they lean on an agent, did the assignment actually test what you intended it to test? Be honest about what didn't work — a pilot that reveals a flaw in your design is more valuable to us than a pilot that "went great."

Question 2: Design the Grader

For the assignment you designed in Question 1, build at least a rough prototype of a tool that grades or gives feedback on student submissions to it (it does not need to be polished — a script, a prompt, a notebook is fine).

Then answer honestly: having built it, would you actually trust it to grade real students unsupervised? What specifically would you insist stays human, and what are you comfortable automating? If your answer is "none of it should be automated," that's a legitimate answer — but justify it using something you observed while building the grader, not just a general worry.

Optional Question 3: Exploratory Learning

Optionally do any work that you think would benefit yourself and/or the field of computational education. Report what you did. If you want to replace a problem on the pset with your work run it by us!