# How to format oral exam questions

You are writing questions for one-on-one oral exams in a statistics course. During the exam, the
question appears on a laptop screen next to RStudio. The student answers out loud and works in R,
while a TA listens, asks follow-up questions, and grades with a rubric. A web app renders the files
you write, so follow this format exactly.

## What to deliver

- **One Markdown file per question**, named `<code>.md`, e.g. `q001.md`, `q002.md`, ....
  - The code is the file name. It's how questions are assigned to students, so every code must be
    unique.
  - Codes may contain letters, digits, `-`, `_` and `.`. No spaces.
- **Supporting files** (images, data) go alongside the questions, and are referred to by file name.
  - Start each one's name with its question code, e.g. `q007-scatter.png`, so names never collide.
- **How questions are used:** each question is given to one exam time. Every student who starts at
  that time gets it, and different times get different questions. So when asked for, say, 30
  questions, make 30 distinct ones of similar difficulty. Don't assign them to students; the
  instructor matches questions to times after the schedule is made.
- **Packaging:** all files at the top level of a single folder, compressed into one `questions.zip`.
  - If you can create files, create the zip.
  - If you can't, output every file in one reply. Put each file right after a line of exactly this
    form, with nothing between files: `=== FILE: q001.md ===`. Don't wrap the files in code fences.
- Write only the files' contents, with no commentary inside them. Use UTF-8 text.

## The shape of a question file

```markdown
---
title: "A short label for staff"
---
The question the student sees.

<!-- notes -->
Notes that only the TA sees.
```

- **Front matter** (between the `---` lines) is optional. It holds a `title`, which is shown only to staff;
  put it in quotes.
- **The student sees only what's above the `<!-- notes -->` line.** Never put answers, hints, or the
  solution there.
- **Everything below `<!-- notes -->` is for the TA only.** Always include:
  - a model answer, with the key numbers and R output the student should get
  - what a strong answer must mention
  - common mistakes and misconceptions to watch for
  - two or three follow-up questions the TA can ask, from easier to harder

## Markdown rules

- **Paragraphs** are separated by a blank line. A single line break inside a paragraph is ignored.
- **Lists:** `1.` for numbered items and `-` for bullets. Put a blank line before a list. Indent nested
  items by 4 spaces.
- **Emphasis:** `**bold**` and `*italic*`. Don't use headings (`#`); use bold text instead.
- **Tables:** standard pipe tables, with a header row and a `|---|---|` separator line.
- **R code:** fenced blocks starting with ```` ```r ```` and ending with ```` ``` ````, at the left margin,
  not indented inside a list item. For short code inside a sentence, use backticks: `` `prop.test(189, 420)` ``.
- **HTML:** don't use any, except to lay out images or embed `<video>` or `<audio>`. Never use
  `<script>` or `<style>`.

## Math

- **Delimiters:** LaTeX inside `$...$` for inline math, and `$$...$$` for displayed math.
  - Put displayed math on its own lines, with blank lines before and after.
  - Don't put a space just inside the dollar signs: `$\hat p$`, not `$ \hat p $`.
- **Money:** write a dollar sign as `\$`, e.g. `\$25`, so it is never mistaken for math.
- **The LaTeX is rendered by KaTeX.** Use standard math commands:
  - `\frac`, `\sqrt`, `\bar`, `\hat`, `\sum`, `\int`
  - `\mathbb{E}`, `\operatorname{Var}`, `\text{...}`
  - Greek letters, `\sim`, `\le`, `\approx`
  - `pmatrix`/`bmatrix`, `cases`
  - For several aligned lines, use `\begin{aligned} ... \end{aligned}` inside `$$...$$`.
- **Don't use:** `\label`, `\ref`, `\eqref`, `\newcommand`, `\usepackage`, or numbered environments such as
  `equation` and `align`.

## Images

- **Format:** PNG, JPG, or SVG files, included in the zip.
- **Referring to an image:** `![description of the image](q007-scatter.png)`.
- **Size:** set it by adding `{: width=500 }` right after, e.g. `![...](q007-scatter.png){: width=500 }`.
  Keep images at most about 700 pixels wide.
- **Description:** always write a real one; it is read aloud by screen readers.
- **If you can't make image files:**
  - For simple diagrams, write the SVG file yourself (it's plain text).
  - For data plots, write an R script `make-figures.R` that, run in the folder, saves every image you
    refer to with exactly that file name. Use base R or ggplot2, around 1000 × 700 pixels.

## Data

- **For loading in R,** put the data at a public web address and show the line to load it:
  `survey <- read.csv("https://...")`.
- **Alternatively,** include a CSV in the zip and link to it: `[q007-survey.csv](q007-survey.csv)`. The
  student clicks to download it, then reads it from their Downloads folder.
- **Links** open in a new tab, so following one never leaves the exam page.

## Parts revealed one at a time (optional)

- A line containing only `+++` splits a question into parts, which the TA reveals one at a time.
  Use it only when asked.
- Never put a line containing only `+++` anywhere else.

## Length

- **Keep the student-facing part short:** under about 200 words, with at most three sub-questions. It's shown on
  half a laptop screen next to RStudio, and the exam lasts only about 10 minutes.
- **Leave the depth to the TA's follow-up questions** in the notes.

## Many versions of one question

When asked for different versions of a question, for example so each student gets their own:

- keep the structure and wording identical
- vary the context, numbers, or data
- give each version its own code and its own data and image files
- make sure every version's notes have the answers for *that* version

## Complete example: `q007.md`

````markdown
---
title: "CI for a proportion: survey of R use"
---
A survey of 420 Stanford students found that 189 had used R before taking this class.

![Bar chart of students who had and had not used R](q007-bars.png){: width=450 }

1. Use R to compute a 95% confidence interval for $p$, the proportion of all Stanford students
   who have used R.
2. Explain what the interval means to someone who hasn't taken statistics.

```r
prop.test(189, 420)
```

<!-- notes -->
**Answer.** $\hat p = 189/420 = 0.45$. `prop.test` gives roughly (0.40, 0.50). The normal
approximation $\hat p \pm 1.96\sqrt{\hat p(1-\hat p)/n}$ gives (0.402, 0.498).

**A strong answer** says the interval is a range of plausible values for the population
proportion, and that the 95% describes how often the method captures $p$.

**Common mistakes:** saying there is a 95% probability that $p$ is in this interval; confusing
$\hat p$ with $p$.

**Follow-ups:**
1. What would happen to the interval with 840 students?
2. Is the sample representative of all Stanford students?
3. When is the normal approximation reasonable here?
````

## Before you finish, check every file

- [ ] It's named `<code>.md`, and no two codes are the same.
- [ ] It has a `<!-- notes -->` line, with nothing revealing above it.
- [ ] Every `$` that starts math has a matching closing `$`, and money is written as `\$`.
- [ ] Every image or data file it mentions is included, with exactly that file name.
- [ ] Lists have a blank line before them, and code blocks start at the left margin.