Judge the Tutor

Two tutor replies. One helped the learner keep going. Can you tell which?

An AI tutor in a free probability course has helped about 2,000 learners. In each round below, two learners typed the same thing, and the tutor answered each one differently.

After one reply, the learner tried the next question. After the other, the learner gave up, got confused, or left. Which reply came before the real try?

The tutor's replies are word for word. The learner's part is only a short answer, like "18" or "yes".

How to play

Read the two replies.
Pick the one you think helped.
See what happened, and compare your score with an AI judge and with "pick the shorter reply".

You0 / 0

AI judge0 / 0

Shorter reply0 / 0

Round1

Step

The tutor had said

The learner typed

Reply A

Reply B

Hard, right?

On 173 pairs like these, Claude picked right 59% of the time. "Pick the shorter reply" got 56%. Guessing gets 50%. What a learner does next depends mostly on the learner, so nobody gets close to 100%.

Replies that often came before a learner stalled: giving the answer away, a vague "why?", repeating, or ending with "Clear?".

Talk about it

What did you look for? Say it in one sentence. Could a tutor that knows your rule fool it?
When you were wrong, was the reply worse, or was the learner just different?
"The learner's next message" is a short-term result. What would be a better one?