CS 124 / LING 180 From Languages to Information
Dan Jurafsky, Winter 2017

Week 7: Group Exercises on QA and Chatbots on Mobile Devices
Feb 21, 2017

Your goal today, appropriately right between the QA and Chatbot homeworks, is to explore one of the most common applications of question answering and chatbots: personal assistants on phones. You are going to explore the conversational and QA features of the personal assistants on your smart phone, Siri, Google Now, Cortana, etc. It's best if you have more than one personal assistant in your group, so you can compare and contrast! But if not, it's ok, just work on one.

You should have read this NY Times article from last year before starting.

After each of the following four questions, we will report back some of your results to the whole room, so make note of your group's interesting results.

  1. Speech Recognition Performance Write a couple of texts or emails using voice. Answer these questions with your group:

  2. Task performance: Test out some tasks like making/canceling some calendar appointments or looking for a business (restaurant or etc.). Analyze any errors and note interesting examples. (For example did they fail because of speech recognition (the wrong words were recognized) or Natural Language Understanding (the words were right, but the system still didn't understand).)

  3. Chatbot and Discourse Performance: Does the system feel conversational? Fluent? For example can it make use of the discourse context (e.g. some previous turns in the conversation)? Does it remember things you told it earlier? Or how about personality? Or does it ever surprise you in a positive way?

  4. Building the better conversational agent: Let's try to design a better dialogue system. Discuss in your groups some abilities you'd like to add, especially those that require real dialogue (i.e. dialogues that have more than one turn each, not just simple command and control).