SYMSYS 130: Research Methods in the Cognitive and Information Sciences

SYMSYS 130: Research Methods in the Cognitive and Information Sciences

Picture of a tool box
with high tech tools in it

2012-2013 Spring Quarter, Stanford University
3 units
Grading: Ltr-CR/NC
Mon, Wed 3:45 PM - 5:05 PM in 460-127B (in the Linguistics Department office cluster)
Instructor: Todd Davies
Instructor email address: davies at stanford.edu
Course website: http://www.stanford.edu/class/symsys130
Course blog: The Research Thinker

[This version: June 20, 2013 - added link to Final - Instructor's Responses]

Course Description:

Understanding the different methodological approaches used in disciplines that study cognition and information. Philosophical/analytical, formal/mathematical, empirical, and computational thinking styles, with some attention to others as well. Scientific methods and foundational techniques will be emphasized. What assumptions underlie these methods? How can they be combined? How do practitioners of each discipline think differently about problems, and what are the challenges involved in studying or working across them?

Intended Population:

Students looking for a broad introduction to interdisciplinary research and methods in the cognitive and information sciences, as preparation for individual research or more advanced coursework. The course has no prerequisites.

This course fulfills the following concentration requirements for Symbolic Systems undergraduate majors.

Cognitive Science - Area 2 (Empirical Methods)
Decision Making and Rationality - Area 5 (Methodological Foundations)
Human-Computer Interaction - Area 5 (Empirical Methods)

The course is intended to complement, not to substitute for, courses in statistics, logic, formal modeling, and other techniques of research. Although we will discuss these topics, it will be at a conceptual level. My aim is to make the course appropriate both for students who have not yet taken more technical courses, and for those who have taken such courses but who want to learn more about the big picture: how methods fit together, why and how they differ, and how they can be used wisely.

Format:

In-class sessions will consist of interactive lectures and discussion. This will be the first time I have taught the course, so it will be somewhat experimental. There will be three homework assignments and a take-home final, all in short-essay format. The goal of the homeworks and final is to get students thinking critically and creatively about research and its methods. Readings for each session will be posted at least one week before the session, and will all be available electronically. The homeworks will each cover about 1/4 of the course. The final will be comprehensive, but weighted toward the last 1/4 of the course.

There is no TA for the course, but I will be available both in office hours and on the class email list to answer questions about class sessions, the readings, and the assignments.

Grading:

Grades will be assigned based on the following factors.

60% Homework assignments (20% each x 3)
30% Take-home final
10% Attendance and participation

Tentative Schedule:

DATE	TOPIC	READINGS	OTHER LINKS	ASSIGNMENTS
Mon. 4/1	Symbolic Systems and Its Cognate Fields	"What is Sym Sys?", Symbolic Systems Program, Stanford University Paul Thagard, "Cognitive Science", The Stanford Encyclopedia of Philosophy (Fall 2012 Edition), Edward N. Zalta (ed.) "Informatics Defined", School of Informatics, Indiana University-Purdue University Indianapolis	blog entry
Wed. 4/3	The Big 4 Methods (Philosophical, Formal, Empirical, and Computational)	"On the Study of Philosophy" and "Logical Toolkit", Introduction to Philosophy: Classical and Contemporary Readings (Sixth Edition), John Perry, Michael Bratman, and John Martin Fischer (eds.), Oxford University Press, 2012, pp. 3-14 Bertrand Russell, "The Value of Philosophy", Introduction to Philosophy: Classical and Contemporary Readings (Sixth Edition), John Perry, Michael Bratman, and John Martin Fischer (eds.), Oxford University Press, 2012, pp. 18-21 James D. Evans, "Getting the Scientific Attitude" (Chapter 1), Invitation to Psychological Research, Holt, Rinehart, and Winston, 1984, pp. 3-15 Herbert A. Simon, "Understanding the Natural and the Artificial Worlds" (Chapter 1), The Sciences of the Artificial, MIT Press, 1981, pp. 2-29	blog entry
Mon. 4/8	Argumentation	Edmund L. Gettier, "Is Justified True Belief Knowledge?", Analysis 23:121-123, 1963 "Human Nature: Justice Versus Power" - Noam Chomsky Debates With Michel Foucault, 1971 [Chomsky.Info] see also the written notes at "The Chomsky-Foucault Debate: Complete Video Recording from 1971" [Aphelis.Net] (Video is not subtitled)	blog entry
Wed. 4/10	Deduction	Ernest Nagel and James R. Newman (Edited and with a new Forward by Douglas R. Hofstadter), Gödel's Proof (Revised Edition), New York University Press, 2001	blog entry
Mon. 4/15	Induction	John Vickers, "The Problem of Induction", The Stanford Encyclopedia of Philosophy (Spring 2013 Edition), Edward N. Zalta (ed.) Joshua B. Tenenbaum, Charles Kemp, Thomas L. Griffiths and Noah D. Goodman, "How to Grow a Mind: Statistics, Structure, and Abstraction", Science 331(6022):1279-1285, 2011	blog entry	*Homework 1 distributed*
Wed. 4/17	Scientific Explanation	David Hume, "Of the Idea of Necessary Connexion" - Part I, An Enquiry Concerning Human Understanding, Section VII, Harvard Classics Edition (1910), originally published in 1748 "Philosophy of Science", Wikipedia, version 01:14, 28 March 2013‎ [.pdf] Karl R. Popper, "Science as Falsification", Conjectures and Refutations, Routledge and Keagan Paul, 1963, pp. 33-39	blog entry
Mon. 4/22	Prediction	Daniel Kahneman and Amos Tversky, "On the Psychology of Prediction", Psychological Review 80(4):237-251, 1973 Daniel Kahneman and Gary Klein, "Conditions for Intuitive Expertise: A Failure to Disagree", American Psychologist 64(6):515-526, 2009	blog entry	*HOMEWORK 1 DUE*
Wed. 4/24	Measurement and Meaning	Clyde H. Coombs, Robyn M. Dawes, and Amos Tversky, "Psychological Measurement Theory" (Chapter 2), Mathematical Psychology: An Elementary Introduction, Prentice-Hall, Inc., 1970, pp. 7-30 Jean-Claude Falmagne, "Measurement Theory and the Research Psychologist", Psychological Science 3(2):88-93, 1992	blog entry
Mon. 4/29	Populations and Sampling	Amitav Banerjee and Suprakash Chaudhury, "Statistics Without Tears: Populations and Samples", Ind Psychiatry J 19(1): 60–65, 2010 Russell V. Lenth, "Some Practical Guidelines for Effective Sample Size Determination", The American Statistician 55(3), August 2001	blog entry
Wed. 5/1	Testing and Surveying	Gerianne de Klerk, "Classical Test Theory (CTT)", In M. Born, C.D. Foxcroft & R. Butter (Eds.), Online Readings in Testing and Assessment, International Test Commission, 2008 Jon A. Krosnick, "Survey Research", Annual Review of Psychology 50:537-567, 1999	blog entry	Homework 1 returned Homework 2 distributed
Mon. 5/6	Experimentation	William M. K. Trochim, "Design" (through "Hybrid Experimental Designs"), Research Methods Knowledge Base (October 20, 2006 Edition) <http://www.socialresearchmethods.net>	blog entry
Wed. 5/8	Observation	William M. K. Trochim, "Quasi-Experimental Design", "Relationships Among Pre-Post Designs", "Qualitative Measures", and "Unobtrusive Measures", Research Methods Knowledge Base (October 20, 2006 Edition) <http://www.socialresearchmethods.net> OPTIONAL: Norm Scheiber, "Freaks and Geeks: How Freakonomics Is Ruining the Dismal Science", The New Republic, April 2, 2007 (with a follow-up exchange between Steven Levitt and Norm Scheiber)	blog entry	*HOMEWORK 2 DUE*
Mon. 5/13	Data Analysis	William M. K. Trochim, "Analysis" (through "Randomized Block Analysis"), Research Methods Knowledge Base (October 20, 2006 Edition) <http://www.socialresearchmethods.net> "Big Data", Wikipedia, version 18:03, 26 April 2013‎ OPTIONAL: Gary Marcus, "Steamrolled By Big Data", The New Yorker, April 3, 2013	blog entry
Wed. 5/15	Modeling and Simulation	Paul M. Churchland, "Cognitive Activity in Artificial Neural Networks", in D. Osherson and E. E. Smith (editors), An Invitation to Cognitive Science, Volume 3: Thinking, MIT Press, 1990 David Ferrucci, Eric Brown, Jennifer Chu-Carroll, James Fan, David Gondek, Aditya A. Kalyanpur, Adam Lally, J. William Murdock, Eric Nyberg, John Prager, Nico Schlaefer, and Chris Welty, "Building Watson: An Overview of the DeepQA Project", AI Magazine 31(3), Fall 2010 Jonathan Rauch, “Seeing Around Corners”, The Atlantic Monthly, April 2002	blog entry	*Homework 3 distributed*
Mon. 5/20	Problem Solving	John R. Anderson, "Problem Solving" (Chapter 8), Cognitive Psychology and Its Implications, Sixth Edition, Worth Publishers, 2005, pp. 242-278 Jeannette M. Wing, "Computational Thinking", Communications of the ACM 49(3):33-35, 2006 OPTIONAL: George Polya, Part I. "In the Classroom" and Part II. "How to Solve It - A Dialogue", How to Solve It : A New Aspect of Mathematical Method, Doubleday Anchor Books, 1957, pp. 1-36	blog entry	*Homework 2 returned*
Wed. 5/22	Design Thinking	Don Norman, "Cognitive Engineering", in Donald A. Norman and Stephen W. Draper (editors), User Centered System Design: New Perspectives on Human-Computer Interaction, Lawrence Erlbaum Associates, 1986, pp. 31-61 Tim Brown, "Design Thinking", Harvard Business Review, June 2008 Andreas Holzinger, "Usability Engineering Methods for Software Developers", Communications of the ACM 48(1):71-74, 2005	blog entry
Fri. 5/24, 5pm				*HOMEWORK 3 DUE*
Wed. 5/29	Research Ethics	Norbert Wiener, "Cybernetics in History" (Chapter I) and "Role of the Intellectual and the Scientist" (Chapter VII), The Human Use of Human Beings (1954 Edition), Free Association Books, 1989, pp. 15-27 and 131-135 William M. K. Trochim, "Ethics in Research", Research Methods Knowledge Base (October 20, 2006 Edition) <http://www.socialresearchmethods.net> Dan Jones, "A WEIRD View of Human Nature Skews Psychologists' Studies", Science 328:1627, 2010 Martha J. Farah, "Neuroethics: The Ethical, Legal, and Societal Impact of Neuroscience", Annual Review of Psychology 63:571-591, 2012 OPTIONAL: Yudhijit Bhattacharjee, "The Mind of a Con Man", New York Times Magazine, April 26, 2013 JUST FOR LAUGHS: "Kripke Resigns As Report Alleges He Faked Results of Thought Experiments", fauxphilnews, February 22, 2012	blog entry
Mon. 6/3	Communicating Research	William M. K. Trochim, "Write-Up", Research Methods Knowledge Base (October 20, 2006 Edition) <http://www.socialresearchmethods.net Oded Goldreich, "How to Write a Paper", January 18, 2013 Joseph P. Simmons, Leif D. Nelson, and Uri Simonsohn, "False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything As Significant", Psychological Science 22(11) 1359 –1366, 2011 Gary Marcus, "Six Ways to Clean Up Science", The New Yorker, December 24, 2012 OPTIONAL: Richard Feynman, "Cargo Cult Science", Commencement Address, Caltech, 1974	blog entry
Wed. 6/5	Current Debates (with a guest appearance by Tom Wasow)	Thomas Wasow and Jennifer Arnold, "Intuitions in Linguistic Argumentation", Lingua 115(11):1481-1496, 2005 Jon Sprouse and Diogo Almeida, "The empirical status of data in syntax: A reply to Gibson and Fedorenko", Language and Cognitive Processes 28(3):222-228, 2013 Edward Gibson, Steven T. Piantadosi and Evelina Fedorenko, "Quantitative methods in syntax/semantics research: A response to Sprouse and Almeida", Language and Cognitive Processes, 28(3):229-240, 2013	blog entry	Homework 3 returned Take-home final distributed
Mon. 6/10, 3:15pm				*TAKE-HOME FINAL DUE*
Thurs. 6/20, 4:30 pm				Take-home final returned