Psych 254: Lab in Experimental Methods

Winter 2017

Class Details

Meeting Days: Monday, Wednesday, Friday
Meeting Times: 1:30PM - 2:50PM
Meeting Places: 420-050 (by the Thai Cafe Window)

Please send all correspondence to the teaching staff at psych254-win1617-staff (at)


Michael Frank

mcfrank (at) stanford (dot) edu
Room 278, Jordan Hall
Office Hours: by appointment here

Teaching Assistants

Cayce Hook

cjhook (at) stanford (dot) edu
Room 378, Jordan Hall
Office Hours: by appointment here.

Eric Smith

ensmith (at) stanford (dot) edu
Room 392, Jordan Hall
Office Hours: by appointment here.

Course Overview

Course Goals

Students completing Psych 254 will:

  • Be able to perform a psychology experiment on the web,
  • Be able to analyze a study in terms of reliability (e.g. power, statistical choices), and validity (e.g. design, sampling),
  • Master best practices for experimental data management, storage, and analysis, and
  • Have a reproducible workflow for experimental data analysis and visualization, including comfort with a variety of R packages, including ggplot2, dplyr, tidyr, and lmer.


This course is a requirement for graduate students in the Psychology department, and generally assumes the skills of an entering graduate student. In particular, students must have:

  • Sufficient background in statistics, having taken Psych 252 or equivalent.
  • Some basic programming experience in R is important as well.
Without these prerequisites, keeping up with course content and assignments will be challenging. If you do not have them, please speak with someone from the course team.

In addition, we will be relying heavily on version control using git and github. Please take a moment to set up a git account at, and to get it working on your computer before class starts. You may also want to look into requesting an education account, at


Readings are in this folder. We will be using scans from two books:

  • Gelman, A. & Hill, J. (2006). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press.
  • Rosenthal, R., & Rosnow, R. L. (2008). Essentials of behavioral research: Methods and data analysis (3rd ed.). McGraw-Hill.
They are great books; I hope they will be useful references for students after the course is finished, so if you would like to try and buy either, go ahead.

Problem Sets

We will have a number of short problem sets that you begin in class and complete at home. Problem sets will be done using R Markdown. We recommend using R Markdown with RStudio.


  • 30%: Problem Sets (4 problem sets)
  • 50%: Final project components, including presentations, data collection, analysis, and writeup
  • 10%: Attendance and participation

Final Project

This course follows the model described in Frank & Saxe (2012): we will be doing replications of recent published papers of interest in order to provide independent, converging evidence for the reproducibility of the scientific literature. The hope is that student projects will lead to broadly-useful knowledge about the viability of published results in an online context.

The final project for this course will be a high-quality, web-based replication of a single experiment from a published paper from Psychological Science in the year 2015. Students will choose a paper and attempt a replication on the web. Your goal will be a replication report suitable for submission to the Open Science Collaboration or, using the Open Science Collaboration replication template for the final writeup ( You can find more information at this link.

Other Policies

  • Class Practicum
    Please bring your computer. You will not need to take notes (we will post all code and PowerPoint slides for each class), but we will be using R or some other piece of software during the second half of every class.

  • Working in groups
    Work for the course will include both problem sets and a final project. For the problem sets, you are encouraged to work in groups. Your writeup must be your own, however. Please mark who you worked with on your writeup. For the final project, you must complete your own replication project, but you are encouraged to ask for help from your peers.

  • Honor Code & Late Policy.
    One-half grade will be subtracted from the assigned grade for each day any assignment is submitted late. Please familiarize yourself with Stanford's honor code, available on the judicial affairs website. We will adhere to it and follow through on its penalty guidelines.

  • Students with Documented Disabilities
    Students who may need an academic accommodation based on the impact of a disability must initiate the request with the Office of Accessible Education (OAE). Professional staff will evaluate the request with required documentation, recommend reasonable accommodations, and prepare an Accommodation Letter for faculty dated in the current quarter in which the request is made. Students should contact the OAE as soon as possible since timely notice is needed to coordinate accommodations. The OAE is located at 563 Salvatierra Walk (phone: 723-1066, OAE website).


Below you will find references for various skills we will be building in class and in problem sets.

You can visit the file directly at this link. Please comment on this doc if you have found a resource that may be helpful to others. Thanks!