STATS364: Selective Inference
=============================

Syllabus
--------

Instructor & TAs
~~~~~~~~~~~~~~~~

Instructors
^^^^^^^^^^^

`Jonathan Taylor <http://statweb.stanford.edu/~jtaylo>`__

-  `Email <https://stanfordwho.stanford.edu/auth/lookup?search=Jonathan%20Taylor>`__
-  Office hours: M 3:30-5:30

Email list
----------

The course has an email list: stats364-spr1920-staff@lists.stanford.edu

Schedule
--------

MW 1:30-2:50, Zoom

Prerequisites
-------------

A course in inference of 300A.

Course description
------------------

This course is focused on inferential methods in the context where, as
Candes has put it: “Scientists collect data first and ask questions
later.” This practice necessitates the need for inferential tools with
statistical guarantees recognizing these questions are derived from the
data we hope to use to answer such questions with hypothesis tests,
confidence intervals, posterior distributions, etc.

Common approaches to this problem include *simultaneous* approaches
(e.g. multiple comparisons procedures that control FWER or FDR) and
*conditional* (e.g. adaptive clinical trials such as “drop the losers”).

This class will have the format of a **reading group** mixed with some
lectures. We *may* have some guest lecturers TBD.

Generally speaking, Mondays will be a lecture and Wednesdays a
discussion of assigned reading.

Rough outline
~~~~~~~~~~~~~

Below is a rough outline of our first 6 weeks. Subject to changes in
pace, interest in topics among students in the class.

.. raw:: html

   <table>

.. raw:: html

   <tr>

.. raw:: html

   <td width="100">

Week

.. raw:: html

   </td>

.. raw:: html

   <td>

Monday lecture

.. raw:: html

   </td>

.. raw:: html

   <td>

Wednesday reading

.. raw:: html

   </td>

.. raw:: html

   </tr>

.. raw:: html

   <tr>

.. raw:: html

   <td>

1 (M: 4/6)

.. raw:: html

   </td>

.. raw:: html

   <td>

`Course introduction <notebooks/Course-introduction.html>`__
(`notebook <https://nbviewer.jupyter.org/format/slides/url/web.stanford.edu/class/stats364/notebooks/Course-introduction.ipynb>`__)

.. raw:: html

   </td>

.. raw:: html

   <td>

.. raw:: html

   <ul>

.. raw:: html

   <li>

Benjamini, Y. `Simultaneous and selective inference: Current successes
and future challenges. <https://www.ncbi.nlm.nih.gov/pubmed/21154895>`__

.. raw:: html

   <li>

Taylor, J. and Tibshirani, R. `Statistical learning and selective
inference <https://www.pnas.org/content/112/25/7629>`__

.. raw:: html

   <li>

`Brief notes <notebooks/Reading1.html>`__

.. raw:: html

   </ul>

.. raw:: html

   </td>

.. raw:: html

   </tr>

.. raw:: html

   <tr>

.. raw:: html

   <td>

2 (M: 4/13)

.. raw:: html

   </td>

.. raw:: html

   <td>

`Multiple comparisons I <notebooks/Multiple-Comparisons-I.html>`__
(`notebook <https://nbviewer.jupyter.org/format/slides/url/web.stanford.edu/class/stats364/notebooks/Multiple-Comparisons-I.ipynb>`__)

.. raw:: html

   </td>

.. raw:: html

   <td>

.. raw:: html

   <ul>

.. raw:: html

   <li>

Berk et al. `Valid Post-Selection
Inference <https://projecteuclid.org/euclid.aos/1369836961>`__

.. raw:: html

   <li>

Taylor, J. `A selective survey of selective
inference <https://www.mathunion.org/fileadmin/ICM/Proceedings/ICM2018/ICM-2018-vol4-ver1-eb.pdf>`__

.. raw:: html

   <li>

`Brief notes <notebooks/Reading2.html>`__

.. raw:: html

   </ul>

.. raw:: html

   </td>

.. raw:: html

   </tr>

.. raw:: html

   <tr>

.. raw:: html

   <td>

3 (M: 4/20)

.. raw:: html

   </td>

.. raw:: html

   <td>

Multiple comparisons II `Guest lecturer: Arun
Khuchibhotla <notebooks/Lecture_VIDE.pdf>`__
`Supplement <notebooks/main_chap_1.pdf>`__

.. raw:: html

   </td>

.. raw:: html

   <td>

.. raw:: html

   <ul>

.. raw:: html

   <li>

Lockhart et al. + Discussants `A significance test for the
LASSO <https://projecteuclid.org/euclid.aos/1400592161>`__

.. raw:: html

   <li>

Zhong and Prentice `Bias-reduced estimators and confidence intervals for
odds ratios in genome-wide association
studies. <https://www.ncbi.nlm.nih.gov/pubmed/18310059>`__

.. raw:: html

   <li>

`Brief notes <notebooks/Reading3.html>`__

.. raw:: html

   </ul>

.. raw:: html

   </td>

.. raw:: html

   </tr>

.. raw:: html

   <tr>

.. raw:: html

   <td>

4 (M: 4/27)

.. raw:: html

   </td>

.. raw:: html

   <td>

`Conditional inference I (simple
problem) <notebooks/Conditional-I.html>`__
(`notebook <https://nbviewer.jupyter.org/format/slides/url/web.stanford.edu/class/stats364/notebooks/Conditional-I.ipynb>`__)

.. raw:: html

   </td>

.. raw:: html

   <td>

.. raw:: html

   <ul>

.. raw:: html

   <li>

`Drop the losers <notebooks/Drop-the-losers.html>`__
(`notebook <https://nbviewer.jupyter.org/format/slides/url/web.stanford.edu/class/stats364/notebooks/Drop-the-losers.ipynb>`__)

.. raw:: html

   <li>

Leeb and Potscher `Can one estimate the conditional distribution of
post-model-selection
estimators? <https://projecteuclid.org/euclid.aos/1169571807>`__

.. raw:: html

   <li>

Lee, Sun, Sun and Taylor `Exact post-selection inference, with
application to the
lasso <https://projecteuclid.org/euclid.aos/1460381681>`__

.. raw:: html

   <li>

`Brief notes <notebooks/Reading4.html>`__

.. raw:: html

   </ul>

.. raw:: html

   </td>

.. raw:: html

   </tr>

.. raw:: html

   <tr>

.. raw:: html

   <td>

5 (M: 5/4)

.. raw:: html

   </td>

.. raw:: html

   <td>

`Conditional inference II (LASSO) <notebooks/Conditional-II.html>`__

.. raw:: html

   </td>

.. raw:: html

   <td>

.. raw:: html

   <ul>

.. raw:: html

   <li>

Fithian, Sun and Taylor `Optimal inference after
selection <https://arxiv.org/abs/1410.2597>`__

.. raw:: html

   <li>

`Brief notes <notebooks/Reading5.html>`__

.. raw:: html

   </ul>

.. raw:: html

   </td>

.. raw:: html

   </tr>

.. raw:: html

   <tr>

.. raw:: html

   <td>

6 (M: 5/11)

.. raw:: html

   </td>

.. raw:: html

   <td>

Conditional inference III (General setup)

.. raw:: html

   <ul>

.. raw:: html

   <li>

`Data carving <notebooks/Data-carving.html>`__
(`notebook <https://nbviewer.jupyter.org/format/slides/url/web.stanford.edu/class/stats364/notebooks/Data-carving.ipynb>`__)

.. raw:: html

   </ul>

.. raw:: html

   </td>

.. raw:: html

   <td>

.. raw:: html

   <ul>

.. raw:: html

   <li>

Fithian, Taylor, Tibshirani and Tibshirani `Selective sequential model
selection <https://arxiv.org/abs/1512.02565>`__ (**Discussed in
discussion of Fithian, Sun, Taylor**)

.. raw:: html

   <li>

Candes and Barber `Controlling the false discovery rate via
knockoffs <https://projecteuclid.org/euclid.aos/1438606853>`__
(**Delayed in schedule**)

.. raw:: html

   </ul>

.. raw:: html

   </td>

.. raw:: html

   </tr>

.. raw:: html

   <tr>

.. raw:: html

   <td>

7 (M: 5/18)

.. raw:: html

   </td>

.. raw:: html

   <td>

Knockoffs for multiple comparisons

.. raw:: html

   </td>

.. raw:: html

   <td>

.. raw:: html

   <ul>

.. raw:: html

   <li>

Lei and Fithian `AdaPT: An interactive procedure for multiple testing
with side information <https://arxiv.org/abs/1609.06035>`__ (**Delayed
in schedule**)

.. raw:: html

   <li>

Li and Foygel Barber `Accumulation tests for FDR control in ordered
hypothesis testing <https://arxiv.org/abs/1505.07352>`__ (**Delayed in
schedule**)

.. raw:: html

   </ul>

.. raw:: html

   </td>

.. raw:: html

   </tr>

.. raw:: html

   <tr>

.. raw:: html

   <td>

8

.. raw:: html

   </td>

.. raw:: html

   <td>

Memorial Day

.. raw:: html

   </td>

.. raw:: html

   <td>

Conditional inference IV (CLT)

.. raw:: html

   <ul>

.. raw:: html

   <li>

`Asymptotics <notebooks/Asymptotics.html>`__
(`notebook <https://nbviewer.jupyter.org/format/slides/url/web.stanford.edu/class/stats364/notebooks/Asymptotics.ipynb>`__)

.. raw:: html

   </ul>

.. raw:: html

   </td>

.. raw:: html

   </tr>

.. raw:: html

   <tr>

.. raw:: html

   <td>

9 (M: 6/1)

.. raw:: html

   </td>

.. raw:: html

   <td>

.. raw:: html

   <ul>

.. raw:: html

   <li>

`Approximate likelihood <notebooks/Likelihood.html>`__
(`notebook <https://nbviewer.jupyter.org/format/slides/url/web.stanford.edu/class/stats364/notebooks/Likelihood.ipynb>`__)

.. raw:: html

   </ul>

.. raw:: html

   </td>

.. raw:: html

   <td>

.. raw:: html

   <ul>

.. raw:: html

   <li>

`Black box <notebooks/Black-box.html>`__
(`notebook <https://nbviewer.jupyter.org/format/slides/url/web.stanford.edu/class/stats364/notebooks/Black-box.ipynb>`__)

.. raw:: html

   <li>

Weinstein and Ramdas `Online Control of the False Coverage Rate and
False Sign Rate <https://arxiv.org/abs/1905.01059>`__ (**Delayed in
schedule**)

.. raw:: html

   </ul>

.. raw:: html

   </td>

.. raw:: html

   </tr>

.. raw:: html

   <tr>

.. raw:: html

   <td>

10

.. raw:: html

   </td>

.. raw:: html

   <td>

`Review <notebooks/Big-picture.html>`__
(`notebook <https://nbviewer.jupyter.org/format/slides/url/web.stanford.edu/class/stats364/notebooks/Big-picture.ipynb>`__)

.. raw:: html

   </td>

.. raw:: html

   <td>

Projects

.. raw:: html

   </td>

.. raw:: html

   </tr>

.. raw:: html

   </table>

Evaluation
----------

-  3-4 shortish assignments (50%)
-  project (50%)

Notes
-----

-  Introduction `HTML <notebooks/Course-introduction.html>`__,
   `notebook <https://nbviewer.jupyter.org/format/slides/url/web.stanford.edu/class/stats364/notebooks/Course-introduction.ipynb>`__

[HTML]

.. raw:: html

   <!--

   - Review: [HTML](notebooks/Review.html), [notebook](https://nbviewer.jupyter.org/format/slides/url/web.stanford.edu/class/stats364/notebooks/Review.ipynb)

   -->

Assignments
-----------

.. raw:: html

   <!--

   * Assignment 1 [HTML](notebooks/Assignment1.html), [notebook](https://nbviewer.jupyter.org/url/web.stanford.edu/class/stats305b/notebooks/Assignment1.ipynb)

   * Assignment 2 [HTML](notebooks/Assignment2.html), [notebook](https://nbviewer.jupyter.org/url/web.stanford.edu/class/stats305b/notebooks/Assignment2.ipynb)


   * Assignment 3 [HTML](notebooks/Assignment3.html), [notebook](https://nbviewer.jupyter.org/url/web.stanford.edu/class/stats305b/notebooks/Assignment3.ipynb)

   * Assignment 4 [HTML](notebooks/Assignment4.html), [notebook](https://nbviewer.jupyter.org/url/web.stanford.edu/class/stats305b/notebooks/Assignment4.ipynb)


   -->