$\DeclareMathOperator{\p}{Pr}$ $\DeclareMathOperator{\P}{Pr}$ $\DeclareMathOperator{\c}{^C}$ $\DeclareMathOperator{\or}{ or}$ $\DeclareMathOperator{\and}{ and}$ $\DeclareMathOperator{\var}{Var}$ $\DeclareMathOperator{\E}{E}$ $\DeclareMathOperator{\std}{Std}$ $\DeclareMathOperator{\Ber}{Bern}$ $\DeclareMathOperator{\Bin}{Bin}$ $\DeclareMathOperator{\Poi}{Poi}$ $\DeclareMathOperator{\Uni}{Uni}$ $\DeclareMathOperator{\Exp}{Exp}$ $\DeclareMathOperator{\N}{N}$ $\DeclareMathOperator{\R}{\mathbb{R}}$ $\newcommand{\d}{\, d}$

Schedule

  • All times listed are Pacific Time.
  • Lecture Notes for future lectures are drafts and may be updated as the course progresses.
  • Optional readings are from Sheldon Ross, A First Course in Probability (10th Ed.), Prentice Hall, 2018. The textbook's 8th and 9th editions have the same readings and corresponding section headers. This quarter we are also writing a Course Reader for CS109 which is free (but will be constructed as we go)!
Date Lecture contents Assignments
Week 1
Jan 11
Mon
1 Counting
  • Welcome/Logistics
  • Sum Rule, Product Rule
  • Inclusion/Exclusion Principle
  • General Rule of Counting
Slides
Syllabus
Read: Counting, or Ross Ch 1.1-1.2.
Jan 13
Wed
2 Permutations and Combinations
  • Permutations
  • Combinations
  • Buckets and dividers

Read: Combinatorics or Ross Ch 1.3-1.6
Out: PSet #1
Jan 15
Fri
3 Axioms of Probability
  • Frequentist definition of probability
  • Axioms and corollaries of Probability
  • Probability of equally likely outcomes

Read: Probability or Ross Ch 2.1-2.5, 2.7
Week 2
Jan 18
Mon
Holiday: Martin Luther King Jr Day
No Class
Jan 20
Wed
4 Conditional Probability and Bayes
  • Conditional probability
  • Chain Rule
  • Law of Total Probability
  • Bayes' Theorem
  • Monty Hall Problem

Read: Ross Ch 3.1-3.3
Jan 22
Fri
5 Independence
  • Generalized Chain Rule
  • Independence
  • Independent Trials
  • deMorgan's Laws

Read: Ross Ch 3.4-3.5
Due: Pset #1
Out: PSet #2
Week 3
Jan 25
Mon
6 Random Variables and Expectation
  • Conditional independence
  • Random variables
  • Probability Mass Function (PMF)
  • Cumulative Distribution Function (CDF)
  • Expectation

Read: Ross Ch 4.1-4.4
Jan 27
Wed
7 Variance, Bernoulli, Binomial
  • Variance
  • Properties of variance
  • Bernoulli RV
  • Binomial RV

Read: Ross Ch 4.5-4.6
Jan 29
Fri
8 Poisson and More
  • Poisson RV
  • Geometric RV and Negative Binomial
  • Poisson approximation to the Binomial
  • Modeling exercise: Hurricanes

Read: Ross Ch 4.7-4.10
Week 4
Feb 1
Mon
9 Continuous Random Variables
  • Probability Density Function
  • Uniform RV
  • Exponential RV
  • Properties of the CDF

Read: Ross Ch 5.1-5.3, 5.5
Due: Pset #2
Out: Pset #3
Feb 3
Wed
10 The Normal Distribution
  • Normal (Gaussian) RV
  • Normal Symmetry, Linear Transforms
  • Standard Normal RV
  • Sampling to compute probabilities involving two Normal RVs

Read: Ross Ch 5.4
Quiz #1
Feb 5
Fri
11 Joint Distributions
  • Normal approximation to the Binomial
  • Discrete joint random variables
  • Multinomial RV

Read: Ross Ch 6.1
Week 5
Feb 8
Mon
12 Multinomial
  • Multinomial
  • Hamilton

Read: Ross Ch 6.2-6.3
Feb 10
Wed
13 Joint RV Statistics
  • Independent discrete RVs
  • Coupon Collecting Problems
  • Covariance
  • Variance for Independent RVs
  • Correlation

Read: Ross Ch 6.4-6.5
Feb 12
Fri
14 Conditional Random Variables
  • Conditional distributions
  • Law of Total Expectation
  • Analyzing Recursive Code

Read: Ch 7.3-7.4
Due: Pset #3
Out: Pset #4
Week 6
Feb 15
Mon
Holiday: Presidents' Day
No Class
Feb 17
Wed
15 Continuous Joint Distributions
  • Continuous joint distributions
  • Joint CDFs
  • Bivariate (Multivariate) Gaussian RVs

Read: Ch 6.1
Feb 19
Fri
16 Continuous Inference
  • Inference with continuous joint distributions

Lecture Notes
Week 7
Feb 22
Mon
17 General Inference
  • Bayesian Networks
  • Inference from first principles
  • Inference with Rejection Sampling

Read: Ch 8.3
Due: Pset #4
Out: Pset #5
Feb 24
Wed
18 Central Limit Theorem
  • i.i.d. Random Variables: Independent and Identically Distributed
  • Central Limit Theorem

Read: Ch 8.3
Quiz #2
Feb 26
Fri
19 Sampling/Bootstrapping
  • Population mean/variance, Sampling mean/variance
  • Unbiased estimators
  • Standard Error
  • Bootstrap for Standard Error
  • Bootstrap for Hypothesis Testing

Read: Lecture Notes
Week 8
Mar 1
Mon
20 Beta
  • MLE: Multinomial
  • Bayesian Definition of Probability
  • Beta Random Variable
  • Beta and flipping a coin with unknown probability

Read: Ch 5.6.1-5.6.4, 7.5-7.6
Mar 3
Wed
21 Parameters and MLE
  • Intro to Parameter Estimation
  • Maximum Likelihood Estimator
  • Argmax and log-likelihood
  • MLE: Bernoulli
  • MLE: Poisson, Uniform, and Gaussian

Read: Lecture Notes
Mar 5
Fri
22 Maximum a Posteriori
  • Maximum a Posterior Estimator
  • Bernoulli MAP: Choosing a prior
  • Conjugate distributions for common RVs
  • Laplace smoothing
  • Bayesian Envelope demo

Read: Lecture Notes
Due: Pset #5
Out: Pset #6
Week 9
Mar 8
Mon
23 Naive Bayes
  • Strawman: 'Brute Force Bayes'
  • Naive Bayes Classifier
  • Naive Bayes example: MLE and MAP
  • Naive Bayes: MAP with email classification

Read: Lecture Notes
Mar 10
Wed
24 Gradient Descent and Logistic Regression
  • Logistic Regression
  • Maximizing Likelihood, Minimizing Loss
  • Gradient Descent

Read: Lecture Notes
Mar 12
Fri
25 Deep Learning
  • Optional content: Intro to neural networks
  • No concept check

Week 10
Mar 15
Mon
26 Ethics in Machine Learning

Read: Lecture Notes
Mar 17
Wed
27 Beyond CS109: The Future of Probability

Quiz #3
Due: Pset #6
Mar 19
Fri
28 Beyond CS109: Next Steps