Lecture | Date | Title | Details | Related Reading | Presenter |
1 | 09/25/2017 | Introduction
[Silvio's slides]
[Amir's slides]
| - Motivation
- Overview of the topics and lectures
- What is a representation (X2VEC)? Why does it matter?
- Logistics and policies
| | Silvio Savarese and Amir Zamir |
2 | 10/02/2017 |
Representation 101
Background (pre 2012)
Fully supervised representations I
Fully supervised representations II
[slides]
| - Basic desirable properties of representations
- Ill-posedness, nonlinearity, complexity, dimensionality, vertical vs horizontal domain, etc.
- Handcrafted representations
- 2D Matching features (e.g. SIFT, HOG, DAISY, Kernels)
- Temporal (video) Features (e.g., 3DSIFT, Dense Trajectory Features - DTF, STIP, ICCV15 Storyline)
- Fully supervised representations:
- 2D matching features (e.g. matchnet)
- 2D object detection features (e.g. Imagenet)
- Brief executive summary of representation understanding methods
| |
Amir Zamir |
3 | 10/9/2017 |
Recurrence/Feedback based representations
Structured representations
Losses and GAN's
[slides]
|
- Curriculum learning
- Feedback
- structure
- CRF+CNN
- Structural-RNN
- Quaternions
- Exponential maps
- Emergence of attributes and parts in Object-based representations
- Emergence of parts in Scene representations
- Minimal Image.
- Empirical Receptive Field
|
|
Amir Zamir |
4 | 10/16/2017 |
Losses
Student Paper Presentation
[slides]
|
- Lnorm
- Metric Learning
- Perceptual loss
- Intro to GAN
- Pitfalls (e.g. mode collapse and training). Remedies.
- Wasserstein GAN. Energy based GAN.
- Conditional GAN
- Generative Latent Optimization
- Cycle consistency and transport loss.
- GAN Applications
- Metric Learning
|
|
Amir Zamir Students |
| 10/18/2017 | Project Proposal Due (11:59 PM) | | | |
5 | 10/23/2017 |
2D and 3D Object and scene representations
[Silvio's slides]
Student Paper Presentations
[slides]
|
- Basic principles for designing a good representation for object recognition
- Why are 3D representations useful for object understanding?
- Overview of classic and more recent methods for 2D and 3D object detection and classification
- 2D & 3D Scene Understanding
- Why is a 3D representation useful for scene understanding?
- Relating objects and space in the 3D physical world
- From objects to activity understanding in the 3D physical world
- Datasets
- Visualizing Representations:
- Representation inversion methods. (E.g. Fergus’s 16)
- Disentangling representations (Reed’14)
- Elements probing methods. (Bengio 17)
- interpretability methods.
|
Inversion:
Neuron domain:
Image Domain:
CS231N slides
|
Silvio Savarese Students |
6 | 10/30/2017 |
Domain Adaptation
Student Paper Presentations
[Slides]
|
Classic domain adaptation
Domain confusion loss
End-to-end learned domain adaptation
Nonparametric domain adaptation
GAN based domain adaptation
|
Domain Adaptation by learning joint or invariant features
Domain adaptation by aligning domains
Also a good paper
|
Ozan Sener Students |
7 | 11/06/2017 |
Broad perception representations
Self supervised learning
Student Paper Presentations
[Slides]
|
Self supervision
Supersized self-supervision.
Broad perception
Multitask. Ubernet.
Curiosity
(unpublic material)
|
Learning to learn
Memory Mechanisms and Lifelong Learning
Self-supervised learning:
|
Amir Zamir Students |
8 | 11/13/2017 |
Representations in the brain
Memory-based representation and inference
Student Paper Presentation
[Slides]
|
Details TBD
|
Hebb’s postulate revisited
Meta-discussion and things inspired by the brain
Using Models to Model the Brain
Representations in the brain
|
Daniel Yamins Students |
| 11/17/2017 | Project Progress Report Due (11:59 PM) | | | |
| 11/20/2017 |
No Class. (Thanksgiving)
|
| |
|
9 | 11/27/2017 |
From representation to actuation: active perception
Student Paper Presentation
|
Active vision.
Reinforcement learning
Action Prediction based frameworks.
Curiosity
Prediction and probability based reps.
Intuitive physics
Video frame prediction
|
Learning by Being Active
Learning by Being Active
|
Amir Zamir Students |
10 | 12/4/2017 |
Unsupervised learning of representations
Student Paper Presentation
|
Clustering
Dimensionality reduction
Sparse coding
Generative unsupervised learning
Discriminative unsupervised learning
|
Generative Models
Discriminative
|
Amir Zamir Students |
11 | 12/8/2017 |
Final Presentations
|
| |
Students |
12 | 12/11/2017 |
Final Presentations
|
| |
Students |
| 12/14/2016 | Final Project Report Due (11:59 PM) | | |
|