# Codebase for EE 368 Project: Learning to Detect Light Field Features
#### Kelly Guan, Farah Memon, Lars Jebe
#### March 16, 2018

##############################

## This Codebase contains the following folder and files: 

##############################

## Folder: 000_Training_Data_Acquisiton
### File: SIFT_Feature_Extraction.m
This MATLAB Script goes through the dataset and extracts SIFT features for each rendered image. 
It also creates samples of bad features, and saves each list of good and bad features in a .csv-file, together with their descriptors. 

### File: harrisdetect.m
This MATLAB Script extracts Harris keypoints for each rendered image in the dataset, and saves the list of features for each
image as a .csv-file. For each detected keypoints, SIFT descriptors are extracted at a fixed scale as well. 

### File: Final_Dataset_Split.ipynb
This notebook was used to split the scenes from the dataset into train and test scenes. 

##############################

## Folder: 001_Autoencoder
This folder contains two versions of the autoencoder described in the report. The second version has been used for the report
results. 
### File: Comparison2D3D_v2.ipynb
This notebook performs evaluation of the autoencoder. It evaluates previously trained and saved models that are not included in this codebase. 

### File: train_2Daenc_v2.py and train_3Daenc_v2.py
These python scripts train and save the autoencoder models / weights. Saved models can be used for the evaluation in Comparison2D3D_v2.ipynb. 

##############################

## Folder: 002_2D_Render_Model
### File: lf_util.py
basic utilities for reading, writing and manipulating light field images. 

### File: data_parser.py
This pyhton script builds the core of the input pipeline. It is used to generate input patches from the dataset and the previously created csv-files containing information about good and bad keypoints.

### File: simple_classification.py
This model can be used for training and classification. We refer to this model as "2D Classification" in the report. 

### File: simple_detection.py
This model is a generalization of simple_classification.py, allowing for training on patches and predictions of complete 2D rendered images of arbitrary size. 

### File: simple_detection_v2.py
This is a second version of the model. We removed a lot of pooling and strides to het a higher-resolution heatmap. 

##############################

## Folder: 003_2D_LF_Model
### Files: analogous to the files in 002_2D_Render_Model. Instead of rendered images, this input pipeline and model architecture uses 2D light field slices. 

##############################

## Folder: 004_3D_LF_Model
### Files: analogous to the files in 003_2D_LF_Model. Instead of 2D light field slices, this input pipeline and model architecture uses 3D light field slices. 

##############################

## Folder: 005_Altwaijry_Model
### File: altwaijry2D.py
This model is an implementation of the multi-branch model as proposed in [Altwaijry2016Learning]. It is a scale-invariant version of the previous models and capable of training on input patches of different sizes. While the current implementation be used to train the model, it is not capable of creating a scale-space heatmap output yet, as this requires the svaing of internal model states (under development as of March 16, 2018)

##############################

## Folder: 006_Heatmap_Evaluation
### File: heatmap.m
This MATLAB scrpit provides utilities to evaluate heatmaps and compare them to SIFT features. Evaluation includes calculation of precision/recall and visualization of the keypoints. 