EE367/CS448I: Computational Imaging and Display

2017 Winter Course Project: object removal using sparse coding

This file describes matlab files that we used to obtain our results,
together with instructions for how can they be reproduced.


I. Object Removal Source File List
----------------------------------

single_image_sr.m:

The program that down-samples the source image, calls the exemplar inpainting program, and performs super resolution afterwards.

Call the program: single_image_sr. Before that you should specify in the code the path and filenames of the source image (RGB channel image) and the mask image (grayscale channel) which indicates the region that needs to be inpainted.

project_inpaint_nlm.m (you do not need to call this program, because it is called within the program above):

Examplar inpainting program using our NLM method, without downsampling and super-resolution. This function calls a precompiled C program bestexemplarhelper.mexmaci64 for sake of efficiency.

Call the program: [inpaintedImg, confidence_term, similarity_term] = project_inpaint_nlm(srcFile_with_3_channels, maskFile_with_3_channels, maskColor)

For example:
[i1,c,d] = project_inpaint_nlm('balloon_d.jpg','balloon_mask_3d_d.jpg',[255,255,255]);
Here balloon_d.jpg and balloon_mask_3d_d.jpg are RGB image files. [255,255,255] indicates that in the region to be unpainted are marked with white pixels in balloon_mask_3d_d.jpg.

Note that the execution time for project_inpaint_nlm.m is about three hours for a 1600x1600 image.

For faster result, run single_image_sr function, this include the entire process pipeline.

Images:

balloon.jpg, parachute.jpg — source RGB images
balloon_mask_1d.jpg, parachute_mask_1d.jpg - grayscale masks
balloon_mask_1d.jpg, parachute_mask_1d.jpg - RGB masks


II. Sparse Coding Source File List
----------------------------------

- dictVisualization.m  		visualize learned or pre-defined dictionary by patches

- getDictionary.m  		get pre-defined or ksvd learned dictionary based on input argument

- inPainting.m  		Perform the actual inpainting of the image

- ksvd.m  			Perform ksvd based on input training samples

- main.m 			main function that allows user to define degraded images, masks and dictionaries by themselves

- mergePatch.m 			merge vectorized patches back to original image

- OMP.m:		 	Our implementation of the Orthogonal Matching Pursuit 

- psnr.m:			Compute PSNR of the reconstructed image

- splitImg.m 			Split image into p patches and then vectorize each patch; form signal matrix using these column vectors

- trainDict.m 			Train a dictionary based on give dataset using ksvd

- ksvd_dict.mat			Our trained ksvd dictionary; you can train your own version using trainDict.m

- dctDict.mat 			Pre-defined DCT dictionary


To run the code you just need to modify parameters in main.m 

Note that for now we just support grayscale square images. And the dimension of the image has to be able to be divided by patch size (neib).

The reconstructed image is single channel a matrix I_rec in [0,1] range of the same size as I.



III. Authors
------------
Haihong Li, Joanna Xu, Chen Zhu
Department of Electrical Engineering
Stanford University



IV. Code Reference
--------------
OMP.m refered the implementatino of https://github.com/porvaznikmichal/ETH-Image-Inpainting
Original Exemplar-based inpaint: http://scarlet.stanford.edu/teach/index.php/Object_Removal

