Misprint software

From VISTA LAB WIKI

Jump to: navigation, search

This page describes how to use the software for the HP project on misprint detection.

Contents

[edit] Some initial remarks

[edit] Code

The whole code is stored in the svn repository, you can download ir with:

svn --username john_doe co https://white.stanford.edu/ee/pdcprojects/misprintDetection/

[edit] Naming Conventions

The naming convention is such that:

  1. All function start wil hpl_
  2. The underscore notation is used, i.e.: an _ separates the different words in most function (though there are one or two counter-examples)
  3. Functions containing _n_sensors are those that work with several sensors.
  4. Functions containing _3misprintsModel are those that work with the second statistical model.
  5. Functions containing _2_0 are those working with the complete system (2 sensors and the second model).
    1. They are designed to be simpler to use.
    2. They are often faster.
    3. You should try to use these functions every time you can.
    4. They may be difficult to modify, indeed they follow the following rules:
      1. All arguments are always necessary
      2. There is therefore no hidden default parameters anywhere
      3. Constants or the like are not set inside the functions
      4. A separate file hpl_default_parameters, sets up all these parameters, as they are very inter-dependant and the functions are expecting these parameters, it is not recommended to change them
      5. The noise can only be in [0.2 0.4 0.6 0.8 1]
      6. The window size can only be kxk with k in [5 10 20 30]
  6. CDP tables are stored in cdpTables/
  7. Results for the full system are stored in results/

[edit] Time considerations

Most of the functions in the code have been speeded up a lot during the development.

The part using ISET is still long, and particularly computing the sensor parameters, this may take several days, but you only have to do it every time you add a new sensor (so it should be pretty rare).

With the 2_0 functions, learning and testing the tables takes a few hours, it is long as there are so many images. To predict the time needed, you can count 2.65 seconds per image, less if you have a fast computer, more it is not that good.

[edit] Convert images in cmyk files

You will need to have imagemagick installed on your computer.

You can then run the

convert inputFile.jpg -colorspace 'CMYK' outputFile.tiff

There is a shell script in the main directory - - conversion.sh - that illustrates how to convert multiple files. You will need to adapt it to make the script work with your database.

It is important to convert the input file data from RGB/jpg to CMYK/tiff with ImageMagick. If you get very dark images, this might be the problem.

[edit] Loading a tiff file

Now you can load the image into Matlab using the function hpl_load_my_image

You have to use this function to load this image to automatically handle special color conversion issues, if you manually load the image, your image will probably look very dark.

image = hpl_load_my_image('Myimage.tiff');

[edit] Visualizing the CMYK Image with MATLAB

You can not visualize image directly with matlab as the colors are stored in the 4D space of CMYK. To visualize the image, use hpl_display_cmyk:

hpl_display_cmyk(image);

Comparison between the original image, and the CMYK image visualized with hpl_display_cmyk:

Original Image Image After
Original image Image after conversion to a tiff file, read by matlab and displayed

[edit] Simulation with ISET

To simulate the line sensor with ISET, you have to use the function hplPrintLineSensor.

output = hplPrintLineSensor(image);
imagesc(output);

You might want to use a small image as this may take a long time for big images.

[edit] Computing sensor properties

If you want to recompute the look-up tables for the sensor, you have to use the function hpl_parameters_calculation. This takes a long time as it needs to run ISET a lot of times.

[out_mean out_std] = hpl_parameters_calculation();

Now you can replace models/out_mean.mat and models/out_var.mat with the new values.

Note that you should not do that unless you made some changes in the ISET model of the sensor.

[edit] Important Remark

For all the following functionalities, it is highly preferable to use the 2_0 functions; unless you have a good reason not to do so. Using the original functions is more complicated, but it gives you access to other modes of the algorithm. Before using the 2_0 functions, you should always check the hpl_default_parameters script to make sure that these parameters are those you want to run the algorithm with.

[edit] Simulation without ISET

To simulate the sensor output without ISET, you have to use the function hpl_simulate_pipeline

To use this function, you need to have previously stored or computed the sensor properties, the default version of these tables (and the best we have) is on the svn repository. So you should automatically have the tables.

result = hpl_simulate_pipeline(image);

result is not an image, it contains 3 fields:

  • result.output_average is the average image.
  • result.output_noisy is the average image, plus a random deviation generated according to the noise properties.
  • result.map_of_stds is the map of the theoretical standard deviations on each pixel of the sensor output. This is a sub product of computing the noisy image, and is useful for computing proba_map

To visualize the results you can do:

imagesc(result.output_average);
figure;
imagesc(result.output_noisy);

This computation is much faster than with ISET, it should take between a few seconds depending on the size of the image.

If you want to use this function inside a script or another function, it is better to use hpl_simulate_pipeline_2_0(CMYKimage, table_sorted, pipe_property, reducing_noise, out_w) you can get the different parameters with hpl_default_parameters:

hpl_default_parameters;
property = pipe_property_array{i};
hpl_simulate_pipeline_2_0(image, table_sorted, pipe_property, reducing_noise, out_w);

and therefore, the only arguments that you need to enter are the image, and i (the sensor number: 1 or 2) as the other arguments would have been created by hpl_default_parameters

[edit] Generating the proba_map (from which we extract the features)

To generate the proba_map, you have to call the function hpl_probability_of_misprint. On a regular simulation, you don't have to do that, as this function is automatically called when needed. But it can be interesting to find new features for the algorithm.

proba_map = hpl_probability_of_print(res.output_average, res.output_noisy, res.map_of_stds);

If you are using this function inside a script or another function, it is better to use hpl_probability_of_print_2_0,

[edit] Learning the CDP tables

To learn the CDP tables, you need to have a image database, and it should be converted to the tiff format and put into the directory: "input/converted". You also need some time, as with an image database of 1000 images, it takes around 2 hours to train the system.

You shouldn't do that unless you want to have result on a larger database, or you changed the algorithm.

[edit] Old way

This method is valid with hpl_compute_cdp, it computes the cdp table with the first statistical model, and only one sensor

cdp_table = hpl_compute_cdp();

cdp_table has 3 fields (one for each feature):

  • cdp_table.mean
  • cdp_table.frac
  • cdp_table.min

Once this is done, you can replace models/cdp_table.mat with the new table.

You can visualize the tables (as in the results) with

plot(cdp_table.mean(i,:));
  1. i = 1 for P(feature | correct print);
  2. i = 2 for P(feature | misprint);

[edit] New way

This method makes it possible to compute the cdp tables with the second statistical model and other sensors: you can use 2 functions: hpl_compute_cdp_3misprintsModel or hpl_compute_cdp_2_0. I'm only going to explain the usage of the latter as it the more convenient to use. But if you want to have more options, you can use the other one.

You have to choose the sensor (here 'ccd'), the noise (here 80%), and the window size (here 20x20)

cdp_table = hpl_compute_cdp_2_0('ccd', 0.8, 20);

then, if you have changed something in the misprint generation, you should copy global_vs_local.mat in models/global_vs_local.mat

cdp_table has also 3 fields (one for each feature):

  • cdp_table.mean
  • cdp_table.frac
  • cdp_table.min

You can still visualize the tables with:

plot(cdp_table.mean(i,:));

but now:

  1. i = 1 for P(feature | correct print);
  2. i = 2 for P(feature | global misprint);
  3. i = 3 for P(feature | local misprint);

[edit] Testing the tables

To learn the CPD tables, you need to have a image database, and it should be converted to the tiff format and put into the directory: "input/test_set". You also need some time, as with an image database of 330 images, it takes a bit less than 1 hour to test all the files.

As this part may be quite long, it automatically saves the results in results.mat

[edit] Old way

As previously, only works with one sensor, and the first model

results = hpl_test_statistics(cdp_table)

[edit] New way

There are several possible flavors of hpl_test_statistics, the most practical being hpl_test_statistics_2_0. You have to choose the noise (here 60%), and the window size (here 10x10)

res = hpl_test_statistics_2_0(0.6,10);

[edit] Visualizing the results

You can plot the ROC curve and the precision-recall curve with:

hpl_roc_curve(results.results);

You can also visualize the different errors with:

results.errors{i}.groundtruth
results.errors{i}.image

groundtruth tells you if there were a misprint or not and image gives you the actual CMYK image

We consider that there is an error with the threshold at 0.5

[edit] Testing on one single image

We are now reusing the image from before. First you have to decide whether you want to misprint it or not.

If misprint:

image_sensed = hpl_generate_misprint(image);

If not:

image_sensed = image;

Now we can compute the probability for image_sensed to be a misprint of image:

[edit] Old Way

Only one sensor, first model

p = hpl_test_one_image(image,image_sensed);

[edit] New way

Two sensors, second model

p = hpl_test_one_image_2_0(image,image_sensed,noise,window_size);

[edit] Demonstration Script

If you don't want to have to manually call all these functions, there is a demonstration script that will lead through these different steps, asking questions so you can choose what you do:

hpl_demo_script;

[edit] Adding a new sensor

It is not so simple to add a new sensor, to do so, you should carefully follow these steps, and probably having an idea of how the code is working:

  1. Load into MATLAB the QE curve if the sensor, save it in models/sensors - suppose we call it myQE.mat -
  2. Decide a name for this new sensor (e.g.: 'myNewSensor')
  3. In hplSensor.m tell the program where to find myQE.mat, you should add something like:
    1. elseif (isequal(qetype,'myNewSensor'))
    2. data = load(fullfile(hplRootPath,'models','sensor','myQE.mat'));
    3. data = data.myQE;
  4. On a computer with ISET compute the sensor properties: (this can take a long time)
    1. [out_avera out_noise_std] = hpl_parameters_calculation([],'myNewSensor')
    2. save out_avera and out_noise_std in models
  5. in hpl_default_parameters add similar lines after setting up pipe_property_array
  6. add 1 to n_sensors in hpl_default_parameters
  7. add 'myNewSensor' to qelist in hpl_default_parameters
  8. Compute new_cdp_table = hpl_compute_cdp_2_0('myNewSensor',noise,window_size)
    1. save fullfile(hplRootPath,'models','cdpTables/mycdptable.mat') cdp_table
  9. You'll have to modify hpl_test_statistics_2_0 and hpl_test_one_image_2_0 in such a way that they can find this cdp table for the given noise and window's size (see hpl_test_one_image_2_0, line 33 to 49 to know what is going one)
    1. Add the cdp table in cdp_table_array.

This should be working now with the 2_0 functions, if you want to use you new sensor with the other function, it means that you know the code enough to be able to add it by yourself then, this is much more messy that for the 2_0 functions

[edit] Other functions

[edit] Look-up table toolbox

There are 2 functions in the look_up_table_toolbox directory

hpl_construct_table_of_indices(table,new_table_size,default_value)
and
hpl_test_balanced_table(table,new_table_size,default_value,totrounds)

The second function calls the first one, so for the meaning of each argument, see hpl_construct_table_of_indices comments.

hpl_construct_table_of_indices builds an organized table from a unorganized table in a 4D space (e.g.: CMYK). The search in this new table is much faster as you can use a divide-and-conquer algorithm, and if the table is nicely "balanced", then you'd have the same results than with most other mesures (see: Look-up tables part)

hpl_test_balanced_table computes an histogram from the table to show how "balanced" it is: ideally it should follow the green envelop. In practice, as long as you don't have a peak far of the middle, everything should be fine.

Table Histogram
Table histogram computed with hpl_test_balanced_table

To compute this histogram, we are following this algorithm:

  1. drawing n random variables, with an uniform probability on the space.
  2. assigning an index for each value through the table
  3. for each index, counting how many values are assigned
  4. making an histogram of these counts
  5. the green curve is the theoretical result for a perfect table.

[edit] Visualization of MATLAB data with VTK

To visual a 3D structure stored in MATLAB with VTK, there is the function:

function hpl_visualize_with_vtk(vertices,faces,color)

vertices must be a Nx3 matrix, faces must be an Mxp matrix with faces(i,:) = [a b c 0 0 0] meaning that the i-th face is a triangle with vertices(a,:) vertices(b,:) and vertices(c,:) as corner points.

color must be either a Mx3 array (RGB values) with 0<=color<=1 or a Mx1 array (scalar values mapped)

example (displays a house with random colours on its faces):

A = [0 0 0; 1 0 0; 1 1 0; 0 1 0; 0 0 1; 1 0 1; 1 1 1; 0 1 1;...
0.5 0 1.5; 0.5 1 1.5;0.5 0 0;0.65 -0.01 0;0.5 0 0.5;0.65 -0.010 0.5];
B = [1 2 3 4;5 6 7 8;1 2 6 5;3 4 8 7;1 4 8 5;2 3 7 6;5 6 9 0;7 8 10 0;5 9
10 8;6 9 10 7;11 12 14 13];
C = rand(11,3);
hpl_visualize_with_vtk(A,B,C);

You should get something close to:

</td> </td>
VTK Visualization of a 3D house
VTK Visualization of a 3D house

(The colors will probably be different as they are random)

[edit] 3D transformation of images from MATLAB with VTK

There is also a function tht can be used for 3D transformation of images, this calls some of the vtk binaries.

hpl_transform_with_vtk(CMYKimage,type3d)

Remark:

type3d has to be a string (like '1')

  • type3d = '0' => no change
  • type3d = '1' => bowl shape
  • type3d = '2' => sphere shape
  • type3d = '3' => cylinder
  • type3d = '4' => some random changes
  • type3d = '5' => folding

example:

 file_name = 'couleur-du-temps-02.1179922439.tiff';
fName = fullfile(hplRootPath(),'input',file_name);
image = hpl_load_my_image(fName);
img2 = hpl_transform_with_vtk(image,'2')

[edit] SVN repository organisation

The main directory misprint/ contains the following subdirectories:


general_functions/ ==> general functions for the project: mainly transforming an image to the sensor output

input/ ==> a set of .tiff files that can be used as CMYK inputs

line_code/ ==> files needed to process the image with the 1D sensor

look_up_table_toolbox ==> some functions to build or check look up tables

misprint_detector/ ==> files for the misprint detector

misprint_toolbox/ ==> files to generate misprints

models/ ==> sensor, ink ... models

multi_sensors/ ==> files for computing the multi_sensor model

old_stuff/ ==> ...

simulating_iset/ ==> files to study how we could simulate the sensor without ISET

s_scene/ ==> different s_scene files, mostly useless now?

tests_toolbox/ ==> files used previously to characterize the sensor output on patches, and analyse its properties

vtkbin ==> binaries from VTK that are called in some of other functions the C++ files and the CMakeLists.txt file are also there. There is also a vtkbin/vtkPython directory with the same vtk files, but written in Python.

vtk_toolbox ==> some functions interfacing VTK with MATLAB

wavelet_toolbox/ ==> some wavelet (Haar) decomposition and reconstruction files

[edit] Tricks for the code

  • In hpl_simulate_pipeline, line 61, the sensor length is manually set to 128
  • When adding a misprint, there are two things to which you should pay attention:
    1. You should add it the same way in hpl_generate_misprint and in hpl_generate_misprint_3misprintsModel, in this last file you should not forget to say if it is a local or global misprint
    2. You need to increase of 1 the variable n_misprints in hpl_default_parameters
Personal tools