# Colorblind Metric

This is the wiki page for colorblind metric project. In this project, we formulate and test an engineering color metric for dichromates and this metric can be directly used to characterize image qualify and visibility.

Most part of the project is based on ctToolBox and coded in Matlab.

Developed by Haomiao Jiang, Joyce Farrell and Brian Wandell

##  Colorblind Metric

The colorblind metric is designed as a combination of two parts: color projection and color difference calculation. For color projection, we follow the approach of Brettel et al. and project color from the 2D dichromates' color space (LM, LS, MS) to the 3D ordinary human vision space (LMS). One remarkable hypothesis for this projection is proposed in paper [1].

For color difference calculation, we use the notion of deltaE in SCIELAB, which is approximately the Euclidean distance in Lab color space (Latest version of deltaE is slightly different from this notion). To calculate the deltaE for certain type of dichromate between two colors, we first map the color to the ordinary 3D color space (LMS space), then transform them to XYZ, to Lab color space with help of measured white point (or PSF, etc.) and at last calculate the Euclidean distance. If the hypothesis in color projection part is correct, this colorblind deltaE metric should be visually uniform in the eye of dichromate of that kind.

##  Experiment Design

Experiment is under framework of staircase control. In each Trial, two pairs of color patches are presented to the subjects sequentially. Among them, colors in one pair are identical while colors in another are slightly apart by pre-designed deltaE and direction in the sense of certain colorblind. Subjects are asked to choose the identical pair and sound feedback is given after each trial.

%%We have not run the experiment yet.%%

Once we identify the colorblindness, we will use this method to collect more data and justify the color-metric for dichromates.

##  Auxiliary Tools

###  Color Match Test for Dichromates

User Interface for Color Match Test

This tool is used to roughly determine the type of colorblind. A lot of Ishihara Plate based colorblind tests are available online. However, due to the similar color performances between protanopia and deuteranopia, the tests can hardly tell them apart. Screen shot for the user interface is shown in figure on right.

This experiment with three sliders is reported to be hard by the subjects and we can using the contraints to reduce it to one slider. The constraints includes: 1) illuminance, Y should always be matched 2) Confusion line, any freedom along the confusion line is meaningless to that type of colorblindness.

The basic scheme for this tool is similar to the color match test, in which subjects are asked to adjust 3 sliders (R,G,B) to make the two color patch identical. The trick for this tool is inside the initial value design. Initial value for the user adjustable color patch is chosen from obviously different color, similar color, color along confusion lines for protanopia, deuteranopia, tritanopia. User responses are used to calculate the four deltaE scores (Normal, Protanopia, Deuteranopia, Tritanopia) and this forms the most important part in making decisions. Besides, response time and other factors are also recorded.

color match analysis - confusion lines

To run this tool, type

 s_colorMatch(displayProfileName)


Note that in this color match test, display calibration file is needed. If it is not provided, the program will by default use the SRGB color space and corresponding D65 white point, which can make the results inaccurate.

File format and calibration instructions can be found from VISTA Lab

After the test, the results will be automatically saved in ColorMatchResults.mat. Some variables in this .mat file can be passed to s_analyzeCbData.m or s_analyzeCaData.m for further analysis.

By assuming the subject is dichromate, we can analyze the data by calling interfaces as below

 s_analyzeCbColorMatchData(hG)

color match analysis - Lab space analysis

hG is the data structure used in the color match test, which is automatically generated and saved by s_colorMatch function.

The analysis includes three parts: colorblind type determination, confusion line analysis and colorblind Lab space analysis. In colorblind type determination part, we calculate modified deltaE score for normal, protanopia, deuteranopia and tritanopia. If one score is much smaller than the other three, the subject tends to be that specific type. In confusion line analysis, we plot user response data and confusion lines to the true color for one specific type of colorblind in xy space. One sample plot for deuteranopia is shown on right.

In the plots, the circle stands for the ground truth (target color) and the crosses are the subject responses. The dashed lines are the confusion line for the estimated type of colorblind (protan, deutan, tritan).

For colorblind Lab space analysis, we compute the reconstructed true color and user response for specific colorblind type. Then we transform them to Lab space and plot the relative position for each user inputs. One sample plot is shown on the right. In this plot, we can see how close the points are to the origin. This euclidean distance (or deltaE score for CIELab 1976) represents a sense of our color metric. Our metric now is based on deltaE score 2000.

color anomalous analysis

If the subject is color anomalous (not a dichromate), we call the following function to analyze the severity of the pigment mutation in the cone cell of the subject.

 s_analyzeCaColorMatchData(hG, type)


hG is the data structure from the color match test and type is a tri-state number (1, 2, 3) indicating the possible anomalous type, Protanomaly, Deuteranomaly and Tritanomaly correspondingly. If type is not set, we arbitrarily set it to the type with smallest average modified deltaE value.

This script tries to predict the responsivity curve of the mutated cone. The analysis is based on minimizing the L1 norm of the deltaE error array, that is

$S = argmin_{S} \sum_i deltaEab(LMS2XYZ*S*\sum_j Gamma(trueColor(i,j))*Spec(:,j),LMS2XYZ*S*\sum_j Gamma(matchColor(i,j))*Spec(:,j))$

This method is also an MAP estimation for the extent of pigment mutation. Some sample pictures for this kind of analysis is shown on the right. The three solid lines are the responsivity lines for normal people and the dash line is the predicted responsivity for the subject. The lower plot shows the total sum of deltaE under different cone responsivity shifting distance.

### Confusion Line and Color Difference Estimator

Color difference and Confusion Line for Protanopia

Conception of confusion lines has been well defined and the description and plots for all three types of dichromates confusion lines in CIE 1931 color space can be found here.

However, the concept of confusion line does not tell us any about the "line width". Also, it does not tell us anything about how visually different are other colors when compared to reference point. This color difference estimator tool can be used to answer this question.

The calling interface of this tool is

 deltaEMap = s_ctConfusionLines(vDisplay, cbType, refColorPoint)


This function will return the deltaE for all visible colors for given reference colorpoint and colorblind type. One sample plot for deltaEMap is shown on the right. The plot is for protanopia with reference point at [x,y] = [0.4 0.2].

Note that this program is based on a series of hypothesis and assumptions and it should not be used for accurate estimations. The hypothesis used can be validated after the fully development of colorblind metric and this part will be updated afterwards.

### Ishihara Plate Generator

Ishihara Plate is widely used in colorblind tests. However, most Ishihara Plates are randomly generated. This kind of Ishihara Plate can hardly be used to test the severity of color deficiencies (or even colorblind). Taking into account the display, display gamma and having more precise control on the difference (deltaE).

This tool is designed to generate more color-precise Ishihara Plate. The color primarily chosen along the confusion lines and the distance from reference point is controlled by passed in parameter, deltaE for normal people.

The calling interface of this tool is

 s_genIshiharaPlate(vDisp, pattern, deltaE,cbType)


This program has not been completed. Will update more details later.

### Colorblind Image Simulator

This tool is used to convert normal images to colorblind images. With this tool, we can easily know what the image is like in the eye of colorblind. One example image is shown on right. %% Insert Image %%

Protocol for this tool is

  cbImage = s_colorBlindImage(vDisplay, imageName,cbType)


Note that the tool requires a display description structure or white point. If these are not provided, the program will use SRGB by default.

Structure of virtual display structure can be found in ISET

%% Will be added soon %%

##  Work To Be Done

1. Finish Experiments and Analyze Results

2. Get codes well-documented and uploaded to GitHup

##  Reference

[1] Brettel et al, Computerized Simulation of Color Appearance for Dichromates