Logo Images with Light Background

Look at the image at the left, and you will notice that when there is supposed to be a dark module ($w_a$ is dark), its background gets no perceivable change.

img

This is due to this part of the code:


float b = 0.25, a = 0.5, bc = 0.15, ac = 0.4;

// for loops here ...

if (M.at<uchar>(i, j) == 127 && qr.at<uchar>(i, j) == black && binBN.at<uchar>(i, j) == black){
    Light::accumulate_brightness_at(i, j, int(hsvfChannel[2].at<uchar>(i, j) * b), hsvfChannel[2]);
} else if (M.at<uchar>(i, j) == 127 && qr.at<uchar>(i, j) == white && binBN.at<uchar>(i, j) == black){
    Light::accumulate_brightness_at(i, j, int(hsvfChannel[2].at<uchar>(i, j) * a), hsvfChannel[2]);
}

where the variable b = 0.25 is crearly smaller than a = 0.5. These two fators are the luminance ratio that is accumulated on the luminance value at pixel i,j. In images with natural backgrounds, like the one on the left below, what happens is that, since the color in the pixels is never at 255, the dark modules accumulate less color than the white modules, giving the impression that they’re darker. However, as shown in the first image, when the background tends to be white, the accumulation is never taking any change since dark and white modules both reached already 255.

The images on the right show the implications of modifying b to be b = -0.25. The accumulating is now for a darker color since it is now negative. It produces now a useful QR for the white background images, but for the natural background images, the QR is of course useful still, but it gets more noticeable.

What can be done is that maybe use the Utils::is_logo_image() function again to only modify the $b$ variable to a negative value if the image is a logo and if the background tends to be white. In the case of natural images, $b$ can stay positive as long as the background is not primarily white. Maybe, then, only convert $b$ to negative whenever the background is mainly white?

img

What was done was precisely to map first in a kind of Look Up Table all the data modules that have an L channel mean > 245 and $\sigma$ < 10 and set all the pixels of those modules as a 1 (255) while all the rest are kept as 0. The resulting look up table for the SAP.io white logo is the following:

img

while the other SAP.io logo table is the following:

img

Now, in the final routine of the code where all the modification of luminance occurs, the code was changed to use:

if (avgModuleColorMat.at<uchar>(yIdx, xIdx) == 255){
    Light::accumulate_brightness_at(i, j, int(hsvfChannel[2].at<uchar>(i, j) * -b), hsvfChannel[2]);
} else {
    Light::accumulate_brightness_at(i, j, int(hsvfChannel[2].at<uchar>(i, j) * b), hsvfChannel[2]);
}

which chooses to use b as a negative value or as a positive one depending on the value of the light modules’ look up table. This in fact does not involve any advanced computer vision such as the BLOB detection I was thinking at first. It was ditched because there might be light modules that are not part of prominent BLOBs throughout the whole image, i. e., in a checkered image, but still the data module non-central pixel darkening would still be crucial. Thus, all the pixels that are part of a light module ($\mu$ > 245 and $\sigma$ < 10>) are darkened with a negative $b$ value.

The result can be compared visually below.

img

Primary Colors Dark and Bright Modules

I’ve got to check the value of noncentral pixels because in the dominos example below, the red area’s black modules are not darkening and the bright ones are not increasing. Maybe it is due to the white modules’ fix and the red is too pure (255, 0, 0) that when accumulated, nothing happens even though is not white. I’m testing with a fully blue image (0, 0, 255) because in the hsv value may not be fully light so that the b is negative, and then it cannot add more color because in RGB there’s a 255 already. Gonna check.

img

Update: the plot shows that for a red image (255, 0, 0) the v channel is fully 255 values. Thus, it should take negative b. Gonna check another possibility for that bug. Note: the first two images are not inside the viridis cmap because viridis is done for single-channel images.

img

This might be due to the HSV color space, where all primary colors have value channel of 100 in a scale of 0 to 100.

img

Update: by changing the criteria to define alight module, so that instead of having $\mu > 245$, we have $\mu > \mu - (\mu \times \delta)$,

if (mean[0] > (255 - (255 * FACTOR_AVG_MODIF)) && stdev[0] < 10){
    avgModuleColorMat.at<uchar>(yIdx, xIdx) = 255;
}

we can achieve the primary color modules to also change. This is increasing the robustness of detection since it is more sensible to light modules which makes the $b$ take negative values and in consequence increases the contrast with the dark modules’ non-central pixels. The outcome is:

img