Reconstruction of a RGB image after PCA is way darker than the original one

Hello, I'm currently struggling with PCA applied to RGB images.

I think that what I've done is correct, yet the reconstructed images are way darker than the original ones or even blueish.

I used DermaMNIST from MedMNIST and divided the dataset by 255.

Here's my code:

x_train_r = [];
x_train_g = [];
x_train_b = [];
for i = 1 : size(x_train, 1)
    % red
    sample = x_train(i,:,:,1);
    sample = squeeze(sample);
    sample = reshape(sample, [size(sample,1)*size(sample,2),1]);
    x_train_r(:,i) = sample;

    % green
    sample = x_train(i,:,:,2);
    sample = squeeze(sample);
    sample = reshape(sample, [size(sample,1)*size(sample,2),1]);
    x_train_g(:,i) = sample;

    % blue
    sample = x_train(i,:,:,3);
    sample = squeeze(sample);
    sample = reshape(sample, [size(sample,1)*size(sample,2),1]);
    x_train_b(:,i) = sample;
end

x_val_r = [];
x_val_g = [];
x_val_b = [];
for i = 1 : size(x_val, 1)
    % red
    sample = x_val(i,:,:,1);
    sample = squeeze(sample);
    sample = reshape(sample, [size(sample,1)*size(sample,2),1]);
    x_val_r(:,i) = sample;

    % green
    sample = x_val(i,:,:,2);
    sample = squeeze(sample);
    sample = reshape(sample, [size(sample,1)*size(sample,2),1]);
    x_val_g(:,i) = sample;

    % blue
    sample = x_val(i,:,:,3);
    sample = squeeze(sample);
    sample = reshape(sample, [size(sample,1)*size(sample,2),1]);
    x_val_b(:,i) = sample;
end

x_test_r = [];
x_test_g = [];
x_test_b = [];
for i = 1 : size(x_test, 1)
    % red
    sample = x_test(i,:,:,1);
    sample = squeeze(sample);
    sample = reshape(sample, [size(sample,1)*size(sample,2),1]);
    x_test_r(:,i) = sample;

    % green
    sample = x_test(i,:,:,2);
    sample = squeeze(sample);
    sample = reshape(sample, [size(sample,1)*size(sample,2),1]);
    x_test_g(:,i) = sample;

    % blue
    sample = x_test(i,:,:,3);
    sample = squeeze(sample);
    sample = reshape(sample, [size(sample,1)*size(sample,2),1]);
    x_test_b(:,i) = sample;
end

%% PCA
n_components = 309;

% PCA canale red
dataset_r = [x_train_r, x_val_r, x_test_r];
[U_r, Z_r] = pca(dataset_r', 'NUMCOMPONENTS', n_components);

% PCA canale green
dataset_g = [x_train_g, x_val_g, x_test_g];
[U_g, Z_g] = pca(dataset_g', 'NUMCOMPONENTS', n_components);

% PCA canale blue
dataset_b = [x_train_b, x_val_b, x_test_b];
[U_b, Z_b] = pca(dataset_b', 'NUMCOMPONENTS', n_components);


%% Ricostruzione

Reconstruction_r = Z_r*U_r';
Reconstruction_g = Z_g*U_g';
Reconstruction_b = Z_b*U_b';

Reconstruction = cat(3, Reconstruction_r, Reconstruction_g, Reconstruction_b);
Reconstruction = double(Reconstruction);

%% 

figure
idx = 1752; %1450 blue
img = squeeze(Reconstruction(idx,:,:));
img = reshape(img, 28, 28, 3);
imagesc(img)

figure
dataset = cat(1, x_train, x_val, x_test);
imagesc(squeeze(dataset(idx,:,:,:)))

This is what I get:

Answer 1

[Neha]

Hi Matteo,

I understand that you want to apply PCA for RGB images obtained from DermMNIST. Rather than applying PCA to individual channels and then reconstructing the image, you can reshape the images into a 2D array without separating the color channels where you will have 3 columns which correspond to the three color channels of the RGB image and the number of rows would be the number of pixels of the image. You can refer to the following MATLAB Answer for the code snippet:

https://www.mathworks.com/matlabcentral/answers/328537-apply-pca-on-a-set-of-gray-faces-images

Apart from PCA, you can also consider using GLCM (Gray-Level Co-occurence Matrix) to extract features from an image. For a skin disease dataset, texture can indeed be a very important feature and since skin diseases manifest as visible changes in the texture of the skin, such as roughness and scaliness. You can refer to the following documentation link for more information on GLCM:

https://www.mathworks.com/help/images/ref/graycomatrix.html

Hope this helps!

[Matteo]

Hi Neha, thanks a lot for your answer!

Yes, I'm trying to apply PCA for images from DermaMNIST. I'm working on a university project and I'm supposed to use only PCA to preprocess my data. Anyway, thanks for your piece of advice about GLCM.

I tried to do what you told me with just an image, but I got the same result. Here's my code:

img = x_train(1752,:,:,:);
img = squeeze(img);

red = img(:,:,1);
red = reshape(red, [28*28,1]);
green = img(:,:,2);
green = reshape(green, [28*28,1]);
blue = img(:,:,3);
blue = reshape(blue, [28*28,1]);

img = [red, green, blue];

[U, Z] = pca(img);

reconstruction = Z * U';

pca_red = reshape(recon(:,1), [28, 28]);
pca_green = reshape(recon(:,2), [28, 28]);
pca_blue = reshape(recon(:,3), [28, 28]);


pca_img = cat(3, pca_red, pca_green, pca_blue);
imshow(pca_img)

I also tried to flatten the image in a 28*28*3 array and applied PCA to that array, but the reconstructed image is always the same.

I also tried with

[U, Z] = pca(img, 'Centered', false)

and it actually works, but I don't really get the reason why. Shouldn't we always normalize and center the data to find principal components?

Answer 2

[Image Analyst]

Here's a demo that converts an RGB image into the PCA channels. Adapt as needed.

[Matteo]

Hello, thanks a lot, I actually found this demo days ago and, indeed, used it to try to apply PCA to my dataset.

My main problem isn't really how to apply PCA, but how to reconstruct the image after having applied PCA, because what I get is very far from the original picture.

I used this demo and tried to reconstruct the image. I did it just by combining the threee channels in a tensor, as in this code:

pcWhole = cat(3, pc1Image, pc2Image, pc3Image);
figure, imshow(pcWhole, [])

This is what I get

whose colours are not that similar to the original ones.

I was told that before apllying PCA, we should always normalize the image, so I divided by 255, as in the code:

rgbImage = imread(fullFileName);
rgbImage = double(rgbImage)/255;

But still I don't get a nice reconstructed image

Besides, as written on MatLab documentation, to reconstruct an image we should calculate score * coeff', why do you compute transformedImagePixelList = listOfRGBValues * coeff instead? Am I missing something?

So I tried to calculate transformedImagePixelList as score * coeff', and this is what I get

which is not that different from the original one, but kind of darker.

Finally, if I try not to normalize, but to calculate transformedImagePixelList as before, I get

I dont' really get what I am doing wrong.