VAE import custom data
20 views (last 30 days)
Show older comments
I recently set up a VAE based on this link: https://au.mathworks.com/help/deeplearning/ug/train-a-variational-autoencoder-vae-to-generate-images.html
I was able to implement this line by line. That being said, I am trying to get my own custom data uploaded to the Ai for training.
I have 2 rar files, with color images (1024x1024 [will be resized]). The first file has a lot of images for training, and the second file has a few images for testing the algorithm.
The issue is, the VAE used in the link uses a function called "processImagesMNIST", I have looked into the function that makes this up, not sure on how I have to structure my training and testing data to meet the criteria that the function allows for.
Or is there another way to upload my own training data that the VAE can use, without using the special function?
function X = processImagesMNIST(filename)
% The MNIST processing functions extract the data from the downloaded IDX
% [What are IDX files???]
% files into MATLAB arrays. The processImagesMNIST function performs these
% operations: Check if the file can be opened correctly. Obtain the magic
% number by reading the first four bytes. The magic number is 2051 for
% image data, and 2049 for label data. Read the next 3 sets of 4 bytes,
% which return the number of images, the number of rows, and the number of
% columns. Read the image data. Reshape the array and swaps the first two
% dimensions due to the fact that the data was being read in column major
% format. Ensure the pixel values are in the range [0,1] by dividing them
% all by 255, and converts the 3-D array to a 4-D dlarray object. Close the
% file.
%[What do I need to do to make my dataset applicable for this function? Or is there another way to implement my own data?]
dataFolder = fullfile(tempdir,'mnist');
gunzip(filename,dataFolder)
[~,name,~] = fileparts(filename);
[fileID,errmsg] = fopen(fullfile(dataFolder,name),'r','b');
if fileID < 0
error(errmsg);
end
magicNum = fread(fileID,1,'int32',0,'b');
if magicNum == 2051
fprintf('\nRead MNIST image data...\n')
end
numImages = fread(fileID,1,'int32',0,'b');
fprintf('Number of images in the dataset: %6d ...\n',numImages);
numRows = fread(fileID,1,'int32',0,'b');
numCols = fread(fileID,1,'int32',0,'b');
X = fread(fileID,inf,'unsigned char');
X = reshape(X,numCols,numRows,numImages);
X = permute(X,[2 1 3]);
X = X./255;
X = reshape(X, [28,28,1,size(X,3)]);
fclose(fileID);
end
0 Comments
Answers (1)
Image Analyst
on 8 Feb 2023
I would just extract all your images to regular image files (like PNG images) and then instead of calling
X = processImagesMNIST(filename)
just use imread()
X = imread(filename)
18 Comments
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!