Storing multiple image frames in a 4 D unit

I have a while loop that aquires images from 2 seperate IP cameras (which works fine).

the images are to be stored in a 4D-unit so that every frame occupies one indes in the 4th dimension of that unit.

Despite the frames showing an image (using imshow to verify) the final 4D units only contains black frames.

while toc(startTime) < videoDuration
    % Acquire frames from each camera
    frame1 = getsnapshot(camA);
    frame2 = getsnapshot(camB);
%     frame1 = im2gray(frame1);
%     frame2 = im2gray(frame2);
    % Store frames in the 4D matrices for each camera
    videoDataCam1(:, :, :,frameIndex) = frame1;
    videoDataCam2(:, :, :,frameIndex) = frame2;
    
    frameIndex = frameIndex + 1;
end

I assume i have some glaring error in my Syntax; especcially in the lines where the array is supposed to be written in. If you find it, i would be thankful for an answer.

[DGM]

Are you preallocating the arrays or growing them? I don't really see anything limiting framerate, though I don't know that it'd be a problem.

inpict = imread('peppers.png');
videoDuration = 1;

frameIndex = 1;
startTime = tic;
while toc(startTime) < videoDuration
    % i don't have a camera
    frame1 = inpict;
    
    % Store frames in the 4D matrices for each camera
    videoDataCam1(:, :, :,frameIndex) = frame1;
    
    frameIndex = frameIndex + 1;
end

% there's image content in the final frame
range(videoDataCam1(:,:,:,end),'all')

ans = uint8 255

[Frederik Vogel]

Thanks for the quick response!

Yes i do initialize the Arrays. for a better overview here is the full script. it is rather short

% Define video input objects for cameras
camA = videoinput("gentl", 1, "BayerBG8"); % Modify properties based on your first camera
camB = videoinput("gentl", 2, "Mono8"); % Modify properties based on your second camera

camB.ReturnedColorspace = "rgb";

src = getselectedsource(camB);
src.AtmosphericTemperature = 223;
src.CMOSBitDepth = "bit8bit";
src.ReflectedTemperature = 223;
src.WindowTemperature = 223;

snapA=getsnapshot(camA)
% snapA=im2gray(snapA)
snapB=getsnapshot(camB)
%snapB=im2gray(snapB);

[szA1,szA2,c1]=size(snapA);
[szB1,szB2,c2]=size(snapB);
% Set the video duration (in seconds)
videoDuration = 10;  % Adjust the duration as needed

% Initialize the 4D matrices to store frames for each camera
videoDataCam1 = zeros(szA1, szA2, 3, round(videoDuration * 30)); % Assuming 30 frames per second
videoDataCam2 = zeros(szB1, szB2, 3, round(videoDuration * 30)); % Assuming 30 frames per second

% Start the cameras
% start(camA);
% start(camB);

% Record the videos
frameIndex = 1;
startTime = tic;

while toc(startTime) < videoDuration
    % Acquire frames from each camera
    frame1 = getsnapshot(camA);
    frame2 = getsnapshot(camB);
%     frame1 = im2gray(frame1);
%     frame2 = im2gray(frame2);
    % Store frames in the 4D matrices for each camera
    videoDataCam1(:, :, :,frameIndex) = frame1;
    videoDataCam2(:, :, :,frameIndex) = frame2;
    
    frameIndex = frameIndex + 1;
end

[DGM]

Unless getsnapshot() is somehow locked to the framerate (which I don't think it is), I don't see anything here that controls how many frames get captured. So you'll either get less than 300, or more than 300 frames. It's unlikely that you'll reliably get exactly 300 frames.

If you want a fixed number of frames, I don't know why you don't just capture a fixed number of frames (use a for loop) instead of relying on the loop timing to be exact.

I gave up trying to get videoinput() to work with my capture card, so this isn't something I can test.

Answer 1

[Nipun]

Hi Frederik,

I understand that you are trying to save the video frames in a four dimensional matrix for computation later, but are getting black frames on retrieving the frames.

Based on the information provided, it looks like you are using "BayerBG8" format for camA and "Mono8" format for camB. The BayerBG8 format needs to be converted to RGB before storing it in a 4D matrix. Additionally, you need to make sure that the data type and range of the images are compatible with the expected format.

Here's an updated version of your code that should handle these issues:

% ... (Your previous code)

% Initialize the 4D matrices to store frames for each camera
videoDataCam1 = zeros(szA1, szA2, 3, round(videoDuration * 30), 'uint8'); % Assuming 30 frames per second
videoDataCam2 = zeros(szB1, szB2, 3, round(videoDuration * 30), 'uint8'); % Assuming 30 frames per second

% Start the cameras
start(camA);
start(camB);

% Record the videos
frameIndex = 1;
startTime = tic;

while toc(startTime) < videoDuration
    % Acquire frames from each camera
    frame1 = getsnapshot(camA);
    frame2 = getsnapshot(camB);
    
    % Convert BayerBG8 to RGB for camA
    frame1 = demosaic(frame1, 'rgb');
    
    % Convert Mono8 to RGB for camB
    frame2 = cat(3, frame2, frame2, frame2);
    
    % Store frames in the 4D matrices for each camera
    videoDataCam1(:, :, :, frameIndex) = frame1;
    videoDataCam2(:, :, :, frameIndex) = frame2;
    
    frameIndex = frameIndex + 1;
end

% ... (Your code continues)

In this version, I added the conversion from BayerBG8 to RGB using the demosaic function for camA. For camB, I used the cat function to create an RGB image from the single-channel Mono8 image. I also specified the data type as 'uint8' when initializing the 4D matrices to ensure compatibility.

Link to documentation:

1. cat in MATLAB - https://www.mathworks.com/help/matlab/ref/double.cat.html
2. demosaic in MATLAB - https://www.mathworks.com/help/images/ref/demosaic.html

Hope this helps.

Regards,

Nipun