readFrame to slow, is there a faster way?

Hello

I want to "cut out" objects that are placed in front of a green screen in real time, and replace the background with a video.

The object extraction itself works fine (tested with a simple image as background).

Then i tried to implement a background video with 'readFrame', but the background video runs in slow motion (see files attached), plus the live stream gets laggy too.

Is there a faster way to acquire single video frames and process them?

Thanks for any help!

function example1
    % webcam
    cam = videoinput('winvideo',1);
    % get background image
    img1 = getsnapshot(cam);
    % video objekt
    vidobj1 = VideoReader('Testvideo.mp4');

      % background
      background = readFrame(vidobj1,'native');
      background = imresize(background, [size(img1, 1), size(img1, 2)]);

      img = imshow(background,'InitialMagnification','fit');

      % Set a custom preview function
      setappdata(img,'UpdatePreviewWindowFcn',@mypreview_fcn);

      % Call preview with the custom image
      preview(cam,img);

      function mypreview_fcn(obj,event,himage)

          % receive the image in event.Data
          foreground = event.Data;
          background = readFrame(vidobj1,'native');

          [overlayed,a] = greenScreen(foreground, background);
          % Display
          set(img, 'CData', overlayed);

      end
  end

Answer 1

[Walter Roberson]

The Computer Vision reader should be faster; see https://www.mathworks.com/help/vision/ref/vision.videofilereader-system-object.html

[david csernay]

Thanks for your fast response.

Unfortunately the video still runs in slow motion (might be a little faster then before).

I also turned off the hardware acceleration.

Is it possible, to run the video in the background, without giving the single frames into the 'overlayed' function, and simply turn the green pixels in the original background to transparent?

(Is it possible to run two layers in the preview() ?)

Would this make the code faster?

[Walter Roberson]

"Is it possible, to run the video in the background, without giving the single frames into the 'overlayed' function, and simply turn the green pixels in the original background to transparent?"

No.

"(Is it possible to run two layers in the preview() ?)"

No.

Note: single precision is not necessarily faster than double precision on the CPU. Although double precision is more computation work, because double precision is used more often, designers often pay more attention to accelerating double precision.

I also suggest you do timing comparisons between

a(a<0) = 0

versus

a = max(a, 0);

This would require that you rename your min and max variables to avoid conflict with the function names.

[david csernay]

If i remove the 'im2single', i have to put a 'double' after the sqrt like this:

distanceToBlue = sqrt(double(....));

Otherwise there is an error: Undefined function 'sqrt' for input arguments of type 'uint8'.

But if i do so, the greenscreen extraction becomes unusable.

The timing comparison showed good results:

a(a<0) = 0

Elapsed time is 0.008337 seconds.

Elapsed time is 0.008922 seconds.

vs

a = max(a, 0);

Elapsed time is 0.002755 seconds.

Elapsed time is 0.003727 seconds.

But the accelerating of the video is not noticeable.

Do you think the code is fundamentally wrong, or is matlab not made for applications of this type?

[Walter Roberson]

There is im2double instead of im2single() . Without testing it is difficult to say which would lead to faster code.

[david csernay]

Ok, with im2double the extraction works, but the video is still slow.

How would you approach the problem? Is there another way for object extraction? Actually, i would like to keep the green screen method, because of its good extraction results.

Answer 2

[Image Analyst]

You could store the video on a solid state drive instead of a hard drive.

[david csernay]

The Laptop i use has an SSD only. Or did i missunderstand your answer?