# How to convert optical flow to feature matrix in videos

8 views (last 30 days)
Kong on 24 Mar 2020
Answered: Raunak Gupta on 30 Apr 2020
Hello.
I want to classify videos using optical flow.
After using this code, I got this result.
Which data can I save as matrix for classification?
I think that I can use optical flow data, but there are 4 matrix(Vx, Vy, Orientation, Magnitude). How can I deal with these data?
I will save the data of opticalFlow and use dimensionality reduction. However, I don't know which property can be used for this process.  clear all
close all
opticFlow = opticalFlowFarneback
h = figure;
movegui(h);
hViewPanel = uipanel(h,'Position',[0 0 1 1],'Title','Plot of Optical Flow Vectors');
hPlot = axes(hViewPanel);
frameGray = rgb2gray(frameRGB);
flow = estimateFlow(opticFlow,frameGray);
imshow(frameRGB)
hold on
plot(flow,'DecimationFactor',[5 5],'ScaleFactor',2,'Parent',hPlot);
hold off
pause(10^-3)
end

Raunak Gupta on 30 Apr 2020
Hi,
I am unsure about what you want to classify in the video using the optical flow estimation workflow. As per my understanding the above code estimate the direction and magnitude of the movement of any pixel in two consecutive frames. So essentially you can use the algorithm to tell if any object which consists of many pixels is moving or not.
For this you may use any of the pair (Vx,Vy) or (Orientation,Magnitude) as a reference to check if an object is moving. These two pairs are related as for every pixel in the image,
Orientation = atan(Vy./Vx);
Magnitude = sqrt(Vx.^2 + Vy.^2);
So basically, you may cluster the points which have same (Vx,Vy) in a region that might represent an object because for a single object its velocity for every point will remain same.
I would suggest saving any of the pair (Vx,Vy) or (Orientation,Magnitude) for dimensionality reduction as that will bring all the data available from optical flow estimation.