Optical Flow with Parrot Minidrones
The Parrot® minidrone uses various onboard sensors like Accelerometer and Gyroscope to stabilize itself. But, due to minor imperfections, the drone does not stay in place and tends to drift. This situation can be avoided by using images from the downward-facing camera of the drone. The camera continuously captures the images and checks for motion in X and Y directions. The flight controller uses the camera data, in addition to Accelerometer and Gyroscope, to stabilize itself in space and hold the x-y position of the drone.
To understand optical flow, consider a case where the drone drifts by a distance of dx in x and dy in y direction, due to some noise in the sensors. In this case, the controller needs to know the distances moved in both x and y directions to bring it back to the original position.
This can be obtained using the downward-facing camera by comparing consecutive images as the drone moves. The downward facing camera fixes onto objects/patterns on the ground and if there is any change in position of that object/pattern, the camera reports it as a movement in terms of velocity – which is optical flow. This velocity measurement can be used to calculate the distance moved by the drone.
Troubleshooting Optical Flow Issues
There are cases in which the camera cannot detect the horizontal movement. If the texture of the surface below the drone is uniform with no patterns, the camera might not detect movement. Also, when the lighting is low, the camera might not detect movement. In such cases, the optical flow values will be zero resulting in drift and possible crash of the drone.
To avoid such situations, we have modelled a logic (crash prediction logic) in the
new Simulink® project template,
Hover Parrot Minidrone. The
parrotMinidroneHover project (which is part of the template)
Crash Predictor Flags subsystem (in the
Simulink model). This subsystem is used to shut down the drone if there is no
optical flow measurement for 50 continuous sample times.
If the drone shuts-down due to optical flow issues, some fixes can be done to the surface to increase contrast. Increasing contrast on the surface ensures that your camera produces non-zero optical flow values.
The below image shows a scenario of a surface where you may see optical flow errors:
The region marked in red is a surface of uniform texture. When the drone is flying over the surface, it receives zero optical flow information. This is what the drone sees:
In this case, the drone will drift and eventually crash.
To increase contrast, add tapes on the surface over which the drone is flying. This change avoids the shutdown of drone due to optical flow issues.
The below image shows an example of adding tapes to the surface to increase contrast.
This is what the drone sees now:
The texture on the surface need not be as shown in the above image. It can be any random pattern as long as there is a contrast on the surface.
In cases where there are optical flow issues even though there is contrast on the surface, the lighting can be an issue. In this case, increase the room's lighting so that the drone can get non-zero optical flow values.
Simulink Model for Optical Flow
parrotMinidroneHover project (which is part of
Hover Parrot Minidrone
Simulink template), the crash prediction logic to shut down the drone is
modeled in the
Crash Prediction Flags subsystem in the
Crash Prediction Flags subsystem, the
<opticalFlow_data> is a vector obtained from
and it has the following fields:
dx_OF (velocity in x-direction),
dy_OF (velocity in y-direction) and
OF_status is ‘0’ when the
drone is unable to detect optical flow and ‘1’ when the drone detects optical flow.
The maximum number of cycles for which there is no optical flow can be changed by
changing the value of the
Compare to constant block, highlighted
in purple. Once the specified value is reached, the drone shuts down.
To disable the Optical flow, perform one of the following options:
Increase the value of the
Compare to constantblock (highlighted in purple) to a very large value.
Increasing the count (maximum number of cycles for which there is no optical flow) to a very large value may cause the drone to drift.
No optical flowsubsystem (highlighted in blue) and set the value of the Constant block inside the subsystem to
Removal of Enable Optical Flow Option from Configuration Parameters
The Enable Optical Flow option in the Configuration
Parameters dialog box is now removed, and the functionality is exposed in the
flightControlSystem model as shown above. The option is
deprecated from R2019a onwards.
Limitations/ Backward Compatibility
For the older Simulink models (before R2019a), the optical flow behavior is
unchanged (the Parrot drone shuts down after 50 cycles without optical flow). However, it is
recommended to move to the new template (
Hover Parrot Minidrone)
introduced in R2019a.