Simulation of Camera Inputs

3 views (last 30 days)
Philippe Decoste
Philippe Decoste on 17 Feb 2021
Answered: Shreshth on 16 May 2024
I am trying to simulate a robotic platform which has a depth camera used to track the trajectory of an object being thrown. I have tried simulating this already by making a few functions which simulates and plots a ball being thrown, but the axes option for simulating what is seen from a particular point of view (especially with the correct field of view that the real camera would have) is very finnicky. Ideally, I would have a camera whose position, direction, and field of view I can chose, so that I can take the frames it sees as inputs for other functions that would process the images and determine where the ball will land. Is there a more clear cut way to do this (maybe with simscape)?

Answers (1)

Shreshth
Shreshth on 16 May 2024
Hello Philippe,
I can understand that you are trying to simulate a robotic platform with a depth camera to track the trajectory of an object, like a ball being thrown. This involves several steps, including simulating the object's motion, the camera's perception of this motion, and the processing of this perception to predict outcomes such as landing points.
While MATLAB offers several tools and toolboxes for such simulations, including Simscape for physical modeling and the Computer Vision Toolbox for camera simulations and image processing, there isn't a direct, out-of-the-box solution for your specific need. However, you can build a simulation by combining functionalities from these toolboxes.
Step 1: Simulate the Ball’s Trajectory
You can start by simulating the ball's trajectory using basic physics equations for projectile motion or using Simscape for a more physical approach that can include air resistance and other factors.
Step 2: Simulate the Camera
Use the Computer Vision Toolbox to simulate the camera with a specific position, orientation and Field of view (FOV).
Step 3: Generate Frames from the camera’s perspective
Using the simulated camera, you can generate frames that represent what the camera sees at different points in time. This involves projecting the 3D trajectory of the ball onto the 2D plane of the camera's image sensor, considering the camera's position, orientation, and FOV.
Step 4: Process the Frames to Predict the Landing Point
Once you have the frames, you can process them using image processing techniques to track the ball's position over time involving Object detection and Trajectory Estimation.
To understand the working of MATLAB tools involved in the process you can refer to the below mentioned MathWorks documentation for reference:
Simscape :
Computer vision Toolbox:
Hope it helps.

Categories

Find more on Simscape Multibody in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!