This example shows you how to capture and process images from a webcam connected to the NVIDIA® Jetson using the
GPU Coder™ Support Package for NVIDIA GPUs. The GPU Coder Support Package for NVIDIA GPUs allows you to capture images from the web camera and bring them right into the MATLAB® environment for processing. In this example you will learn how to develop a Soble edge detection algorithm by using this capability.
Target Board Requirements
NVIDIA Jetson TX1/TX2 embedded platform.
Ethernet crossover cable to connect the target board and host PC (if the target board cannot be connected to a local network).
V4L2 library on the target.
Environment variables on the target for the compilers and libraries. For information on the supported versions of the compilers and libraries and their setup, see installing and setting up pre-requisites for NVIDIA boards.
USB webcam connected to the USB host port of the target.
Development Host Requirements
GPU Coder (TM) for code generation. If you are new to GPU Coder, visit the GPU Coder product page for an overview and tutorials.
GPU Coder Support Package for NVIDIA GPUs. If you are new to this support package, we recommend completing the Getting Started with GPU Coder Support Package for NVIDIA GPUs example.
NVIDIA CUDA® toolkit on the host.
Environment variables on the host for the compilers and libraries. For information on the supported versions of the compilers and libraries, see Third-party Products. For setting up the environment variables, see Environment Variables.
Connect to the NVIDIA Hardware The GPU Coder Support Package for NVIDIA GPUs uses an SSH connection over TCP/IP to execute commands while building and running the generated CUDA code on the DRIVE or Jetson platforms. You must therefore connect the target platform to the same network as the host computer or use an Ethernet crossover cable to connect the board directly to the host computer. Refer to the NVIDIA documentation on how to set up and configure your board.
To communicate with the NVIDIA hardware, you must create a live hardware connection object by using the drive or jetson function. You must know the host name or IP address, username, and password of the target board to create a live hardware connection object. For example, use the following command to create live object for Jetson hardware,
hwobj = jetson('jetson-tx2-name','ubuntu','ubuntu');
webcamlist property of the
hwobj object to find the web camera name. If this property is empty, then try re-connecting the USB webcam and run the following command. This command tries to scan the availble webcams on the target when there is a addition or deletion.
Use the coder.checkGpuInstall function and verify that the compilers and libraries needed for running this example are set up correctly.
envCfg = coder.gpuEnvConfig('jetson'); envCfg.BasicCodegen = 1; envCfg.Quiet = 1; envCfg.HardwareObject = hwobj; coder.checkGpuInstall(envCfg);
Create a webcam object using the name or the index number of the webcamlist. If no webcam name or the index is specified, then it uses the default webcam.
cam = webcam(hwobj);
cam is a handle to a webcam object. To display the images captured from webcam in MATLAB, use the following commands.
for i = 1:100 img = snapshot(cam); imagesc(img); drawnow; end
This webcam object captures RGB and 3-channel gray scale images.
imageDisplay function to create a display object. This is a system object that uses
imshow function to display the images in MATLAB.
dispObj = imageDisplay(hwobj);
Display an image using the above created display object.
img = snapshot(cam); image(dispObj,img);
The Sobel edge detection algorithm is a popular yet simple edge detection algorithm. In this algorithm, a 2-D spatial gradient operation on a grayscale image is performed. This operation emphasizes the high spatial frequency regions which corresponds to edges.
Calculate the Gradients
We will find horizontal gradient(h) and vertical gradient (v) of the input image with respective Sobel kernels. These two Sobel kernels are orthogonal to each other. We will make sure our algorithm works on the test image before moving on to live data.
kern = [1 2 1; 0 0 0; -1 -2 -1]; img = imread('peppers.png'); imagesc(img); h = conv2(img(:,:,2),kern,'same'); v = conv2(img(:,:,2),kern','same');
Calculate the Gradient Magnitude
Next we find the gradient magnitude from the horizontal and vertical gradients (h and v).
e = sqrt(h.*h + v.*v);
Threshold the Edge Image
We threshold the image to find the regions of image that we consider to be edges.
edgeImg = uint8((e > 100) * 240); imagesc(edgeImg);
We can create a MATLAB function, sobelEdgeDetectionAlg.m, out of the MATLAB code we developed in the previous sections of this example. View the MATLAB function in the editor.
The function sobelEdgeDetectionAlg() takes image and threshold input for edge detection and returns the results of edge detection algorithm. We will call this function on the images captured in a loop. The threshold variable
thresh can be varied to get a proper edge image. This way we can use the webcam access capability of the support package to tune the algorithm suitable for the specified webcam.
for i = 1:200 img = snapshot(cam); thersh = 100; edgeImage = sobelEdgeDetectionAlg(img, thersh); image(dispObj,edgeImage); end
To deploy the above example as a stand-alone application on the target, follow the example Deploy and Run Sobel Edge Detection with I/O on NVIDIA Jetson
Close the opened windows
This example introduced an application where images coming from a webcam connected to an NVIDIA Jetson are processed in MATLAB using a Sobel edge detection algorithm.