Sobel Edge Detection using Webcam on NVIDIA Jetson

This example shows you how to capture and process images from a webcam connected to the NVIDIA® Jetson using the GPU Coder™ Support Package for NVIDIA GPUs. The GPU Coder Support Package for NVIDIA GPUs allows you to capture images from the web camera and bring them right into the MATLAB® environment for processing. In this example you will learn how to develop a Sobel edge detection algorithm by using this capability.

Prerequisites

Target Board Requirements

  • NVIDIA Jetson TX1/TX2 embedded platform.

  • Ethernet crossover cable to connect the target board and host PC (if the target board cannot be connected to a local network).

  • V4L2 library on the target.

  • Environment variables on the target for the compilers and libraries. For information on the supported versions of the compilers and libraries and their setup, see installing and setting up pre-requisites for NVIDIA boards.

  • USB webcam connected to the USB host port of the target.

Development Host Requirements

Create a Web Camera Object

Connect to the NVIDIA Hardware The GPU Coder Support Package for NVIDIA GPUs uses an SSH connection over TCP/IP to execute commands while building and running the generated CUDA code on the DRIVE or Jetson platforms. You must therefore connect the target platform to the same network as the host computer or use an Ethernet crossover cable to connect the board directly to the host computer. Refer to the NVIDIA documentation on how to set up and configure your board.

To communicate with the NVIDIA hardware, you must create a live hardware connection object by using the drive or jetson function. You must know the host name or IP address, username, and password of the target board to create a live hardware connection object. For example, use the following command to create live object for Jetson hardware,

hwobj = jetson('jetson-tx2-name','ubuntu','ubuntu');

Check the webcamlist property of the hwobj object to find the web camera name. If this property is empty, then try re-connecting the USB webcam and run the following command. This command tries to scan the available webcams on the target when there is a addition or deletion.

hwobj.updatePeripheralInfo;

Verify the GPU Environment

Use the coder.checkGpuInstall function and verify that the compilers and libraries needed for running this example are set up correctly.

envCfg = coder.gpuEnvConfig('jetson');
envCfg.BasicCodegen = 1;
envCfg.Quiet = 1;
envCfg.HardwareObject = hwobj;
coder.checkGpuInstall(envCfg);

Create a Web Camera Object

Create a webcam object using the name or the index number of the webcamlist. If no webcam name or the index is specified, then it uses the default webcam.

cam = webcam(hwobj);

cam is a handle to a webcam object. To display the images captured from webcam in MATLAB, use the following commands.

for i = 1:100
    img = snapshot(cam);
    imagesc(img);
    drawnow;
end

This webcam object captures RGB and 3-channel gray scale images.

Create a Display Object

Use the imageDisplay function to create a display object. This is a system object that uses imshow function to display the images in MATLAB.

dispObj = imageDisplay(hwobj);
Display an image using the above created display object.
img = snapshot(cam);
image(dispObj,img);

Sobel Edge Detection Algorithm

The Sobel edge detection algorithm is a popular yet simple edge detection algorithm. In this algorithm, a 2-D spatial gradient operation on a grayscale image is performed. This operation emphasizes the high spatial frequency regions which corresponds to edges.

Calculate the Gradients

We will find horizontal gradient(h) and vertical gradient (v) of the input image with respective Sobel kernels. These two Sobel kernels are orthogonal to each other. We will make sure our algorithm works on the test image before moving on to live data.

kern = [1 2 1; 0 0 0; -1 -2 -1];
img = imread('peppers.png');
imagesc(img);
h = conv2(img(:,:,2),kern,'same');
v = conv2(img(:,:,2),kern','same');

Calculate the Gradient Magnitude

Next we find the gradient magnitude from the horizontal and vertical gradients (h and v).

e = sqrt(h.*h + v.*v);

Threshold the Edge Image

We threshold the image to find the regions of image that we consider to be edges.

edgeImg = uint8((e > 100) * 240);
imagesc(edgeImg);

Run Sobel Edge Detection Algorithm on Live Data

We can create a MATLAB function, sobelEdgeDetectionAlg.m, out of the MATLAB code we developed in the previous sections of this example. View the MATLAB function in the editor.

edit('sobelEdgeDetectionAlg.m');

The function sobelEdgeDetectionAlg() takes image and threshold input for edge detection and returns the results of edge detection algorithm. We will call this function on the images captured in a loop. The threshold variable thresh can be varied to get a proper edge image. This way we can use the webcam access capability of the support package to tune the algorithm suitable for the specified webcam.

for i = 1:200
    img = snapshot(cam);
    thersh = 100;
    edgeImage = sobelEdgeDetectionAlg(img, thersh);
    image(dispObj,edgeImage);
end

To deploy the above example as a stand-alone application on the target, follow the example Deploy and Run Sobel Edge Detection with I/O on NVIDIA Jetson.

Close the opened windows

close all

Summary

This example introduced an application where images coming from a webcam connected to an NVIDIA Jetson are processed in MATLAB using a Sobel edge detection algorithm.