Main Content

Sobel Edge Detection on NVIDIA Jetson Nano Using Raspberry Pi Camera Module V2

Open Script

This example shows you how to capture and process images from a Raspberry Pi Camera Module V2 connected to the NVIDIA® Jetson Nano. The MATLAB® Coder™ Support Package for NVIDIA Jetson and NVIDIA DRIVE Platforms allows you to capture images from the Camera Module V2 and bring them into the MATLAB environment for processing. In this example you learn how to develop a Sobel edge detection algorithm by using this capability.

Prerequisites

Target Board Requirements

  • NVIDIA Jetson Nano embedded platform.

  • Raspberry Pi Camera Module V2 connected to the CSI host port of the target.

  • Ethernet crossover cable to connect the target board and host PC (if you cannot connect the target board to a local network).

  • NVIDIA CUDA toolkit installed on the board.

  • V4L2 and SDL (v1.2) libraries on the board.

  • GStreamer libraries on the board.

  • Environment variables on the target for the compilers and libraries. For more information, see Install and Setup Prerequisites for NVIDIA Boards.

Development Host Requirements

Create a Folder and Copy Relevant Files

The following line of code creates a folder in your current working folder on the host and copies all the relevant files into this folder. If you cannot generate files in this folder, before running this command, change your current working folder.

nvidiademo_setup('sobel_edge_detection');

Connect to NVIDIA Jetson Nano

The support package uses an SSH connection over TCP/IP to execute commands while building and running the generated CUDA code on the Jetson Nano platforms. Connect the target platform to the same network as the host computer or use an Ethernet crossover cable to connect the board directly to the host computer. For information on how to set up and configure your board, see NVIDIA documentation.

To communicate with the NVIDIA hardware, create a live hardware connection object by using the jetson function. You must know the host name or IP address, user name, and password of the target board to create a live hardware connection object. For example, when connecting to the target board for the first time, create a live object for Jetson hardware by using the command:

hwobj = jetson('jetson-nano-name','ubuntu','ubuntu');

During the hardware live object creation, the support package performs hardware and software checks, IO server installation, and gathers peripheral information on target. This information is displayed in the Command Window.

Run the getCameraList function of the hwobj object to find the available cameras. If this function outputs an empty table, then try re-connecting the camera and execute the function again.

camlist = getCameraList(hwobj);

Verify GPU Environment on Target Board

To verify that the compilers and libraries necessary for running this example are set up correctly, use the coder.checkGpuInstall (GPU Coder) function.

envCfg = coder.gpuEnvConfig('jetson');
envCfg.BasicCodegen = 1;
envCfg.Quiet = 1;
envCfg.HardwareObject = hwobj;
coder.checkGpuInstall(envCfg);

Create a Camera Object

Create a camera object by using the name from the getCameraList function. For example, if the camera is named vi-output, imx219 6-0010, use:

camObj = camera(hwobj,"vi-output, imx219 6-0010",[640 480]);

camObj is a handle to a camera object. To display the images captured from the Camera Module V2 in MATLAB, use these commands:

for i = 1:100
    img = snapshot(camObj);
    imagesc(img);
    drawnow;
end

This camera object captures RGB and 3-channel grayscale images.

Create a Display Object

To create a display object, use the imageDisplay function. This object is a system object that uses imshow function to display the images in MATLAB.

dispObj = imageDisplay(hwobj);
img = snapshot(camObj);
image(dispObj,img);

Sobel Edge Detection Algorithm

The Sobel edge detection algorithm is a 2-D spatial gradient operation on a grayscale image. This operation emphasizes the high spatial frequency regions pf the image that corresponds to edges.

Calculate Gradients

Find horizontal gradient(h) and vertical gradient (v) of the input image with respective Sobel kernels. These two Sobel kernels are orthogonal to each other. Before processing live image data from the camera, test the algorithm on a sample image.

kern = [1 2 1; 0 0 0; -1 -2 -1];
img = imread('peppers.png');

imagesc(img);


h = conv2(img(:,:,2),kern,'same');
v = conv2(img(:,:,2),kern','same');

Calculate Gradient Magnitude

Find the gradient magnitude from the horizontal and vertical gradients (h and v).

e = sqrt(h.*h + v.*v);

Threshold the Edge Image

Threshold the image to find the regions of image that are edges.

edgeImg = uint8((e > 100) * 240);

imagesc(edgeImg);

Run Sobel Edge Detection Algorithm on Live Data

Create a MATLAB entry-point function, sobelEdgeDetectionAlg.m, out of the MATLAB code developed in the previous sections of this example. View the code in MATLAB editor.

edit('sobelEdgeDetectionAlg.m');

The function sobelEdgeDetectionAlg takes image and threshold input for edge detection and returns the results of edge detection algorithm. Call this function on the images captured from inside a loop. You can vary the threshold variable thresh to get a proper edge image. This way you can use the camera access capability of the support package to tune the algorithm suitable for the specified camera.

for i = 1:200
    img = snapshot(camObj);
    thresh = 100;
    edgeImage = sobelEdgeDetectionAlg(img, thresh);
    image(dispObj,edgeImage);
end

To deploy this example as a standalone application on the target board, see Deploy and Run Sobel Edge Detection with I/O on NVIDIA Jetson Nano.

Cleanup

To remove the example files and return to the original folder, call the cleanup function.

cleanup