Main Content

Sobel Edge Detection on NVIDIA Jetson Nano Using Raspberry Pi Camera Module V2

This example uses:

Open Script

This example shows you how to capture and process images from a Raspberry Pi® Camera Module V2 connected to the NVIDIA® Jetson™ Nano. The MATLAB® Coder™ Support Package for NVIDIA Jetson and NVIDIA DRIVE® Platforms allows you to capture images from the Camera Module V2 and bring them into the MATLAB environment for processing. In this example you learn how to develop a Sobel edge detection algorithm by using this capability.

Prerequisites

Target Board Requirements

  • NVIDIA Jetson Nano embedded platform.

  • Raspberry Pi Camera Module V2 connected to the CSI host port of the target.

  • Ethernet crossover cable to connect the target board and host PC (if you cannot connect the target board to a local network).

  • NVIDIA CUDA® toolkit installed on the board.

  • V4L2 and SDL (v1.2) libraries on the board.

  • GStreamer libraries on the board.

  • Environment variables on the target for the compilers and libraries. For more information, see Install and Setup Prerequisites for NVIDIA Boards.

Development Host Requirements

Connect to NVIDIA Jetson Nano

The support package uses an SSH connection over TCP/IP to execute commands while building and running the generated CUDA code on the Jetson Nano platforms. Connect the target platform to the same network as the host computer or use an Ethernet crossover cable to connect the board directly to the host computer. For information on how to set up and configure your board, see NVIDIA documentation.

Create Jetson Object

To communicate with the NVIDIA hardware, create a live hardware connection object by using the jetson function.

hwobj = jetson('jetson-nano-name','ubuntu','ubuntu');

When connecting to the target board for the first time,you must provide the host name or IP address, user name, and password of the target board. On subsequent connections, you do not need to supply the address, user name, and password. The hardware object reuses these settings from the most recent successful connection to an NVIDIA board.

By default, this example reuses the settings from the most recent successful connection to a NVIDIA Jetson board.

hwobj = jetson;

Checking for CUDA availability on the Target...
Checking for 'nvcc' in the target system path...
Checking for cuDNN library availability on the Target...
Checking for TensorRT library availability on the Target...
Checking for prerequisite libraries is complete.
Gathering hardware details...
Checking for third-party library availability on the Target...
Gathering hardware details is complete.
 Board name              : NVIDIA Jetson AGX Xavier Developer Kit
 CUDA Version            : 11.4
 cuDNN Version           : 8.4
 TensorRT Version        : 8.4
 GStreamer Version       : 1.16.3
 V4L2 Version            : 1.18.0-2build1
 SDL Version             : 1.2
 OpenCV Version          : 4.5.4
 Available Webcams       : Logitech Webcam C925e
 Available GPUs          : Xavier
 Available Digital Pins  : 7  11  12  13  15  16  18  19  21  22  23  24  26  29  31  32  33  35  36  37  38  40

During the hardware live object creation, the support package performs hardware and software checks, installs MATLAB IO server on the target board, and gathers information on peripheral devices connected to the target. This information is displayed in the Command Window. In case of a connection failure, a diagnostics error message is reported at the MATLAB command line. If the connection has failed, the most likely cause is incorrect IP address or host name.

Connect to Camera

List Available Cameras

Run the getCameraList function of the hwobj object to find the available cameras. If this function outputs an empty table, then try re-connecting the camera and execute the function again.

camlist = getCameraList(hwobj);

          Camera Name          Video Device     Available Resolutions    Pixel Formats
    _______________________    _____________    _____________________    _____________

    "Logitech Webcam C925e"    "/dev/video2"    "(<a href="matlab:disp('Resolutions: [0     0],[160    90],[160   120],[176   144],[320   180],[320   240],[352   288],[432   240],[640   360],[640   480],[800   448],[800   600],[864   480],[960   720],[1024   576],[1280   720],[1600   896],[1920  1080],[2304  1296],[2304  1536]')">View resolutions</a>)"      "YUYV,MJPG"

The getCameraList function lists the optimum resolutions supported by the camera sensor. At these resolutions, the image acquisition pipeline works efficiently. Based on the requirements of your algorithm, you can pick any supported resolution.

This example uses the first camera from the list and a resolution of 1280-by-720 pixels.

camName = table2array(camlist(1,"Camera Name"));
camResolution = [1280,720];

Create a Camera Object

Create a camera object by using the camera method of the jetson object.

camObj = camera(hwobj,camName,camResolution);

camObj is a handle to a camera object. To display the images captured from the Camera Module V2 in MATLAB, use these commands:

for i = 1:5
    img = snapshot(camObj);
    imshow(img);
    drawnow;
end

This camera object captures RGB and 3-channel grayscale images.

Create a Display Object

To create a display object, use the imageDisplay function. This object is a system object that uses imshow function to display the images in MATLAB.

img = snapshot(camObj);
reshapedImg = cat(3, img(:,:,1).',img(:,:,2)',img(:,:,3)');
dispObj = imageDisplay(hwobj);
image(dispObj,reshapedImg);
delete(dispObj);

Sobel Edge Detection Algorithm

The Sobel edge detection algorithm is a 2-D spatial gradient operation on a grayscale image. This operation emphasizes the high spatial frequency regions pf the image that corresponds to edges.

Calculate Gradients

Find horizontal gradient(h) and vertical gradient (v) of the input image with respective Sobel kernels. These two Sobel kernels are orthogonal to each other. Before processing live image data from the camera, test the algorithm on a sample image.

kern = [1 2 1; 0 0 0; -1 -2 -1];
img = imread('peppers.png');
imshow(img);
h = conv2(img(:,:,2),kern,'same');
v = conv2(img(:,:,2),kern','same');

Calculate Gradient Magnitude

Find the gradient magnitude from the horizontal and vertical gradients (h and v).

e = sqrt(h.*h + v.*v);

Threshold the Edge Image

Threshold the image to find the regions of image that are edges.

edgeImg = uint8((e > 100) * 240);
imshow(edgeImg);

Run Sobel Edge Detection Algorithm on Live Data

Create a MATLAB entry-point function, sobelEdgeDetectionAlg.m, out of the MATLAB code developed in the previous sections of this example.

type('sobelEdgeDetectionAlg.m')

function edgeImg = sobelEdgeDetectionAlg(img,thresh)  %#codegen
%sobelEdgeDetection Example MATLAB function for edge detection.
% Copyright 2018 The MathWorks, Inc.

kern = half([1 2 1; 0 0 0; -1 -2 -1]);

% Finding horizontal and vertical gradients.
h = conv2(img(:,:,2),kern,'same');
v = conv2(img(:,:,2),kern','same');

% Finding magnitude of the gradients.
e = sqrt(h.*h + v.*v);

% Threshold the edges
edgeImg = uint8((e > thresh) * 240);

end

The function sobelEdgeDetectionAlg takes image and threshold input for edge detection and returns the results of edge detection algorithm. Call this function on the images captured from inside a loop. You can vary the threshold variable thresh to get a proper edge image. This way you can use the camera access capability of the support package to tune the algorithm suitable for the specified camera.

dispObj = imageDisplay(hwobj);
for i = 1:10
    img = snapshot(camObj);
    thresh = 60;
    edgeImage = sobelEdgeDetectionAlg(img, thresh);
    reshapedImg = edgeImage';
    image(dispObj,reshapedImg);
end

To deploy this example as a standalone application on the target board, see Deploy and Run Sobel Edge Detection with I/O on NVIDIA Jetson Nano.