Deploy Object Detection Model as Microservice

Supported platform: Linux^®, Windows^®, macOS

This example shows how to create a microservice Docker^® image from a MATLAB^® object detection model. The microservice image created by MATLAB Compiler SDK™ provides an HTTP/HTTPS endpoint to access MATLAB code.

You package a MATLAB function into a deployable archive, and then create a Docker image that contains the archive and a minimal MATLAB Runtime package. You can then run the image in Docker and make calls to the service using any of the MATLAB Production Server™ client APIs.

Required Products

Type ver at the MATLAB command prompt to verify whether the following products are installed:

MATLAB
Image Processing Toolbox™
Deep Learning Toolbox™
Computer Vision Toolbox™
MATLAB Compiler™
MATLAB Compiler SDK

Type matlabshared.supportpkg.getInstalled at the MATLAB command prompt to verify whether the following add-on is installed:

Computer Vision Toolbox Model for YOLO v4 Object Detection

If you need to install the add-on, click the Add-Ons icon in the MATLAB toolstrip and search for the add-on. You can also download and install it from the MathWorks File Exchange.

Prerequisites

Verify that you have MATLAB Compiler SDK installed on the development machine.
Verify that you have Docker installed and configured on the development machine by typing [~,msg] = system('docker version') in a MATLAB command window.

Note
If you are using WSL, use the command [~,msg] = system('wsl docker version') instead.

If you do not have Docker installed, follow the instructions on the Docker website to install and set up Docker.
docs.docker.com/engine/install/
- To build microservice images on Windows, you must install either Docker Desktop or Docker on Windows Subsystem for Linux v2 (WSL2). To install Docker Desktop, see docs.docker.com/desktop/install/windows-install//.
  For instructions on how to install Docker on WSL2, see https://www.mathworks.com/matlabcentral/answers/1758410-how-do-i-install-docker-on-wsl2.
If the computer you are using is not connected to the Internet, you must download the MATLAB Runtime installer for Linux from a computer that is connected to the Internet and transfer the installer to the computer that is not connected to the Internet. Then, on the offline machine, run the command compiler.runtime.createInstallerDockerImage(filepath), where filepath is the path to the MATLAB Runtime installer archive.
You can download the installer from the MathWorks^® website.
https://www.mathworks.com/products/compiler/matlab-runtime.html

Create MATLAB Function to Detect Objects

For this example, write an object detection function named cvt.m using the following code.

function [bboxes, scores, labels] = cvt(imageUrl)
iminfo = imfinfo(imageUrl);
    % Read image
    % If indexed image, read colormap and convert to rgb
    if strcmp(iminfo.ColorType,'indexed') == 1
        [im, cmap] = webread(imageUrl, 'Timeout', 10);
        im = ind2rgb(im, cmap);
    else
        im = webread(imageUrl, 'Timeout', 10);
    end
% Add  pretrained YOLO v4 dataset tinyYOLOv4COCO.mat to MATLAB path for testing
% Comment or remove the next 2 lines of code prior to deploying as microservice
detectorPath = [matlabshared.supportpkg.getSupportPackageRoot, '/toolbox/vision/supportpackages/yolov4/data'];
addpath(detectorPath)
load('tinyYOLOv4COCO.mat', 'detector');

% Detect objects in image using detector
[bboxes,scores,labels] = detect(detector,im);
labels = cellstr(labels);

Test the function from the MATLAB command line:

%% Specify image URL
imageUrl = "https://www.mathworks.com/help/examples/deeplearning_shared/win64/TrafficSignDetectionAndRecognitionExample_02.png"
%% Display image
imageFile = "trafficimage.jpg";
imageFileFullPath = websave(imageFile, imageUrl);
[im, cmap] = imread(imageFileFullPath);
imshow(im, cmap)
%% Detect objects in image
[bboxes, scores, labels] = cvt(imageUrl)

bboxes =
  2×4 single matrix
  445.3871  326.4009  223.3270   98.7086
  504.2861  271.4571   45.7471   41.0955
scores =
  2×1 single column vector
    0.9151
    0.6610
labels =
  2×1 cell array
    {'truck'    }
    {'stop sign'}

Create Deployable Archive

Caution

Comment the following lines of code in the cvt.m file prior to creating a deployable archive.

% detectorPath = [matlabshared.supportpkg.getSupportPackageRoot, '/toolbox/vision/supportpackages/yolov4/data'];
% addpath(detectorPath)

Package the cvt function into a deployable archive using the compiler.build.productionServerArchive function.

You can specify additional options in the compiler.build command by using name-value arguments. For details, see compiler.build.productionServerArchive.

buildResults = compiler.build.productionServerArchive('cvt.m', ...
    'ArchiveName','yolov4od','Verbose',true, ...
    'SupportPackages',{'Computer Vision Toolbox Model for YOLO v4 Object Detection'});

buildResults = 
  Results with properties:

                  BuildType: 'productionServerArchive'
                      Files: {'/home/mluser/work/yolov4odproductionServerArchive/yolov4od.ctf'}
    IncludedSupportPackages: {'Computer Vision Toolbox Model for YOLO v4 Object Detection'}
                    Options: [1×1 compiler.build.ProductionServerArchiveOptions]

The compiler.build.Results object buildResults contains information on the build type, generated files, included support packages, and build options.

Once the build is complete, the function creates a folder named yolov4odproductionServerArchive in your current directory to store the deployable archive.

Package Archive into Microservice Docker Image

Build the microservice Docker image using the buildResults object that you created.
You can specify additional options in the compiler.build command by using name-value arguments. For details, see compiler.package.microserviceDockerImage.
```
compiler.package.microserviceDockerImage(buildResults,...
    'ImageName','yolov4od-microservice',...
    'DockerContext',fullfile(pwd,'microserviceDockerContext'));
```
The function generates the following files within a folder named microserviceDockerContext in your current working directory:
- applicationFilesForMATLABCompiler/yolov4od.ctf — Deployable archive file.
- Dockerfile — Docker file that specifies Docker run-time options.
- GettingStarted.txt — Text file that contains deployment information.

Test Docker Image

In a system command window, verify that your yolov4od-microservice image is in your list of Docker images.

docker images

REPOSITORY                                      TAG           IMAGE ID            CREATED             SIZE
yolov4od-microservice                           latest        4401fa2bc057        33 seconds ago      7.56GB
matlabruntime/r2024b/update0/4200000000000000   latest        5259656e4a32        24 minutes ago      7.04GB

Run the yolov4od-microservice microservice image from the system command prompt.
```
docker run --rm -p 9900:9910 yolov4od-microservice -l trace &
```
Port 9910 is the default port exposed by the microservice within the Docker container. You can map it to any available port on your host machine. For this example, it is mapped to port 9900.
You can specify additional options in the Docker command. For a complete list of options, see Microservice Command Arguments.
Once the microservice container is running in Docker, you can check the status of the service by going to the following URL in a web browser:
```
http://hostname:9900/api/health
```
If the service is ready to receive requests, you see the following message:
```
"status:  ok"
```

Test the running service. In the terminal, use the curl command to send a JSON query with the input argument 4 to the service through port 9900. For more information on constructing JSON requests, see JSON Representation of MATLAB Data Types (MATLAB Production Server).

curl -v -H Content-Type:application/json \
-d '{"nargout":3,"rhs":["https://www.mathworks.com/help/examples/deeplearning_shared/win64/TrafficSignDetectionAndRecognitionExample_02.png"]}' \ 
"http://hostname:9900/yolov4od/cvt" | jq -c

The output is:

{"lhs":[{"mwdata":[445.387146,504.286102,326.40094,271.457092,223.327026,45.7471,98.7086487,41.09552],"mwsize":[2,4],"mwtype":"single"},
{"mwdata":[0.91510725,0.661022],"mwsize":[2,1],"mwtype":"single"},
{"mwdata":[{"mwdata":["truck"],"mwsize":[1,5],"mwtype":"char"},
{"mwdata":["stop sign"],"mwsize":[1,9],"mwtype":"char"}],"mwsize":[2,1],"mwtype":"cell"}]}

You can also test from the MATLAB desktop:

%% Import MATLAB HTTP interface packages
import matlab.net.*
import matlab.net.http.*
import matlab.net.http.fields.*

%% Setup message body
body = MessageBody;
body.Payload = ...
    '{"nargout": 3,"rhs": ["https://www.mathworks.com/help/examples/deeplearning_shared/win64/TrafficSignDetectionAndRecognitionExample_02.png"]}';

%% Setup request
requestUri = URI('http://hostname:9900/yolov4od/cvt');
options = matlab.net.http.HTTPOptions('ConnectTimeout',20,...
    'ConvertResponse',false);
request = RequestMessage;
request.Header = HeaderField('Content-Type','application/json');
request.Method = 'POST';
request.Body = body;

%% Send request & view raw response
response = request.send(requestUri, options);
disp(response.Body.Data)

%% Decode JSON
lhs = mps.json.decoderesponse(response.Body.Data);

%% Clean up printed output
for i = 1:length(lhs)
    [r,c] = size(lhs{i});
    if ~iscell(lhs{i}) && c==1
        tmp(:,i) = num2cell(lhs{i});
    elseif ~iscell(lhs{i}) && c~=1
        tmp(:,i) = num2cell(lhs{i},2);
    else
        tmp(:,i) = lhs{i};
    end
end
%% Display response as a table
T = cell2table(tmp,'VariableNames',{'Boxes', 'Scores', 'Labels'})

The output is:

T =

  2×3 table

                   Boxes                    Scores        Labels    
    ____________________________________    _______    _____________

    445.39     326.4    223.33    98.709    0.91511    {'truck'    }
    504.29    271.46    45.747    41.096    0.66102    {'stop sign'}

To stop the service, use the following command to display the container id.

docker ps

CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS              PORTS                    NAMES
f372b8b574e8        yolov4od-microservice   "/opt/matlabruntime/…"   6 hours ago         Up 6 hours          0.0.0.0:9900->9910/tcp   distracted_panini

Stop the service using the specified container id.

docker stop f372b8b574e8

Share Docker Image

You can share your Docker image in various ways.

Push your image to the Docker central registry Docker Hub, or to your private registry. This is the most common workflow.
Save your image as a tar archive and share it with others. This workflow is suitable for immediate testing.

For details about pushing your image to Docker Hub or your private registry, consult the Docker documentation.

Save Docker Image as Tar Archive

To save your Docker image as a tar archive, open a system command window, navigate to the Docker context folder, and type the following.

docker save yolov4od-microservice -o yolov4od-microservice.tar

This command creates a file named yolov4od-microservice.tar in the current folder. Set the appropriate permissions (for example, using chmod) prior to sharing the tarball with other users.

Load Docker Image from Tar Archive

Load the image contained in the tarball on the end user machine.

docker load --input yolov4od-microservice.tar

Verify that the image is loaded.

docker images

Run Docker Image

docker run --rm -p 9900:9910 yolov4od-microservice