Instance Segmentation
R2026bInstance Segmentation tools in Computer Vision Toolbox™ enable you to detect, classify, and segment individual objects within an image, even when multiple objects are overlapping. You can start by creating labeled ground truth using the Image Labeler and Video Labeler apps, which support interactive and AI-assisted annotation of object instances with polygons or rectangle ROIs. For more information, see Label Objects Using Polygons for Instance Segmentation.
The toolbox provides pretrained instance segmentation networks such as SOLOv2
and Mask R-CNN. You can use these models directly for inference or adapt them to
specific applications through transfer learning. For more information, see Get Started with Instance Segmentation Using Deep Learning and Get Started with SOLOv2 for Instance Segmentation. For class
agnostic instance segmentation, the toolbox supports the Segment Anything Model
(SAM) through the imsegsam function and the segmentAnythingModel object. For video object segmentation, the
toolbox provides sam2VideoObjectSegmenter object that uses SAM 2 to segment and
track objects consistently across video frames using point or bounding box
prompts.
To prepare training data, the toolbox offers utilities for managing and
organizing data sets along with data augmentation and preprocessing. For more
information, see instanceSegmentationTrainingData and Create Instance Segmentation Training Data from Labeled Ground Truth.
After you generate predictions using pretrained or custom models, you can
evaluate instance segmentation performance and generate detailed insights into
segmentation accuracy, object-level precision, and performance across different
object sizes. These metrics help assess the quality of both mask predictions and
bounding box localization. For more information, see evaluateInstanceSegmentation.
The toolbox also supports 3-D object pose estimation using instance segmentation through the Pose Mask R-CNN framework, enabling fine-grained analysis of object orientation and structure. For more information, see Perform 6-DoF Pose Estimation for Bin Picking Using Deep Learning.
Apps
| Image Labeler | Label images for computer vision applications |
| Video Labeler | Label video for computer vision applications |
Functions
Topics
Get Started
- Get Started with Instance Segmentation Using Deep Learning
Segment objects using an instance segmentation model such as SOLOv2 or Mask R-CNN. - Get Started with SOLOv2 for Instance Segmentation
Perform multiclass instance segmentation using SOLOv2 and deep learning. - Getting Started with Mask R-CNN for Instance Segmentation
Perform multiclass instance segmentation using Mask R-CNN and deep learning. - Get Started with Segment Anything Model for Image Segmentation
Perform interactive image segmentation using Segment Anything Model 2 (SAM 2) and deep learning.
Create Ground Truth for Instance Segmentation
- Label Objects Using Polygons for Instance Segmentation
Label ground truth objects using polygons for instance segmentation.
Prepare Training Data for Instance Segmentation
- Create Instance Segmentation Training Data from Labeled Ground Truth
Create training data for Mask R-CNN or Solov2 instance segmentation networks from polygon-labeled ground truth using theinstanceSegmentationTrainingDatafunction. - Get Started with Image Preprocessing and Augmentation for Deep Learning
Preprocess data for deep learning applications with deterministic operations such as resizing, or augment training data with randomized operations such as random cropping. - Datastores for Deep Learning (Deep Learning Toolbox)
Learn how to use datastores in deep learning applications.
Featured Examples
Automate Ground Truth Polygon Labeling Using Grounded SAM Model
Combine Grounding DINO and the Segment Anything Model 2 (SAM 2) to automatically produce polygon labels using the Video Labeler app.
Automate Ground Truth Labeling for Instance Segmentation
Create an automation algorithm to automatically label data for instance segmentation using a pretrained SOLOv2 network in the Video Labeler app.
Automatically Search and Label Video Frames Using VLMs
Automatically search and detect objects based on natural language text queries using vision-language models (VLMs).
Perform Instance Segmentation Using SOLOv2
Segment object instances of randomly rotated machine parts in a bin using a deep learning SOLOv2 network.
Perform Instance Segmentation Using Mask R-CNN
Segment individual instances of people and cars using a multiclass mask region-based convolutional neural network (R-CNN).
Automatically Label Ground Truth Using Segment Anything Model
Label pixels using the Segment Anything Model (SAM) in the Image Labeler app.
Segment Objects in Interactive ROI Using Segment Anything Model
Perform interactive segmentation of an object in a selected region of interest (ROI) of an image using the Segment Anything Model (SAM).
Perform 6-DoF Pose Estimation for Bin Picking Using Deep Learning
Perform six degrees-of-freedom (6-DoF) pose estimation by estimating the 3-D position and orientation of machine parts in a bin using RGB-D images and a deep learning network.
