Main Content

segmentAnythingAerialLidar

Perform semantic segmentation of aerial lidar data using pretrained Segment Anything Model (SAM)

Since R2024b

expand all in page

    Description

    The segmentAnythingAerialLidar object creates a Segment Anything Model (SAM) to segment objects in an aerial lidar point cloud using visual prompts.

    The segmentAnythingAerialLidar object enables you to semantically segment objects in aerial lidar point cloud data without retraining the model. The object builds upon the segmentAnythingModel object, which configures a pretrained SAM for image segmentation.

    After creating the segmentAnythingAerialLidar object, follow these steps to segment objects in the point cloud data:

    1. Extract feature embeddings of the point cloud data by using the extractEmbeddings object function. The function converts the point cloud data into a bird’s-eye-view (BEV) image, and then extracts the feature embeddings of the BEV image using the SAM encoder.

    2. Segment objects by using the segmentObjectsFromEmbeddings object function. The function first segments objects from the extracted feature embeddings of the BEV image using the SAM decoder. Then, the function converts the generated 2-D object masks of the BEV image to 3-D object masks for the point cloud data.

    For information on using SAM for image segmentation, see Get Started with Segment Anything Model for Image Segmentation.

    Creation

    Syntax

    sam = segmentAnythingAerialLidar

    Description

    sam = segmentAnythingAerialLidar creates a pretrained Segment Anything Model. Use this model to segment objects in aerial lidar point clouds using visual prompts.

    Note

    This functionality requires Deep Learning Toolbox™ and the Image Processing Toolbox™ Model for Segment Anything Model support package. You can download and install the Image Processing Toolbox Model for Segment Anything Model from Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons.

    example

    Object Functions

    extractEmbeddingsExtract feature embeddings of lidar point cloud from Segment Anything Model (SAM) encoder
    segmentObjectsFromEmbeddingsSegment objects in lidar point cloud using Segment Anything Model (SAM) feature embeddings

    Examples

    collapse all

    This example uses:

    Open Live Script

    Specify a full file path for a LAS file that contains aerial lidar data. Then, read the point cloud data from the file using the readPointCloud function of the lasFileReader object.

    filename = fullfile(matlabroot,"toolbox","lidar", ...
         "lidardata","las","aerialLidarData2.las");
lasReader = lasFileReader(filename);
ptCloud = readPointCloud(lasReader);

    Remove the ground plane from the point cloud to get better segmentation results. Display the resulting point cloud.

    [~,nonGroundPtCloud] = segmentGroundSMRF(ptCloud);
figure
pcshow(nonGroundPtCloud)

    Create a Segment Anything Model object for aerial lidar point cloud segmentation.

    samLidar = segmentAnythingAerialLidar;

    Extract the feature embeddings from the point cloud.

    embeddings = extractEmbeddings(samLidar,nonGroundPtCloud);

    Specify a bounding box that contains an object to segment, and display it in green.

    boxPrompt = [429321 3680081 79.89 14 7 3 0 0 0];
showShape("cuboid",boxPrompt,Color="green")

    Segment the object using the segmentObjectsFromEmbeddings function, which runs the SAM decoder on the feature embeddings.

    mask = segmentObjectsFromEmbeddings(samLidar, ...
    embeddings,nonGroundPtCloud,BoundingBox=boxPrompt);

    Visualize the segmentation mask overlaid on the point cloud.

    figure
pcshow(nonGroundPtCloud.Location,single(mask))

    Version History

    Introduced in R2024b

    See Also

    | |

    Topics