Work with Blocked Images Using Volume Segmenter

This example shows how to work with a single-resolution blocked image in the Volume Segmenter app.

Use blocked images when the original volume is too large to fit into memory. By using blocked images, you can segment volumes without running out of memory.

To use the Volume Segmenter app with a blocked image, you must create a blocked image from the original volume and open the blocked image in the app. Once in the app, working with the blocked image is very similar to working with any volume.

• Explore the blocked image just as you would any volume, by viewing each slice individually or manipulating the 3-D representation of the volume. However, with a blocked image, you view the volume one block at a time. The app includes navigation aids you can use to view each block in the blocked image.

• Segment the blocked image just as you would any volume, drawing labels on areas of the volume. However, with a blocked image, you draw labels on the volume a block at a time. To label the blocked image, use the drawing tools in the app to create ROIs. You can also use interpolation to automatically label intermediate slices in a block. As you view each block, you segment the part of the object you find in that block. You can also use automated methods to segment a blocked image. When using automation, you can process all blocks at the same time..

When working with blocked images in the Volume Segmenter app, create all the labels you want to use and then save the segmentation. This is more efficient than adding or removing labels individually. Also, as you finish processing a block, before you begin processing the next block, you must save the processed block in a file. When you are done, the blockedImage object combines the individually processed block files into one volume

Create Blocked Image

If you want to segment a volume that does not fit into memory, create a blockedImage object to represent the volume. This example uses a stack of MRI brain images as a volume, stored in the MAT-file vol_001.mat. The MRI data is a modified subset of the BraTS data set [1]. In this MRI data, you can see the tumor that you want to segment in the temporal lobe.

load(fullfile(toolboxdir('images'),'imdata','BrainMRILabeled','images','vol_001.mat'));

Reading the file loads a 240-by-240-by-155 volume named vol into the workspace.

whos vol

  Name        Size                    Bytes  Class     Attributes

  vol       240x240x155            17856000  uint16              

Create a blocked image from the volume, specifying the size of the blocks. (If you have a volume that does not fit in memory, you can specify the file name to blockedImage.)

bim = blockedImage(vol,'BlockSize',[120 120 120])

bim = 
  blockedImage with properties:

   Read only properties
             Source: [240x240x155 uint16]
            Adapter: [1x1 images.blocked.InMemory]
               Size: [240 240 155]
       SizeInBlocks: [2 2 2]
    ClassUnderlying: "uint16"

   Settable properties
          BlockSize: [120 120 120]

Given the specified block size, the blocked image creates two blocks in each dimension.

Open Volume Segmenter

Open the Volume Segmenter app. Select the Apps tab on the MATLAB® toolstrip. In the Image Processing and Computer Vision section, select Volume Segmenter.

Load the Blocked Image into the Volume Segmenter

To load the blocked image into the Volume Segmenter app, select Open Volume on the app toolstrip. For this example, select Open Blocked Image from Workspace. In the Import Volume dialog box, select the blocked image you created in the workspace, bim, and click OK. Alternatively, you can specify a blocked image when you open the app by using the volumeSegmenter command: volumeSegmenter(bim).

The app loads the volume and displays its content. When working with a blocked image, the app displays the contents of one block at a time. The Overview tab indicates which block you are currently viewing in the context of the entire volume.

Explore the Blocked Image

Using the Volume Segmenter app, explore the volume to determine what you want to segment. With a blocked image, the app includes several navigational aids that help explore each block.

Current Block -- View a 3-D representation of the block contents in the Current Block tab. To add orientation axes and a wireframe to the display, go to the 3-D Display tab in the app toolstrip. To view the block from different angles, use the mouse to rotate the display.

Overview -- Shows the location of the current block in relation to the other blocks in the blocked image. To add orientation axes and a wireframe to the display, use the options on the 3-D Display tab of the app toolstrip. To view the block from different angles, use the mouse to rotate the display. As you explore blocks, the display updates to show which block you currently have selected, as well as which you have visited and which you have marked as done. The current block is shown in red. Visited blocks or processed blocks are yellow. Blocks that you mark as done are green.

You can also customize the display of the volume in the 3-D Display tab in the app toolstrip. For example, if you have metadata that describes the relative size of the voxels, you can specify it in the Spatial Referencing part of the 3-D Display tab. To improve your view of the data, you can change the background color used in the 3-D display, modify the threshold and opacity of the display, and include orientation axes with the display.

Blocked Image tab -- For blocked images, the app adds a Blocked Image tab to the app toolstrip. This tab contains navigation aids that help you move among the blocks in the blocked image. For example, to move to the next unprocessed block, click Next Block. You can also move to a particular block by specifying block coordinates along the X-, Y-, and Z-axes. To indicate that you are done processing a block, click Mark Block Complete. When you mark a block complete, the app calculates the percentage of doneness for the entire volume.

Slice pane -- View each slice of the volume in the Slice pane. Use the slider at the bottom of the tab to move from slice to slice. By default, the Slice pane displays the volume oriented along the X-Y axis, but you can change this using buttons in the Orientation section of the toolstrip on the Segmenter tab. The Slice pane is also where you use drawing tools to define the ROIs. With blocked images, the view of the slice shows only the current block. The object you want to segment may span several blocks. The app displays the number of the current slice, out of the total number of slices, at the top of the pane. For example, 50/120..

Use Drawing Tools to Label Regions in Blocked Images

Once you have identified the object that you want to segment, use the tools on the Draw tab in the app toolstrip to label the object in each block where it appears. You can use any of drawing tools with blocked images: the Paint Brush tool, the Fill Region tool, the Eraser tool, and the Freehand, Assisted Freehand, and Polygon region-of-interest (ROI) shapes.

As with any volume, to start labeling the brain, first create all the labels you want to use in the segmentation. In the Labels pane, the app provides one label by default, named Label1. To change the name of the label to be more descriptive for your application, double-click the label and type in the new name. To change the default color associated with the label, double-click the colored square associated with the label and select a color from the Color dialog box. When one object is nested in another object, as the tumor appears over the brain on slices, label the larger region first. Click the plus button to create additional labels.

In the Slice pane, navigate to a slice where the object appears in the block and use a drawing tool to label the object. This figure shows the Paint Brush tool, but you can use any of the drawing tools.

Use Interpolation to Speed Object ROI Creation

You could move through a block, slice-by-slice, and draw an ROI on each slice where the object appears. However, the Volume Segmenter app provides several automated tools that can help with segmenting an object across slices. These automated options process only the slices within a block.

To use interpolation to speed up labeling, you must first manually label the region on two slices. For example, create a label on one slice and use the same process to define the label on another slice. The app places two bars on the slider, using the color associated with the label, to indicate the slices with defined ROIs.

With the object defined on two slices, click Auto Interpolate. The app automatically defines the ROI on all the intervening slides. The app uses a solid blue bar to indicate that all the slices have ROIs.

Alternatively, after defining an ROI on two slices, click Manually Interpolate. With this option, the app opens the Manually Interpolate dialog box. You select the two regions from which you want to interpolate, Region One and Region Two. By default, the dialog box opens on a slice on which you have defined a region. To select the first region, click Region One. Navigate to the other slice on which you have defined a region, using the slider or by clicking the blue indicator above the slider. To select the second region, click Region Two. After selecting both regions, click Run to interpolate the ROI on all intervening slices.

Use Automation to Refine Labels and Perform Custom Processing

You can use an algorithm to refine label definitions and perform other processing of blocked images automatically. The app includes several slice-based and volume-based algorithms on the Automate tab. First, select the algorithm. For example, select the volume-based algorithm Otsu's Threshold in the Algorithm section of the Automate tab toolstrip. Once you select the algorithm, select Algorithm Parameters to specify values for any algorithm-specific parameters that might be associated with the algorithm. Because Otsu's threshold algorithm does not support any parameters, this option is not enabled. For slice-based algorithms, you can specify which slices you want to process: the current slice, a set of slices from the current slice back to the beginning or from the current slice to the end. After selecting the algorithm, specifying algorithm-specific parameters, if available, and choosing the slides to operate on, click Run.

Process All Blocks and Review Results

When working with blocked images, you have several other options for automated processing. For blocked images, by default, automation algorithms operate on the slices in the current block. However, to perform automated processing on all the blocks in the blocked image at one time, click Automate On All Blocks. If you have already marked some blocks completed, make sure Skip Completed is not enabled. To enable parallel processing of blocks, click Use Parallel.

To review the results of the processing and accept or reject each block, click Review Results to select this option before clicking Run. The app displays the Review and accept automation results dialog box. Select the check box for each block you accept and click Accept Selected to finish.

Add Custom Automation Algorithms

You can also add your own algorithm to operate on the ROIs. On the Automate tab, click Add Algorithm. Choose whether you want your processing to operate on each 2-D slice (Slice-based) or on the entire 3-D volume (Volume-based).

For this example, under Slice-Based, select the New option and click Function Template to create a new function that operates on each 2-D slice. The app opens the template in the MATLAB editor. Replace the sample code in the template with code that you want to use. Your function must accept two arguments: each slice as a separate image and a mask. Your function must also return a mask image.

When you are done editing the template, save the file. The Volume Segmenter app automatically creates a button in the Automate tab toolstrip for your function. To test your function on one slice, click Run. By default, the app applies the function to only the current slice.

Save the Segmentation

When you complete labeling the brain and the tumor in the volume, save the segmentation. Save the labels to a new empty folder by selecting Save Labels > Save As from the Segmenter tab. Image Segmenter saves each block of labels as a separate H5 image file.

If you continue to modify the labels in the app, then you can overwrite the old label files by selecting Save Labels > Save. After you save the segmentation, you can optionally turn on Autosave, which periodically saves the segmentation automatically.

View the Labeled Volume

To view the mask, use the bigimageshow function.

References

[1] Medical Segmentation Decathlon. "Brain Tumours." Tasks. Accessed May 10, 2018. http://medicaldecathlon.com/.

The BraTS data set is provided by Medical Segmentation Decathlon under the CC-BY-SA 4.0 license. All warranties and representations are disclaimed. See the license for details. MathWorks® has modified the subset of data used in this example. This example uses the MRI data of one scan from the original data set, saved to a MAT file.