Can we fine tune Segmentation Anything Model (SAM) in MATLAB?

I want to fine tune SAM using our own datasets. I know we can do it in Python. But is there any way to allow me to fine tune it in Matlab? Thanks,

[Sam Chak]

Hi @Bo

Using this "pears" image, could you describe how you intend to use SAM and the aspects of the model that you wish to fine-tune flexibly?

img = imread("pears.png");
imshow(img)

[Bo]

Thanks, Sam! I will collect our new dataset with segementation masks, e.g., medical images. Can we in Matlab finetune the existing SAM model by adjusting its model weights? Currently, Matlab only provides SAM as a class object with two functions. Is it possible to allow Matlab users to open it up and fine tune it like other deep neural networks in Matlab?

[Sam Chak]

Hi @Bo

Since a MathWorks staff member has clarified this matter, you could submit enhancement suggestions on the Product Usage page regarding the aspects or features you would like to see in the Segment Anything Model (SAM).

Answer 1

[Birju Patel]

As of R2025b, finetuning SAM is not supported. This will be considered for future releases.

You mentioned you were interested in medical images. Try out MedSAM in the Medical Imaging Toolbox as it may work on your images out-of-the-box:

https://www.mathworks.com/help/medical-imaging/ref/medicalsegmentanythingmodel.html

[Bo]

Yes, we have tried MedSAM. Thanks for the clarification. Hopefully the funetuning SAM will be available soon. Thanks.