Feature Request: Fine-tuning

[austinmw]

Description

I know this is a big feature request, but the addition of fine-tuning support would be nice as a component of this package.

TorchGeo could probably be used nicely for sampling from large images when training

[giswqs]

For now, we can only fine-tune what SAM allows. See the parameters below. It might take sometime before we can support other model fine-tuning. Contributions are always welcome.

https://github.com/opengeos/segment-geospatial/blob/main/samgeo/samgeo.py#L43

  points_per_side: Optional[int] = 32,
  points_per_batch: int = 64,
  pred_iou_thresh: float = 0.88,
  stability_score_thresh: float = 0.95,
  stability_score_offset: float = 1.0,
  box_nms_thresh: float = 0.7,
  crop_n_layers: int = 0,
  crop_nms_thresh: float = 0.7,
  crop_overlap_ratio: float = 512 / 1500,
  crop_n_points_downscale_factor: int = 1,
  point_grids: Optional[List[np.ndarray]] = None,
  min_mask_region_area: int = 0,
  output_mode: str = "binary_mask",

[giswqs]

https://github.com/bowang-lab/MedSAM

[giswqs]

Additional resources for fine-tuning

• https://encord.com/blog/learn-how-to-fine-tune-the-segment-anything-model-sam/
• https://github.com/ziqi-jin/finetune-anything
• https://github.com/luca-medeiros/lightning-sam
• https://github.com/tianrun-chen/SAM-Adapter-PyTorch
• Finetuning  facebookresearch/segment-anything#5
• https://colab.research.google.com/drive/1F6uRommb3GswcRlPZWpkAQRMVNdVH7Ww?usp=sharing
• https://github.com/ArminMoghimi/Fine-tune-the-Segment-Anything-Model-SAM-

[fly1989]

so cool，bro！

[JoyLinWQ]

Hi @giswqs , any updates on how we can fine-tune on new custom classes for GroundingDINO, or maybe replacing GroundingDINO with other pre-trained models that produces bboxes for downstream SAM? Thanks in advance!

[massidi]

Is it possible to fine tune sam_geo with our own pertained model I have a dataset in form of yolov8 annotations can I use it with sam geospatial ?

[izdrail]

I guess the code from here
https://github.com/bowang-lab/MedSAM/blob/main/train_one_gpu.py
could be addapted to train.

[giswqs]

SAM 3 fine-tuning: https://blog.roboflow.com/fine-tune-sam3/