Skip to content

Conversation

@anwai98
Copy link
Collaborator

@anwai98 anwai98 commented Nov 6, 2025

This PR adds preliminary (and super dirty) integration of SAM2 to our annotator.

As the current status:

  • 2d works super smooth (all features from our current UI more or less) - and it's ready to test.
  • For 3d, I have a few tiny puzzles to join, then it'll be ready. I'll ping @Marei33 once there's more progress here.

Also, keeping @constantinpape and @caroteu in loop!

@anwai98 anwai98 self-assigned this Nov 6, 2025
@anwai98 anwai98 added the enhancement New feature or request label Nov 6, 2025
@anwai98
Copy link
Collaborator Author

anwai98 commented Jan 5, 2026

Additional TODOs spotted:

  • Remove "s" keybinding for the 3d annotator tool

EDIT: Naaaah, we are trying to bring single slice segmentation back! ;)

@anwai98
Copy link
Collaborator Author

anwai98 commented Jan 5, 2026

Another comment: I figured out the error for 3d annotator (finally) I'll debug it quickly and try to fix it!

@anwai98
Copy link
Collaborator Author

anwai98 commented Jan 5, 2026

It's fixed now! @constantinpape

Can you give it a shot? (by pulling the latest state here and https://github.com/computational-cell-analytics/SAM2_Experiments/pull/43)

@constantinpape
Copy link
Contributor

I gave it a try and it seems to work better now but is still not very stable.

What I tried:

  • Add a single point and segment in 3D via Segment All Slices
  • This worked (but takes quite long on CPU). The result is not great, but ok enough.
  • Add a second negative point.
  • It fails, see the traceback below.
ValueError                                Traceback (most recent call last)
File ~/Work/my_projects/micro-sam/micro_sam/sam_annotator/_widgets.py:1737, in SegmentNDWidget.__call__(self=<micro_sam.sam_annotator._widgets.SegmentNDWidget object>)
   1735     return self._run_tracking()
   1736 else:
-> 1737     return self._run_volumetric_segmentation()
        self = <micro_sam.sam_annotator._widgets.SegmentNDWidget object at 0x7fca95b12570>
File ~/Work/my_projects/micro-sam/micro_sam/sam_annotator/_widgets.py:1720, in SegmentNDWidget._run_volumetric_segmentation(self=<micro_sam.sam_annotator._widgets.SegmentNDWidget object>)
   1717     self._viewer.layers["current_object"].data = seg
   1718     self._viewer.layers["current_object"].refresh()
-> 1720 seg = volumetric_segmentation_impl()
   1721 self._viewer.layers["current_object"].data = seg
   1722 self._viewer.layers["current_object"].refresh()

File ~/Work/my_projects/micro-sam/micro_sam/sam_annotator/_widgets.py:1690, in SegmentNDWidget._run_volumetric_segmentation.<locals>.volumetric_segmentation_impl()
   1685 z_values_boxes = np.concatenate(
   1686     [box[:1, 0] for box in box_prompts.data]
   1687 ) if box_prompts.data else np.zeros(0, dtype="int")
   1689 # TODO: Make this modular.
-> 1690 if z_values:
        z_values = array([7., 7.])   1691     frame_id = z_values[0]
   1692 else:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

@constantinpape
Copy link
Contributor

I also tried with a box prompt first -- the result looked much better, but trying to add more prompts fails with a similar error as before.

anwai98 and others added 4 commits January 7, 2026 18:44
* Test claude code by implementing 'Segment Slice' for SAM2 in the 3D annotator (not yet checked)

* Refactor per-slice segmentation into PromptableSegmentation3D class

Move SAM2 single-slice segmentation logic from _widgets.py into the PromptableSegmentation3D class for better encapsulation and reusability. This consolidates all SAM2 segmentation functionality in the v2 module.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Refactor SAM2 inference state initialization to use volume directly

Replace disk-based frame initialization with direct volume passing in AnnotatorState. This aligns with PromptableSegmentation3D implementation and removes unnecessary disk I/O overhead.

Changes:
- Remove _sam2_temp_dir attribute
- Simplify _initialize_sam2_inference_state to pass volume directly
- Remove temporary file cleanup from clear_inference_state

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Simplify and add notes

* Clean up

* Fix axis order

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants