Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
cbb334f
feat: Add support/config for InstanceSegmentationMask
aranega Aug 15, 2025
c4aa044
chore: Add new generated code from the new config for
aranega Aug 15, 2025
c4527da
feat: add instance segmentation mask class and instance creation
aranega Sep 4, 2025
2bb7506
chore: remove debug/test code in configuration 10441.yaml
aranega Sep 4, 2025
4a3d320
Merge branch 'main' of github.com:chanzuckerberg/cryoet-data-portal-b…
aranega Sep 12, 2025
bdf0085
feat: remove the instancesegmentation mask from v1, and enhance it for
aranega Sep 12, 2025
5d4bb52
Merge branch 'main' of github.com:chanzuckerberg/cryoet-data-portal-b…
aranega Sep 17, 2025
598e594
feat: add first steps for the InstanceSegmentationMask ingestion
aranega Oct 1, 2025
00a379d
fix: remove bad guard and simplify it
aranega Oct 1, 2025
6dec931
feat: add dedicated shape for the InstanceSegmentationMask
aranega Oct 2, 2025
5d6ac85
fix: remove labels param to convert function of instance seg masks
seankmartin Oct 2, 2025
6101bb2
feat: allow passing seg list into gen and v1 of instance seg mask config
seankmartin Oct 2, 2025
129fde5
feat: Add configuration generation for InstanceSegmentationMask
aranega Oct 7, 2025
0cf3ffe
feat: Remove mask_label for InstanceSegmentationMask
aranega Oct 7, 2025
7eccd55
feat: add label computation and storing before configuration generation
aranega Oct 8, 2025
0968ac4
Merge branch 'main' of github.com:chanzuckerberg/cryoet-data-portal-b…
aranega Oct 17, 2025
a438369
feat: clean schema attribute for the InstanceSegmentationMask
aranega Oct 27, 2025
ac1f46e
feat: add tests for the instance segmentation mask
aranega Oct 27, 2025
0af2440
chore: Updating schema to add InstanceSegmentationMask
aranega Oct 27, 2025
23cd1c5
fix: change is_a property of InstanceSegmentationMask in metadata.yaml
aranega Oct 27, 2025
2233c94
chore: update portal-neuroglancer to v1.7.0
aranega Oct 27, 2025
6f024df
feat: change metadata storing for labels in image-label
aranega Oct 27, 2025
8ec69d7
feat: add tests for instance segmentation mask db ingestion
aranega Oct 27, 2025
6f48941
feat: add a different method for downscaling multi-label segmentation
aranega Oct 28, 2025
6702b85
chore: Add back missing migration script
aranega Oct 28, 2025
37005bb
chore: Update schema to add InstanceSegmentationMask shape
aranega Oct 28, 2025
6bc118c
chore: Fix comment and poetry version
aranega Oct 28, 2025
72a0379
feat: change the container for the image-label metadata
aranega Oct 28, 2025
9ce3ceb
chore: apply old formatter for the migration
aranega Oct 29, 2025
4ac545e
chore: remove InstanceSegmentationMask from v1 schema
aranega Oct 29, 2025
bf96a29
chore: remove lefovers for v1 for InstanceSegmenationMask
aranega Oct 29, 2025
a15f3e6
chore: add back AnnotationInstanceSegmentationFile in metadata.yaml f…
aranega Oct 31, 2025
b31d94c
Merge branch 'main' of github.com:chanzuckerberg/cryoet-data-portal-b…
aranega Oct 31, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion apiv2/db_import/tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,5 +133,5 @@ def expected_dataset(http_prefix: str) -> dict[str, Any]:
"key_photo_url": f"{http_prefix}/{DATASET_ID}/KeyPhoto/snapshot.png",
"key_photo_thumbnail_url": f"{http_prefix}/{DATASET_ID}/KeyPhoto/thumbnail.png",
"deposition_id": 300,
"file_size": 1374354.0,
"file_size": 1374808.0,
}
18 changes: 18 additions & 0 deletions apiv2/db_import/tests/test_db_annotation_import.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,24 @@ def expected_annotations(http_prefix: str) -> list[dict[str, Any]]:
def expected_annotation_files(http_prefix: str) -> list[dict[str, Any]]:
path = f"{DATASET_ID}/RUN1/Reconstructions/VoxelSpacing12.300/Annotations/"
return [
{
"tomogram_voxel_spacing_id": TOMOGRAM_VOXEL_ID1,
"s3_path": f"s3://test-public-bucket/{path}100-foo-1.0_instancesegmask.mrc",
"https_path": f"{http_prefix}/{path}100-foo-1.0_instancesegmask.mrc",
"source": "community",
"format": "mrc",
"is_visualization_default": False,
"file_size": 0,
},
{
"tomogram_voxel_spacing_id": TOMOGRAM_VOXEL_ID1,
"s3_path": f"s3://test-public-bucket/{path}100-foo-1.0_instancesegmask.zarr",
"https_path": f"{http_prefix}/{path}100-foo-1.0_instancesegmask.zarr",
"source": "community",
"format": "zarr",
"is_visualization_default": False,
"file_size": 0,
},
{
"id": ANNOTATION_FILE_ID,
"tomogram_voxel_spacing_id": TOMOGRAM_VOXEL_ID1,
Expand Down
7 changes: 4 additions & 3 deletions apiv2/graphql_api/schema.graphql

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions apiv2/graphql_api/schema.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions apiv2/graphql_api/types/annotation_shape.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 5 additions & 2 deletions apiv2/schema/schema.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,9 @@ enums:
InstanceSegmentation:
text: InstanceSegmentation
description: A volume with labels for multiple instances
InstanceSegmentationMask:
text: InstanceSegmentationMask
description: A mask with labels for multiple instances
Mesh:
text: Mesh
description: A surface mesh volumes
Expand Down Expand Up @@ -922,11 +925,11 @@ classes:
annotations:
cascade_delete: true
shape_type:
description: The shape of the annotation (SegmentationMask, OrientedPoint, Point, InstanceSegmentation, Mesh)
description: The shape of the annotation (SegmentationMask, OrientedPoint, Point, InstanceSegmentation, Mesh, InstanceSegmentationMask)
name: shape_type
from_schema: cdp-dataset-config
range: annotation_file_shape_type_enum
pattern: (^SegmentationMask$)|(^OrientedPoint$)|(^Point$)|(^InstanceSegmentation$)|(^Mesh$)
pattern: (^SegmentationMask$)|(^OrientedPoint$)|(^Point$)|(^InstanceSegmentation$)|(^Mesh$)|(^InstanceSegmentationMask$)
Annotation:
name: Annotation
annotations:
Expand Down
1 change: 1 addition & 0 deletions apiv2/support/enums.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion apiv2/test_infra/factories/annotation_shape.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions ingestion_tools/dataset_configs/template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,15 @@ annotations: OPTIONAL
delimiter: OPTIONAL, STRING (DEFAULT ',')
parent_filters: see InstanceSegmentation.parent_filters
exclude: SEE InstanceSegmentation.exclude
- InstanceSegmentationMask:
file_format: see InstanceSegmentation.file_format
glob_string: see InstanceSegmentation.glob_string
glob_strings: see InstanceSegmentation.glob_strings
is_visualization_default: see InstanceSegmentation.is_visualization_default
is_portal_standard: OPTIONAL, BOOLEAN (DEFAULT FALSE)
rescale: OPTIONAL, BOOLEAN (DEFAULT FALSE)
parent_filters: see InstanceSegmentation.parent_filters
exclude: SEE InstanceSegmentation.exclude
- SegmentationMask:
file_format: see InstanceSegmentation.file_format
glob_string: see InstanceSegmentation.glob_string
Expand Down
Loading