You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
2. Create a conda environment with Python 3.9.7 version and install conda requirements: `conda env create -f environment.yml`. You can change the environment name by editing the `name` tag in the environment.yaml file.
73
+
2. Create a conda environment with Python 3.10.12 version and install conda requirements: `conda env create -f environment.yml`. You can change the environment name by editing the `name` tag in the environment.yaml file.
74
74
This step is necessary, as we need to install `Openslide` with binary files. This is easier with conda. Otherwise, installation from [source](https://openslide.org/api/python/) needs to be performed and packages installed with pi
4. Install torch for for system, as described [here](https://pytorch.org/get-started/locally/). Preferred version is 1.13, see [optional_dependencies](./optional_dependencies.txt) for help. You can find all version here: https://pytorch.org/get-started/previous-versions/
77
-
Example for CUDA 11.7: `pip install torch==1.13.0+cu117 torchvision==0.14.0+cu117 torchaudio==0.13.0 --extra-index-url https://download.pytorch.org/whl/cu117`
76
+
4. Install torch (>=2.0) for your system, as described [here](https://pytorch.org/get-started/locally/). Preferred version is 2.0, see [optional_dependencies](./optional_dependencies.txt) for help. You can find all version here: https://pytorch.org/get-started/previous-versions/
78
77
79
78
5. Install optional dependencies `pip install -r optional_dependencies.txt` to get a speedup using [NVIDIA-Clara](https://www.nvidia.com/de-de/clara/) and [CuCIM](https://github.com/rapidsai/cucim) for preprocessing during inference. Please select your CUDA versions. Help for installing cucim can be found [online](https://github.com/rapidsai/cucim).
80
79
**Note Error: cannot import name CuImage from cucim**
Copy file name to clipboardExpand all lines: docs/readmes/preprocessing.md
+21-4
Original file line number
Diff line number
Diff line change
@@ -1,18 +1,21 @@
1
1
# Preprocessing
2
2
3
+
In our Pre-Processing pipeline, we are able to extract quadratic patches from detected tissue areas, load annotation files (`.json`) and apply color normlizations. We make use of the popular [OpenSlide](https://openslide.org/) library, but extended it with the [RAPIDS cuCIM](https://github.com/rapidsai/cucim) framework for a speedup in patch-extraction.
4
+
3
5
The CLI of the main script for patch extraction ([main_extraction](preprocessing/main_extraction.py)) is as follows:
6
+
4
7
```bash
5
8
python3 main_extraction.py [-h]
6
-
usage: main_extraction.py [-h]
7
9
[--wsi_paths WSI_PATHS]
8
10
[--wsi_filelist WSI_FILELIST]
9
11
[--output_path OUTPUT_PATH]
10
12
[--wsi_extension {svs}]
11
13
[--config CONFIG]
12
14
[--patch_size PATCH_SIZE]
13
15
[--patch_overlap PATCH_OVERLAP]
14
-
[--downsample DOWNSAMPLE]
16
+
[--target_mpp TARGET_MPP]
15
17
[--target_mag TARGET_MAG]
18
+
[--downsample DOWNSAMPLE]
16
19
[--level LEVEL]
17
20
[--context_scales [CONTEXT_SCALES ...]]
18
21
[--check_resolution CHECK_RESOLUTION]
@@ -32,9 +35,11 @@ usage: main_extraction.py [-h]
32
35
[--tissue_annotation TISSUE_ANNOTATION]
33
36
[--masked_otsu]
34
37
[--otsu_annotation OTSU_ANNOTATION]
38
+
[--filter_patches FILTER_PATCHES]
35
39
[--log_path LOG_PATH]
36
40
[--log_level {critical,error,warning,info,debug}]
37
41
[--hardware_selection {cucim,openslide}]
42
+
[--wsi_properties DICT]
38
43
39
44
optional arguments:
40
45
-h, --help show this help message and exit
@@ -62,10 +67,16 @@ optional arguments:
62
67
downsampling should be used with respect to the highest possible resolution. Medium
63
68
priority, gets overwritten by target_mag if provided, but overwrites level. (default:
64
69
None)
70
+
--target_mpp TARGET_MPP
71
+
If this parameter is provided, the output level of the WSI corresponds to the level that
72
+
is at the target microns per pixel of the WSI. Alternative to target_mag, downsaple and level.
73
+
Highest priority,
74
+
overwrites target_mag, downsample and level if provided. (default: None)
65
75
--target_mag TARGET_MAG
66
76
If this parameter is provided, the output level of the WSI corresponds to the level that
67
-
is at the target magnification of the WSI. Alternative to downsaple and level. Highest
68
-
priority, overwrites downsample and level if provided. (default: None)
77
+
is at the target magnification of the WSI. Alternative to target_mpp, downsaple and level.
78
+
High priority, just target_mpp has a higher priority,
79
+
overwrites downsample and level if provided. (default: None)
69
80
--level LEVEL The tile level for sampling, alternative to downsample. Lowest priority, gets overwritten
70
81
by target_mag and downsample if they are provided. (default: None)
71
82
--context_scales [CONTEXT_SCALES ...]
@@ -112,13 +123,19 @@ optional arguments:
112
123
--otsu_annotation OTSU_ANNOTATION
113
124
Can be used to name a polygon annotation to determine the area for masked otsu
114
125
thresholding. Seperate multiple labels with '' (whitespace) (default: None)
126
+
--filter_patches FILTER_PATCHES
127
+
Post-extraction patch filtering to sort out artefacts, marker and other non-tissue patches with a DL model. Time consuming. Defaults to False.
128
+
(default: False)
115
129
--log_path LOG_PATH Path where log files should be stored. Otherwise, log files are stored in the output
116
130
folder (default: None)
117
131
--log_level {critical,error,warning,info,debug}
118
132
Set the logging level. Options are ['critical', 'error', 'warning', 'info', 'debug']
119
133
(default: None)
120
134
--hardware_selection {cucim,openslide}
121
135
Select hardware device (just if available, otherwise always cucim). Defaults to cucim.)
Copy file name to clipboardExpand all lines: preprocessing/patch_extraction/src/cli.py
+41-11
Original file line number
Diff line number
Diff line change
@@ -33,8 +33,9 @@ class PreProcessingYamlConfig(BaseModel):
33
33
# basic setups
34
34
patch_size: Optional[int]
35
35
patch_overlap: Optional[float]
36
-
downsample: Optional[int]
36
+
target_mpp: Optional[float]
37
37
target_mag: Optional[float]
38
+
downsample: Optional[int]
38
39
level: Optional[int]
39
40
context_scales: Optional[List[int]]
40
41
check_resolution: Optional[float]
@@ -62,11 +63,13 @@ class PreProcessingYamlConfig(BaseModel):
62
63
tissue_annotation: Optional[str]
63
64
masked_otsu: Optional[bool]
64
65
otsu_annotation: Optional[str]
66
+
filter_patches: Optional[bool]
65
67
66
68
# other
67
69
log_path: Optional[str]
68
70
log_level: Optional[str]
69
71
hardware_selection: Optional[str]
72
+
wsi_properties: Optional[dict]
70
73
71
74
72
75
classPreProcessingConfig(BaseModel):
@@ -84,12 +87,15 @@ class PreProcessingConfig(BaseModel):
84
87
patch_overlap (float, optional): The percentage amount pixels that should overlap between two different patches.
85
88
Please Provide as integer between 0 and 100, indicating overlap in percentage.
86
89
Defaults to 0.
90
+
target_mpp (float, optional): If this parameter is provided, the output level of the WSI
91
+
corresponds to the level that is at the target microns per pixel of the WSI.
92
+
Alternative to target_mag, downsaple and level. Highest priority, overwrites all other setups for magnifcation, downsample, or level.
93
+
target_mag (float, optional): If this parameter is provided, the output level of the WSI
94
+
corresponds to the level that is at the target magnification of the WSI.
95
+
Alternative to target_mpp, downsaple and level. High priority, just target_mpp has a higher priority, overwrites downsample and level if provided. Defaults to None.
87
96
downsample (int, optional): Each WSI level is downsampled by a factor of 2, downsample
88
97
expresses which kind of downsampling should be used with
89
98
respect to the highest possible resolution. Defaults to 0.
90
-
target_mag (float, optional): If this parameter is provided, the output level of the WSI
91
-
corresponds to the level that is at the target magnification of the WSI.
92
-
Alternative to downsaple and level. Defaults to None.
93
99
level (int, optional): The tile level for sampling, alternative to downsample. Defaults to None.
94
100
context_scales ([List[int], optional): Define context scales for context patches. Context patches are centered around a central patch.
95
101
The context-patch size is equal to the patch-size, but downsampling is different.
@@ -125,9 +131,12 @@ class PreProcessingConfig(BaseModel):
125
131
masked_otsu (bool, optional): Use annotation to mask the thumbnail before otsu-thresholding is used. Defaults to False.
126
132
otsu_annotation (bool, optional): Can be used to name a polygon annotation to determine the area
127
133
for masked otsu thresholding. Seperate multiple labels with ' ' (whitespace). Defaults to None.
134
+
filter_patches (bool, optional): Post-extraction patch filtering to sort out artefacts, marker and other non-tissue patches with a DL model. Time consuming.
135
+
Defaults to False.
128
136
log_path (str, optional): Path where log files should be stored. Otherwise, log files are stored in the output folder. Defaults to None.
129
137
log_level (str, optional): Set the logging level. Defaults to "info".
130
138
hardware_selection (str, optional): Select hardware device (just if available, otherwise always cucim). Defaults to "cucim".
139
+
wsi_properties (dict, optional): Dictionary with manual WSI metadata. Required keys are: ... TODO: add keys
131
140
132
141
Raises:
133
142
ValueError: Patch-size must be positive
@@ -150,6 +159,7 @@ class PreProcessingConfig(BaseModel):
150
159
patch_size: Optional[int] =256
151
160
patch_overlap: Optional[float] =0
152
161
downsample: Optional[int] =1
162
+
target_mpp: Optional[float]
153
163
target_mag: Optional[float]
154
164
level: Optional[int]
155
165
context_scales: Optional[List[int]]
@@ -178,11 +188,13 @@ class PreProcessingConfig(BaseModel):
0 commit comments