Skip to content

Commit 925df6f

Browse files
chore: test on gh (#180)
* breaking(api): Support Platform API 1.0.0-beta.7 * chore(lint): Integrate pyright as additional type checker * chore(test): Introduce schedule tests against staging * feat(platform): Retries and caching for read-only and auth operations * feat(platform): Dynamic user agent for all operations * feat(application): Custom metadata with run and scheduling information in custom metadata * chore(tests): Introduce very long running tests * chore(tests): Introduce pytest-timeout and 10s default timeout for all tests * style(application): Layout improvements on application detail page * chore(tests): Improve test coverage * chore(AI): Improve CLAUDE.md files and AI workflows
1 parent cb18241 commit 925df6f

File tree

159 files changed

+23487
-8947
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

159 files changed

+23487
-8947
lines changed

.github/CLAUDE.md

Lines changed: 1097 additions & 0 deletions
Large diffs are not rendered by default.

.github/actions/run-tests/action.yml

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,17 +14,20 @@ inputs:
1414
summary-title:
1515
description: 'Title for the GitHub step summary'
1616
required: true
17+
commit-message:
18+
description: 'The commit message to check for skip markers'
19+
required: true
1720

1821
runs:
1922
using: 'composite'
2023
steps:
2124
- name: Run tests
2225
if: |
2326
(inputs.skip-marker == '' || (
24-
!contains(github.event.head_commit.message, inputs.skip-marker) &&
27+
!contains(inputs.commit-message, inputs.skip-marker) &&
2528
!contains(github.event.pull_request.labels.*.name, inputs.skip-marker)
2629
)) &&
27-
(!contains(github.event.head_commit.message, 'skip:test:all')) &&
30+
(!contains(inputs.commit-message, 'skip:test:all')) &&
2831
(!contains(github.event.pull_request.labels.*.name, 'skip:test:all'))
2932
shell: bash
3033
run: |

.github/copilot-instructions.md

Lines changed: 225 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,228 @@
1-
Always conform to the coding styles defined in CODE_STYLE.md in the root
2-
directory of this repository when generating code.
1+
# Copilot Instructions - Aignostics Python SDK
32

4-
Learn about tools to use in CONTRIBUTING.md in the root directory of this
5-
repository.
3+
## Project Overview
64

7-
## Important notice for creating pull requests
5+
The Aignostics Python SDK is a **computational pathology platform** providing multiple interfaces to process whole slide images (WSI) with AI/ML applications. It follows a **modulith architecture** with independent modules connected via dependency injection.
86

9-
If you are GitHub Copilot, and you are creating a pull request yourself, add a label skip:test_long_running, to skip running long running tests. This is important because some tests in this repository are marked as long_running and can take a significant amount of time to complete. By adding this label, you help ensure that the CI pipeline runs efficiently and avoids unnecessary delays.
7+
**Key Components:**
8+
- **Launchpad**: Desktop GUI (NiceGUI + webview)
9+
- **CLI**: Command-line interface (Typer)
10+
- **Client Library**: Python API wrapper
11+
- **Notebook Integration**: Marimo/Jupyter support
12+
13+
## Architecture Principles
14+
15+
### 1. Modulith Design Pattern
16+
Each module follows a consistent three-layer structure:
17+
```
18+
module/
19+
├── _service.py # Business logic (inherits BaseService)
20+
├── _cli.py # CLI commands (Typer)
21+
├── _gui.py # GUI interface (NiceGUI)
22+
├── _settings.py # Configuration (Pydantic)
23+
└── CLAUDE.md # Detailed documentation
24+
```
25+
26+
### 2. Service Discovery & Dependency Injection
27+
- All services inherit from `BaseService` and implement `health()` and `info()` methods
28+
- Use `locate_implementations(BaseService)` for runtime service discovery
29+
- No decorators - pure runtime DI container pattern
30+
- Services are singletons within the DI container
31+
32+
### 3. Presentation Layer Independence
33+
```
34+
CLI Layer ─┐
35+
├─→ Service Layer
36+
GUI Layer ─┘
37+
```
38+
CLI and GUI layers depend on Service layer, never on each other.
39+
40+
## Module Dependencies & Communication
41+
42+
**Foundation Layer:**
43+
- `utils`: DI container, logging, settings, health checks
44+
45+
**API Layer:**
46+
- `platform`: OAuth 2.0 auth, JWT tokens, API client
47+
48+
**Domain Modules:**
49+
- `application`: ML run orchestration (depends on: platform, bucket, wsi, qupath optional)
50+
- `wsi`: Medical image processing (OpenSlide, PyDICOM)
51+
- `dataset`: IDC downloads with s5cmd
52+
- `bucket`: Cloud storage (S3/GCS)
53+
54+
**Integration:**
55+
- `qupath`: Bioimage analysis (requires `ijson`)
56+
- `notebook`: Marimo server (requires `marimo`)
57+
- `gui`: Desktop launchpad (aggregates all GUIs)
58+
- `system`: Health monitoring (queries ALL services)
59+
60+
## Development Workflow Commands
61+
62+
**Primary Commands:**
63+
```bash
64+
make install # Install dev deps + pre-commit hooks
65+
make all # Full CI pipeline (lint, test, docs, audit)
66+
make test # Run tests with coverage (85% minimum)
67+
make test 3.12 # Run on specific Python version
68+
make lint # Ruff formatting + MyPy type checking
69+
```
70+
71+
**Package Management:**
72+
- Uses `uv` (not pip/poetry): `uv sync --all-extras`
73+
- Add dependencies: `uv add <package>`
74+
75+
**Testing:**
76+
- Pytest with markers: `sequential`, `long_running`, `scheduled`, `docker`, `skip_with_act`
77+
- Run specific tests: `uv run pytest tests/path/test.py::test_function`
78+
- Docker integration: `make test-docker`
79+
80+
## Code Patterns & Standards
81+
82+
### Service Implementation
83+
```python
84+
from aignostics.utils import BaseService, Health
85+
86+
class Service(BaseService):
87+
def __init__(self):
88+
super().__init__(SettingsClass) # Optional settings
89+
90+
def health(self) -> Health:
91+
return Health(status=Health.Code.UP)
92+
93+
def info(self, mask_secrets: bool = True) -> dict:
94+
return {"version": "1.0.0"}
95+
```
96+
97+
### CLI Pattern
98+
```python
99+
import typer
100+
from ._service import Service
101+
102+
cli = typer.Typer(name="module", help="Module description")
103+
104+
@cli.command("action")
105+
def action_command(param: str):
106+
"""Command description."""
107+
service = Service()
108+
result = service.perform_action(param)
109+
console.print(result)
110+
```
111+
112+
### GUI Pattern
113+
```python
114+
from nicegui import ui
115+
116+
def create_page():
117+
ui.label("Module Interface")
118+
# Components auto-register with GUI launcher
119+
```
120+
121+
## Testing Conventions
122+
123+
**File Structure:**
124+
- Tests in `tests/aignostics/<module>/`
125+
- Use `conftest.py` fixtures for common setup
126+
- Mock external dependencies
127+
128+
**Patterns:**
129+
- Use `CliRunner` from `typer.testing` for CLI tests
130+
- Use `normalize_output()` helper for cross-platform CLI output
131+
- Cleanup fixtures for processes (e.g., `qupath_teardown`)
132+
133+
## Medical Domain Context
134+
135+
**Key Technologies:**
136+
- **DICOM**: Medical imaging standard
137+
- **WSI**: Gigapixel pathology images (pyramidal multi-resolution)
138+
- **IDC**: NCI Imaging Data Commons for public datasets
139+
- **QuPath**: Leading bioimage analysis platform
140+
- **H&E**: Hematoxylin & Eosin histological staining
141+
142+
**Processing Patterns:**
143+
- Tile-based processing for memory efficiency
144+
- Streaming for large file transfers
145+
- Chunked uploads/downloads (1MB/10MB chunks)
146+
- Signed URLs for secure data access
147+
148+
## Security & Performance
149+
150+
**Authentication:**
151+
- OAuth 2.0 device flow via `platform` module
152+
- Tokens cached in `~/.aignostics/token.json`
153+
- 5-minute refresh buffer before expiry
154+
155+
**Performance:**
156+
- Lazy evaluation for large datasets
157+
- Process management for subprocesses
158+
- Memory-efficient WSI processing in tiles
159+
- Async operations for I/O-bound tasks
160+
161+
## Configuration & Environment
162+
163+
**Settings Pattern:**
164+
```python
165+
from pydantic_settings import BaseSettings
166+
167+
class Settings(BaseSettings):
168+
api_root: str = "https://platform.aignostics.com"
169+
170+
class Config:
171+
env_prefix = "AIGNOSTICS_"
172+
```
173+
174+
**Environment Variables:**
175+
- `AIGNOSTICS_API_ROOT`: Platform endpoint
176+
- `AIGNOSTICS_CLIENT_ID_DEVICE`: OAuth client ID
177+
- `AIGNOSTICS_REFRESH_TOKEN`: Auth token
178+
179+
## Common Integration Points
180+
181+
**Application Run Workflow:**
182+
```python
183+
# 1. Authenticate
184+
client = platform.Client()
185+
186+
# 2. Submit run
187+
run = client.runs.create(
188+
application_id="heta",
189+
application_version="1.0.0", # version number without 'v' prefix, omit for latest
190+
items=[platform.InputItem(...)]
191+
)
192+
193+
# 3. Monitor & download
194+
run.download_to_folder("./results")
195+
```
196+
197+
**Service Health Monitoring:**
198+
```python
199+
from aignostics.utils import locate_implementations, BaseService
200+
201+
# Discover all services
202+
services = locate_implementations(BaseService)
203+
204+
# Check health of all services
205+
for service_class in services:
206+
service = service_class()
207+
health = service.health()
208+
print(f"{service.key()}: {health.status}")
209+
```
210+
211+
## Important Notes
212+
213+
**Optional Dependencies:**
214+
- GUI requires: `pip install "aignostics[gui]"`
215+
- QuPath requires: `pip install "aignostics[qupath]"`
216+
- Notebooks require: `pip install "aignostics[notebook]"`
217+
218+
**Platform Constraints:**
219+
- Windows path length limitations
220+
- Memory usage for large WSI files
221+
- Token expiry handling (force refresh with `remove_cached_token()`)
222+
223+
**Build System:**
224+
- Main config: `pyproject.toml`
225+
- Build tasks: `noxfile.py` (not tox)
226+
- Quality gates: Ruff (formatting/linting), MyPy (typing), 85% test coverage
227+
228+
Always conform to the coding styles defined in [CODE_STYLE.md](../CODE_STYLE.md) and development processes in [CONTRIBUTING.md](../CONTRIBUTING.md), and have a look at [OPERATIONAL_EXCELLENCE.md](../OPERATIONAL_EXCELLENCE.md) for release readiness.

.github/labels.yml

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
# GitHub Labels Configuration
2+
#
3+
# This file defines all labels used in the repository.
4+
# Labels are automatically synced by .github/workflows/labels-sync.yml
5+
# when this file is modified and pushed to main branch.
6+
#
7+
# Manual sync: gh label sync -f .github/labels.yml
8+
9+
# Standard GitHub Labels
10+
- name: bug
11+
description: Something isn't working
12+
color: "d73a4a"
13+
14+
- name: documentation
15+
description: Improvements or additions to documentation
16+
color: "0075ca"
17+
18+
- name: duplicate
19+
description: This issue or pull request already exists
20+
color: "cfd3d7"
21+
22+
- name: enhancement
23+
description: New feature or request
24+
color: "a2eeef"
25+
26+
- name: good first issue
27+
description: Good for newcomers
28+
color: "7057ff"
29+
30+
- name: help wanted
31+
description: Extra attention is needed
32+
color: "008672"
33+
34+
- name: invalid
35+
description: This doesn't seem right
36+
color: "e4e669"
37+
38+
- name: question
39+
description: Further information is requested
40+
color: "d876e3"
41+
42+
- name: wontfix
43+
description: This will not be worked on
44+
color: "ffffff"
45+
46+
# Dependency Management Labels
47+
- name: dependencies
48+
description: Pull requests that update a dependency file
49+
color: "0366d6"
50+
51+
- name: github_actions
52+
description: Pull requests that update GitHub Actions
53+
color: "000000"
54+
55+
- name: python
56+
description: Pull requests that update python code
57+
color: "2b67c6"
58+
59+
# Bot Labels
60+
- name: bot
61+
description: Automated pull requests or issues
62+
color: "b1aa07"
63+
64+
- name: dependabot
65+
description: Pull requests from Dependabot
66+
color: "97827a"
67+
68+
- name: renovate
69+
description: Pull requests from Renovate
70+
color: "d1df30"
71+
72+
# CI/CD Skip Labels
73+
- name: skip:ci
74+
description: Skip entire CI/CD pipeline
75+
color: "b167f6"
76+
77+
- name: skip:test:all
78+
description: Skip all test executions
79+
color: "5e0647"
80+
81+
- name: skip:test:long_running
82+
description: Skip long-running tests (≥5min)
83+
color: "910432"
84+
85+
- name: skip:test:e2e
86+
description: Skip end-to-end tests
87+
color: "0dc59c"
88+
89+
- name: skip:test:integration
90+
description: Skip integration tests
91+
color: "64ac56"
92+
93+
- name: skip:test:unit
94+
description: Skip unit tests
95+
color: "614318"
96+
97+
- name: skip:ketryx
98+
description: Skip Ketryx ALM reporting and checks
99+
color: "8b4513"
100+
101+
# CI/CD Enable Labels
102+
- name: enable:test:very_long_running
103+
description: Enable very long-running tests (≥60min)
104+
color: "9033fe"
105+
106+
# Build Control Labels
107+
- name: build:native:only
108+
description: Build native packages only (macOS/Windows apps)
109+
color: "1d76db"
110+
111+
# AI Assistant Labels
112+
- name: claude
113+
description: Trigger Claude Code automation
114+
color: "b41d8f"
115+
116+
- name: copilot
117+
description: GitHub Copilot related
118+
color: "e6dac6"
119+
120+
# Quality Assurance Labels
121+
- name: code-quality
122+
description: Code quality and maintainability issues
123+
color: "fbca04"
124+
125+
- name: automated-check
126+
description: Issue created by automated quality checks
127+
color: "ededed"
128+
129+
- name: documentation-drift
130+
description: Documentation out of sync with code
131+
color: "ff6b6b"

.github/workflows/_audit.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ jobs:
1818
fetch-depth: 0
1919

2020
- name: Install uv
21-
uses: astral-sh/setup-uv@3259c6206f993105e3a61b142c2d97bf4b9ef83d # v7.1.0
21+
uses: astral-sh/setup-uv@2ddd2b9cb38ad8efd50337e8ab201519a34c9f24 # v7.1.1
2222
with:
2323
version-file: "pyproject.toml"
2424
enable-cache: true

0 commit comments

Comments
 (0)