Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 23 additions & 3 deletions .github/instructions/connector-table.instructions.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,28 @@
---
applyTo: "src/azure/connectors/*.py"
---
# Connector Table Maintenance
# Connector Code Maintenance

When a new connector module is added to `src/azure/connectors/`, update the supported SDK connector names list in `.github/skills/connection-setup/SKILL.md` (Step 2). Add the new connector's API name (e.g., `office365`, `sharepointonline`) to the inline list.
## Generated Code Rules

Also update the validated connectors table in `README.md` if the connector has been validated end-to-end.
Generated connector files in `src/azure/connectors/` are read-only. Do NOT hand-edit generated files. If the generated code has bugs:

1. Fix the generator in BPM repo (`src/tools/CodefulSdkGenerator/DirectClient/DirectClientPythonCodeGenerator.cs`)
2. Regenerate the connector
3. Add the defect to the Known Generator Defects Registry in `GENERATION.md`

See `GENERATION.md` for:
- **Generated Client Acceptance Criteria** — invariants every generated client must satisfy
- **Generator File Locations** — BPM repo paths for fixing generator bugs
- **Known Generator Defects Registry** — tracking known issues

## Validation Checklist

When a new connector module is added to `src/azure/connectors/`:

1. Run the code quality tests: `pytest tests/test_code_quality.py -v`
2. Update the supported SDK connector names list in `.github/skills/connection-setup/SKILL.md` (Step 2)
3. Update the validated connectors table in `README.md`
4. Ensure unit tests cover:
- Request body assertions for create/update operations
- Error handling (non-2xx raises `ConnectorException`) for all operations including void ones
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **13 new connector clients** with unit tests and samples:
- ARM (Azure Resource Manager), Azure AD, Azure Cosmos DB, Azure Event Hubs, Azure Queues, Azure Tables, Excel Online (Business), Microsoft Dataverse, Microsoft Defender ATP, Outlook, Service Bus, SMTP, Word Online (Business)

### Changed

- **Comprehensive error handling improvements** across 14 connector clients:
- Added error handling tests for all HTTP operations (4xx/5xx responses)
- Fixed variable shadowing in `outlook.py` and `office365.py` (`respond_to_event_async`)
- Fixed incorrect `ConnectorException` signature in `teams.py` (2 methods)

## [0.2.0b2] - 2026-05-13

### Added
Expand Down
113 changes: 111 additions & 2 deletions GENERATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -323,8 +323,10 @@ If a connector name is not recognized:
If generated code has syntax errors or type issues:

1. Check BPM repository is up to date (especially PR 15456622 for `--python` support)
2. Report issues to BPM team with connector name and error details
3. Review swagger definition for edge cases
2. Fix the generator, not the generated output — see [Generator File Locations](#generator-file-locations-bpm-repository)
3. Report issues to BPM team with connector name and error details (file in [BPM Azure DevOps](https://dev.azure.com/msazure/One/_workitems))
4. Add the defect to the [Known Generator Defects Registry](#known-generator-defects-registry)
5. Review swagger definition for edge cases

## Example Workflows

Expand Down Expand Up @@ -357,6 +359,113 @@ LogicAppsCompiler.exe $outputDir unused --directClient --python --connectors=$co
4. **Testing** - Always run full test suite after regeneration
5. **Documentation** - Keep README and ROADMAP in sync with available connectors

## Generated Client Acceptance Criteria

Every generated connector client MUST satisfy these invariants before merge:

### 1. Request Body Handling (Create/Update Operations)

- Every `POST`, `PUT`, or `PATCH` operation that accepts input parameters MUST construct and send a request body
- All input parameters marked as body parameters in swagger MUST be serialized into the request
- Create/update operations MUST NOT silently drop input parameters

**Validation:** Unit tests MUST assert that the outgoing request body contains expected fields for create/update operations.

### 2. Response Status Checking (All Operations)

- Every HTTP operation MUST capture the response and check the status code
- Non-2xx responses MUST raise `ConnectorException` with status code and message
- Void operations (DELETE, PATCH, fire-and-forget POST) MUST NOT discard the response unchecked

**Correct pattern:**
```python
response = await self.http_client.send_async(
method="DELETE",
url=request_uri,
)
if not (200 <= response.status < 300):
raise ConnectorException(
f"Operation failed with status {response.status}: {await response.text()}"
)
```

**Incorrect pattern (MUST NOT pass review):**
```python
await self.http_client.send_async( # Result discarded - silent failures!
method="DELETE",
url=request_uri,
)
```

**Validation:** The `test_no_unchecked_send_async_calls` test in `tests/test_code_quality.py` scans for unchecked `send_async` calls. Additionally, unit tests for void methods MUST assert that non-2xx responses raise `ConnectorException`.

### 3. Test Coverage

- No skipped tests without a tracked issue link in the skip reason
- Each public method MUST have tests for: success case, error response handling, parameter serialization
- Void methods MUST have explicit tests verifying non-2xx raises exception

**Validation:** Tests with `@pytest.mark.skip` MUST include a GitHub issue link (e.g., `reason="Template variable bug - see #123"`).

### 4. Type Completeness

- All public types (dataclasses) referenced by method signatures MUST be defined
- No `# type: ignore` comments unless accompanied by a tracked issue link
- All optional parameters MUST have `Optional[T]` type hints and default values

## Generator File Locations (BPM Repository)

When fixing generator bugs (per the "fix the generator, not generated output" rule), these are the key files:

### Python Generator

| File | Purpose |
|------|--------|
| `src/tools/CodefulSdkGenerator/DirectClient/DirectClientPythonGenerator.cs` | Entry point for Python generation; orchestrates the generation pipeline |
| `src/tools/CodefulSdkGenerator/DirectClient/DirectClientPythonCodeGenerator.cs` | Core code emitter; generates Python client classes, methods, and type definitions |

### Key Differences from .NET Generator

The .NET SDK centralizes error handling in `ConnectorClientBase.CallConnectorAsync()` — this base method auto-throws on non-2xx for both value-returning and void operations. The Python emitter must explicitly generate the status check in each method body because Python's `http_client.send_async()` does not auto-raise.

**If you find a Python generator bug:**
1. Fix it in `DirectClientPythonCodeGenerator.cs`
2. Add a test case in `CodefulSdkGenerator.Tests/`
3. Regenerate affected connectors
4. Add the defect to the Known Generator Defects Registry below

### C# Generator

| File | Purpose |
|------|--------|
| `src/tools/CodefulSdkGenerator/DirectClient/DirectClientCSharpGenerator.cs` | C# generation entry point |
| `src/tools/CodefulSdkGenerator/DirectClient/DirectClientCSharpCodeGenerator.cs` | C# code emitter |

### Shared Infrastructure

| File | Purpose |
|------|--------|
| `src/tools/CodefulSdkGenerator/ConnectorsGenerator.cs` | ARM API fetching, swagger parsing, JSON sanitization |
| `src/tools/CodefulSdkGenerator/Data/ManagedConnectorSkipList.txt` | Connectors skipped due to known issues |
| `src/tools/CodefulSdkGenerator/Data/*.txt` | Character replacement and reformatting lists |

## Known Generator Defects Registry

Track known generator issues here to prevent silent recurrence across releases.

| Defect | BPM Issue | Affected Connectors | Status | Notes |
|--------|-----------|---------------------|--------|-------|
| Teams template variable causes NameError at import | [BPM #TBD](https://dev.azure.com/msazure/One/_workitems/edit/TBD) | `teams` | Workaround (skip import test) | `DynamicValueLookup` references undefined variable; test skipped with issue link |
| Void operations not checking response status | Fixed in generator | Pre-2024 connectors | Fixed | Regenerate affected connectors: `azurequeues`, `azuretables`, `commondataservice`, `eventhubs`, `excelonlinebusiness`, `office365`, `onedrive`, `outlook`, `servicebus`, `sharepointonline`, `smtp`, `azuread` |
| api_version parameters missing default values | Fixed in generator | `azuredigitaltwins`, `azurevm`, others | Fixed | Generator now uses swagger default value for api_version/api-version parameters |
| Create/update operations missing request body | [BPM #TBD](https://dev.azure.com/msazure/One/_workitems/edit/TBD) | Varies by connector | Investigation | Some POST/PATCH operations may drop body parameters |

**Adding a new defect:**
1. File an issue in BPM Azure DevOps
2. Add a row to this table with connector list
3. Add `@pytest.mark.skip(reason="Generator defect - see GENERATION.md")` to affected tests
4. Update status when fixed and connectors regenerated

## Related Documentation

- [README.md](README.md) - SDK overview and quick start
Expand Down
36 changes: 18 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,40 +154,40 @@ The following connectors have been generated and validated with comprehensive te

| Connector | Package | Status | Coverage | Tests |
|-----------|---------|--------|----------|-------|
| **ARM (Azure Resource Manager)** | `azure.connectors.arm` | ✅ Complete | 🔄 SDK Generated | 39 tests |
| **ARM (Azure Resource Manager)** | `azure.connectors.arm` | ✅ Complete | 🔄 SDK Generated | 57 tests |
| **Azure Automation** | `azure.connectors.azureautomation` | ✅ Complete | 🔄 SDK Generated | 32 tests |
| **Azure AD** | `azure.connectors.azuread` | ✅ Complete | 🔄 SDK Generated | 35 tests |
| **Azure Blob Storage** | `azure.connectors.azureblob` | ✅ Complete | ✅ E2E Validated | 52 tests |
| **Azure AD** | `azure.connectors.azuread` | ✅ Complete | 🔄 SDK Generated | 68 tests |
| **Azure Blob Storage** | `azure.connectors.azureblob` | ✅ Complete | ✅ E2E Validated | 51 tests |
| **Azure Cosmos DB** | `azure.connectors.documentdb` | ✅ Complete | 🔄 SDK Generated | 46 tests |
| **Azure Data Factory** | `azure.connectors.azuredatafactory` | ✅ Complete | 🔄 SDK Generated | 38 tests |
| **Azure Digital Twins** | `azure.connectors.azuredigitaltwins` | ✅ Complete | 🔄 SDK Generated | 44 tests |
| **Azure Data Explorer** | `azure.connectors.kusto` | ✅ Complete | ✅ E2E Validated | 37 tests |
| **Azure Event Hubs** | `azure.connectors.eventhubs` | ✅ Complete | 🔄 SDK Generated | 30 tests |
| **Azure Event Hubs** | `azure.connectors.eventhubs` | ✅ Complete | 🔄 SDK Generated | 32 tests |
| **Azure Key Vault** | `azure.connectors.keyvault` | ✅ Complete | 🔄 SDK Generated | 42 tests |
| **Azure Queues** | `azure.connectors.azurequeues` | ✅ Complete | 🔄 SDK Generated | 34 tests |
| **Azure Tables** | `azure.connectors.azuretables` | ✅ Complete | 🔄 SDK Generated | 43 tests |
| **Azure Queues** | `azure.connectors.azurequeues` | ✅ Complete | 🔄 SDK Generated | 38 tests |
| **Azure Tables** | `azure.connectors.azuretables` | ✅ Complete | 🔄 SDK Generated | 53 tests |
| **Azure VM** | `azure.connectors.azurevm` | ✅ Complete | 🔄 SDK Generated | 43 tests |
| **Excel Online (Business)** | `azure.connectors.excelonlinebusiness` | ✅ Complete | 🔄 SDK Generated | 37 tests |
| **IBM MQ** | `azure.connectors.mq` | ✅ Complete | ✅ E2E Validated | 30 tests |
| **Excel Online (Business)** | `azure.connectors.excelonlinebusiness` | ✅ Complete | 🔄 SDK Generated | 53 tests |
| **IBM MQ** | `azure.connectors.mq` | ✅ Complete | ✅ E2E Validated | 33 tests |
| **Microsoft Bookings** | `azure.connectors.microsoftbookings` | ✅ Complete | 🔄 SDK Generated | 36 tests |
| **Microsoft Dataverse** | `azure.connectors.commondataservice` | ✅ Complete | 🔄 SDK Generated | 46 tests |
| **Microsoft Dataverse** | `azure.connectors.commondataservice` | ✅ Complete | 🔄 SDK Generated | 53 tests |
| **Microsoft Defender ATP** | `azure.connectors.wdatp` | ✅ Complete | 🔄 SDK Generated | 32 tests |
| **Microsoft Graph** | `azure.connectors.msgraphgroupsanduser` | ✅ Complete | ✅ E2E Validated | 46 tests |
| **Microsoft Teams** | `azure.connectors.teams` | ✅ Complete | ✅ E2E Validated | 27 tests |
| **Office 365 Outlook** | `azure.connectors.office365` | ✅ Complete | ✅ E2E Validated | 41 tests |
| **Office 365 Users** | `azure.connectors.office365users` | ✅ Complete | ✅ E2E Validated | 40 tests |
| **Microsoft Teams** | `azure.connectors.teams` | ✅ Complete | ✅ E2E Validated | 44 tests |
| **Office 365 Outlook** | `azure.connectors.office365` | ✅ Complete | ✅ E2E Validated | 72 tests |
| **Office 365 Users** | `azure.connectors.office365users` | ✅ Complete | ✅ E2E Validated | 46 tests |
| **Office 365 Groups** | `azure.connectors.office365groups` | ✅ Complete | 🔄 SDK Generated | 50 tests |
| **OneDrive for Business** | `azure.connectors.onedrive` | ✅ Complete | 🔄 SDK Generated | 41 tests |
| **OneDrive for Business** | `azure.connectors.onedrive` | ✅ Complete | 🔄 SDK Generated | 49 tests |
| **OneNote** | `azure.connectors.onenote` | ✅ Complete | 🔄 SDK Generated | 60 tests |
| **Outlook.com** | `azure.connectors.outlook` | ✅ Complete | 🔄 SDK Generated | 42 tests |
| **Outlook.com** | `azure.connectors.outlook` | ✅ Complete | 🔄 SDK Generated | 49 tests |
| **Planner** | `azure.connectors.planner` | ✅ Complete | 🔄 SDK Generated | 66 tests |
| **Service Bus** | `azure.connectors.servicebus` | ✅ Complete | 🔄 SDK Generated | 37 tests |
| **SharePoint Online** | `azure.connectors.sharepointonline` | ✅ Complete | ✅ E2E Validated | 44 tests |
| **SMTP** | `azure.connectors.smtp` | ✅ Complete | 🔄 SDK Generated | 27 tests |
| **Service Bus** | `azure.connectors.servicebus` | ✅ Complete | 🔄 SDK Generated | 52 tests |
| **SharePoint Online** | `azure.connectors.sharepointonline` | ✅ Complete | ✅ E2E Validated | 80 tests |
| **SMTP** | `azure.connectors.smtp` | ✅ Complete | 🔄 SDK Generated | 28 tests |
| **Word Online (Business)** | `azure.connectors.wordonlinebusiness` | ✅ Complete | 🔄 SDK Generated | 30 tests |
| **Yammer** | `azure.connectors.yammer` | ✅ Complete | 🔄 SDK Generated | 38 tests |

**Total:** 685 connector tests + 110 SDK component tests
**Total:** 1049 connector tests + 141 SDK component tests

See [ROADMAP.md](ROADMAP.md) for planned connector additions and [tests/README.md](tests/README.md) for detailed test coverage.

Expand Down
Loading