diff --git a/AGENTS.md b/AGENTS.md index e86fc55..9ff0aef 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,13 +4,13 @@ clickhousectl (or chctl) is the official CLI for ClickHouse, by ClickHouse Inc. ## Architecture -This is a Cargo workspace with two crates: +This is a Cargo workspace with three crates: ### CLI (`crates/clickhousectl/`) The user-facing CLI surface. Contains all logic for local commands, wraps `clickhouse-cloud-api` for cloud. -- Cloud handlers go through `CloudClient` wrapper methods co-located in each domain module, not `clickhouse_cloud_api::Client` directly. `src/cloud/client.rs` owns the core client, credential precedence, error conversion, and response unwrapping. +- New Cloud handlers go through `CloudClient` wrapper methods co-located in each domain module, not `clickhouse_cloud_api::Client` directly. `src/cloud/client.rs` owns the core client, credential precedence, error conversion, and response unwrapping. Some pre-modularization Postgres and service-query paths still call the API client directly; do not copy that pattern into new commands. - Cloud handlers always support `--json` output unless there is good reason not to. JSON is emitted automatically when `--json` is passed or a coding agent is detected (`is_ai_agent::detect()` via the `json_output()` helper in `main.rs`). - `CloudError` carries a `kind: CloudErrorKind` (`Auth` for 401/403 and missing credentials, else `Generic`). It maps to `Error::AuthRequired` / `Error::Cloud` in `cloud::run`. Dispatched commands exit with `0` on success or use `Error::exit_code()` for failures: `1` error, `3` cancelled, `4` auth required. Clap uses `2` for usage errors. @@ -22,12 +22,12 @@ The CLI does not need to have 100% coverage of endpoints exposed by the API libr #### Adding a command -Local clap definitions live in `src/local/cli.rs`. Cloud clap definitions, handlers, builders, wrapper methods, dispatch, and tests are co-located in the owning domain module under `src/cloud/`; `src/cloud/cli.rs` contains only the top-level cloud arguments and command enum. +Local clap definitions live in `src/local/cli.rs`. Cloud clap definitions, handlers, builders, wrapper methods, dispatch, and tests are co-located in the owning domain module under `src/cloud/`; `src/cloud/cli.rs` owns the top-level cloud arguments, command enum, domain re-exports, delegation, and top-level tests. **Local subcommand:** 1. Add a variant to the relevant enum in `src/local/cli.rs` using clap derive macros. -2. Add the match arm in `run_local()` in `src/main.rs`. +2. Add the match arm in `run()` in `src/local/mod.rs`; `main.rs` delegates to that boundary. 3. Implement the handler in a dedicated module under `src/local/` (e.g. `src/local/server.rs`, `src/local/postgres.rs`). Don't pile new logic into `main.rs`. **Cloud subcommand:** @@ -53,7 +53,7 @@ Local clap definitions live in `src/local/cli.rs`. Cloud clap definitions, handl ### API library (`crates/clickhouse-cloud-api/`) -Typed Rust client library for the ClickHouse Cloud API. The library owns all OpenAPI interaction and all cloud integration testing. +Typed Rust client library for the ClickHouse Cloud API. The library owns typed HTTP interaction and all cloud integration testing; the private analyzer owns OpenAPI parsing and comparison. - `src/client.rs` — `Client` and shared HTTP machinery; endpoint methods live in private per-domain `src/client/*.rs` files. - `src/models.rs` — the public model facade and shared discriminated-union macro. Request/response structs, enums, aliases, and their implementations live in private per-domain `src/models/*.rs` files and are re-exported without changing the crate-root or `models::*` paths. @@ -61,6 +61,10 @@ Typed Rust client library for the ClickHouse Cloud API. The library owns all Ope The drift analyzer recursively traverses the private module trees rooted at `client.rs`, `models.rs`, and `meta.rs`. Model declarations remain literal source in that tree; declarations in conversion files do not count as models. +### OpenAPI analyzer (`crates/clickhouse-openapi-analyzer/`) + +Private workspace tooling for OpenAPI and Rust inventory, direction-aware comparison, policy configuration, and stable drift reports. It is not published. + The API library can be updated independently of the CLI. When OpenAPI drifts, prefer updating API library on its own, add to CLI separately. #### Request and response models @@ -98,6 +102,7 @@ A key that is *present* with a changed type still fails. `Option` absorbs abs - `every_response_tree_option_field_omits_none_when_serialized` — `skip_serializing_if` on every response `Option` field. - `models_carry_no_serde_default` — via the analyzer's `model_fields_with_serde_default()`. - `scim_models_are_outside_the_response_tree` — the 40 `Scim*` schemas have no path in the spec and no `Client` method, so they are legitimately strict, and the test fails if one becomes response-reachable. +- `integer_schema_fields_are_not_typed_as_float` — integer schemas do not use floating-point Rust fields. Scope enforcement to the response tree, never to "every model type": operation-unreferenced and request-only schemas resolve in request position, so making them all-`Option` reports genuine `FieldOptionalityMismatch` drift. @@ -180,15 +185,16 @@ Use cargo build, cargo test, cargo clippy, locally. Real cloud integration tests, 100% OpenAPI spec coverage. Cost is not a reason to skip a test. - `tests/common/support.rs` — generic test infra (polling, logging, env helpers, ClickHouse provisioning & cleanup, HTTP query helper). Used by every integration binary. Call `Client` directly from Rust. -- `tests/integration_test.rs`, `tests/integration_postgres_test.rs` — cloud-service / Postgres-service CRUD lifecycle tests. +- `tests/integration_test.rs`, `tests/integration_postgres_test.rs`, `tests/integration_org_test.rs` — cloud-service, Postgres-service, and organization lifecycle tests. - `tests/clickpipes/` — ClickPipes E2E suite, including external cloud services. Only Postgres CDC (uses ClickHouse & Postgres inside ClickHouse Cloud) is run in CI. Tests for third party services must be executed manually. CI also optionally runs `clickpipe_smoke_test` against a long-lived service when the `CLICKHOUSE_CLOUD_TEST_CLICKPIPE_SERVICE_ID` repo variable is set (see `.github/workflows/cloud-integration.yml`); the step is skipped when the variable is unset. - `spec_coverage_test.rs`: runs the shared analyzer against the vendored OpenAPI snapshot and requires an actionable-drift-free report. +- Labeled internal PRs classify the exact base-to-head diff with `scripts/classify-cloud-integration.py` and run only affected `service`, `postgres`, `organization`, and `clickpipes` suites. New or renamed API source/test files must be added to its explicit mappings; unknown paths fail closed to all suites. Scheduled runs still select all suites, while manual runs use the requested scope. ### clickhousectl CLI - **Clap parsing** — `Cli::try_parse_from` tests next to each command definition (`src/cli.rs`, the owning `src/cloud/.rs`, and `src/local/cli.rs`). Assert flag names, types, defaults, and repeatability. - **Request builders** — unit tests for `build_*_request` helpers next to the owning cloud domain code, asserting on library request-struct fields with minimal + maximal inputs. -- **Subprocess + wiremock** — `tests/cli_request_shape_test.rs`. Spawn the real binary against a local mock server and assert on the recorded request JSON. Used when the handler has runtime behavior beyond struct construction (file reads, base64 encoding, etc.) — currently ClickPipes. +- **Subprocess + wiremock** — `tests/cli_request_shape_test.rs`. Spawn the real binary against a local mock server and assert on requests, auth, errors, and output across Cloud domains. Use it when handler runtime behavior is not covered by clap or request-builder tests. - **Pure logic** — inline `mod tests` blocks across `src/` for version resolution, auth precedence, output formatting, platform detection, and other module-local helpers. ## Dependencies diff --git a/crates/clickhouse-cloud-api/clickhouse_cloud_openapi.json b/crates/clickhouse-cloud-api/clickhouse_cloud_openapi.json index 2f5a147..e8640cd 100644 --- a/crates/clickhouse-cloud-api/clickhouse_cloud_openapi.json +++ b/crates/clickhouse-cloud-api/clickhouse_cloud_openapi.json @@ -5,7 +5,7 @@ "version": "1.0", "contact": { "name": "ClickHouse Support", - "url": "https://clickhouse.com/docs/en/cloud/manage/openapi?referrer=openapi-1037030", + "url": "https://clickhouse.com/docs/en/cloud/manage/openapi?referrer=openapi-1064384", "email": "support@clickhouse.com" } }, @@ -363,7 +363,7 @@ "/v1/organizations/{organizationId}/prometheus": { "get": { "summary": "Get organization metrics", - "description": "Returns prometheus metrics for all services in an organization.", + "description": "Returns Prometheus metrics for the services in an organization that the caller is authorized to view. Services the caller lacks view access to are omitted.", "operationId": "organizationPrometheusGet", "parameters": [ { @@ -455,6 +455,111 @@ ] } }, + "/v1/organizations/{organizationId}/prometheus/discovery": { + "get": { + "summary": "Discover Prometheus scrape targets for an organization", + "description": "**Disclaimer:** This beta endpoint is evolving; the API contract may change.

Returns one Prometheus scrape target per service in the organization that the caller is authorized to view, in HTTP service discovery (http_sd) format. Services the caller lacks view access to, and services in a terminated, terminating, or (soft-)deleted state, are omitted — the response can have fewer groups than the organization has services. Point a Prometheus http_sd_configs job at this endpoint to discover and scrape all services automatically; targets are refreshed on every discovery poll, so created and deleted services are picked up without reconfiguration. Targets scrape with filtered_metrics=true by default; pass ?filtered_metrics=false to this endpoint to discover unfiltered targets.\n\nSample Prometheus scrape config:\n\n```yaml\nscrape_configs:\n - job_name: clickhouse-cloud\n http_sd_configs:\n - url: https://api.clickhouse.cloud/v1/organizations//prometheus/discovery\n refresh_interval: 60s\n basic_auth:\n username: \n password: \n basic_auth:\n username: \n password: \n```\n\nThe `basic_auth` block must be set both under `http_sd_configs` (used to authenticate discovery polls against this endpoint) and on the scrape job itself (used to authenticate the actual per-service scrapes) — omitting either one is the most common misconfiguration.", + "operationId": "organizationPrometheusDiscoveryGet", + "parameters": [ + { + "in": "path", + "name": "organizationId", + "description": "ID of the requested organization.", + "required": true, + "schema": { + "type": "string", + "format": "uuid" + } + }, + { + "in": "query", + "name": "filtered_metrics", + "description": "Whether discovered targets carry filtered_metrics=true or =false as a scrape param. Accepts true or false; defaults to true — the opposite default from the per-service prometheus endpoint.", + "schema": { + "type": "string", + "format": "boolean" + }, + "example": "true" + } + ], + "responses": { + "200": { + "description": "Successful response", + "content": { + "application/json": { + "schema": { + "type": "array", + "items": { + "$ref": "#/components/schemas/PrometheusDiscoveryTargetGroup" + } + } + } + } + }, + "400": { + "description": "The request cannot be processed due to a client error. Please verify your request parameters and try again.", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "status": { + "type": "number", + "description": "HTTP status code.", + "example": 400 + }, + "error": { + "type": "string", + "description": "Detailed error description." + }, + "requestId": { + "type": "string", + "description": "Unique id assigned to every request. UUIDv4", + "format": "uuid" + } + } + } + } + } + }, + "500": { + "description": "An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance.", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "status": { + "type": "integer", + "description": "HTTP status code.", + "example": 500 + }, + "error": { + "type": "string", + "description": "Detailed error description." + }, + "requestId": { + "type": "string", + "description": "Unique id assigned to every request. UUIDv4", + "format": "uuid" + } + } + } + } + } + } + }, + "tags": [ + "Prometheus" + ], + "x-badges": [ + { + "name": "Beta", + "position": "after" + } + ] + } + }, "/v1/organizations/{organizationId}/roles": { "get": { "summary": "List all available roles for an organization", @@ -6961,16 +7066,16 @@ ] } }, - "/v1/organizations/{organizationId}/services/{serviceId}/clickpipes": { + "/v1/organizations/{organizationId}/activeBalances": { "get": { - "summary": "List ClickPipes", - "description": "Returns a list of ClickPipes.", - "operationId": "clickPipeGetList", + "summary": "Get organization active prepaid balances", + "description": "**This endpoint is in beta.** API contract is stable, and no breaking changes are expected in the future.

Returns the active prepaid credit balances for the organization, each with its own balance ID and remaining credits, along with the total remaining credits across all active balances. A balance is active when it has started, has not expired, and has credits remaining. Balances are ordered by expiration date, soonest first, and the returned page is capped at `limit` (default and maximum 100). When `totalCount` exceeds the number of returned balances, page with `limit`/`offset` to retrieve them all. `totalRemainingPrepaidCredits` always covers every active balance, not just the returned page.", + "operationId": "activeBalancesGet", "parameters": [ { "in": "path", "name": "organizationId", - "description": "ID of the organization that owns the service.", + "description": "ID of the requested organization.", "required": true, "schema": { "type": "string", @@ -6978,13 +7083,24 @@ } }, { - "in": "path", - "name": "serviceId", - "description": "ID of the service that owns the ClickPipe.", - "required": true, + "in": "query", + "name": "limit", + "description": "Maximum number of results to return.", "schema": { - "type": "string", - "format": "uuid" + "type": "integer", + "minimum": 1, + "maximum": 100, + "default": 100 + } + }, + { + "in": "query", + "name": "offset", + "description": "Number of results to skip before returning.", + "schema": { + "type": "integer", + "minimum": 0, + "default": 0 } } ], @@ -7007,10 +7123,7 @@ "format": "uuid" }, "result": { - "type": "array", - "items": { - "$ref": "#/components/schemas/ClickPipe" - } + "$ref": "#/components/schemas/ActiveBalances" } } } @@ -7071,13 +7184,21 @@ } }, "tags": [ - "ClickPipes" + "Billing" + ], + "x-badges": [ + { + "name": "Beta", + "position": "after" + } ] - }, - "post": { - "summary": "Create ClickPipe", - "description": "Create a new ClickPipe.", - "operationId": "clickPipeCreate", + } + }, + "/v1/organizations/{organizationId}/services/{serviceId}/clickpipes": { + "get": { + "summary": "List ClickPipes", + "description": "Returns a list of ClickPipes.", + "operationId": "clickPipeGetList", "parameters": [ { "in": "path", @@ -7092,7 +7213,7 @@ { "in": "path", "name": "serviceId", - "description": "ID of the service to create the ClickPipe for.", + "description": "ID of the service that owns the ClickPipe.", "required": true, "schema": { "type": "string", @@ -7100,15 +7221,6 @@ } } ], - "requestBody": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/ClickPipePostRequest" - } - } - } - }, "responses": { "200": { "description": "Successful response", @@ -7128,7 +7240,10 @@ "format": "uuid" }, "result": { - "$ref": "#/components/schemas/ClickPipe" + "type": "array", + "items": { + "$ref": "#/components/schemas/ClickPipe" + } } } } @@ -7191,13 +7306,11 @@ "tags": [ "ClickPipes" ] - } - }, - "/v1/organizations/{organizationId}/services/{serviceId}/clickpipes/{clickPipeId}": { - "get": { - "summary": "Get ClickPipe", - "description": "Returns the specified ClickPipe.", - "operationId": "clickPipeGet", + }, + "post": { + "summary": "Create ClickPipe", + "description": "Create a new ClickPipe.", + "operationId": "clickPipeCreate", "parameters": [ { "in": "path", @@ -7212,17 +7325,7 @@ { "in": "path", "name": "serviceId", - "description": "ID of the service that owns the ClickPipe.", - "required": true, - "schema": { - "type": "string", - "format": "uuid" - } - }, - { - "in": "path", - "name": "clickPipeId", - "description": "ID of the requested ClickPipe.", + "description": "ID of the service to create the ClickPipe for.", "required": true, "schema": { "type": "string", @@ -7230,6 +7333,15 @@ } } ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ClickPipePostRequest" + } + } + } + }, "responses": { "200": { "description": "Successful response", @@ -7312,11 +7424,13 @@ "tags": [ "ClickPipes" ] - }, - "patch": { - "summary": "Update ClickPipe", - "description": "Update the specified ClickPipe.", - "operationId": "clickPipeUpdate", + } + }, + "/v1/organizations/{organizationId}/services/{serviceId}/clickpipes/{clickPipeId}": { + "get": { + "summary": "Get ClickPipe", + "description": "Returns the specified ClickPipe.", + "operationId": "clickPipeGet", "parameters": [ { "in": "path", @@ -7331,7 +7445,7 @@ { "in": "path", "name": "serviceId", - "description": "ID of the service to create the ClickPipe for.", + "description": "ID of the service that owns the ClickPipe.", "required": true, "schema": { "type": "string", @@ -7349,15 +7463,6 @@ } } ], - "requestBody": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/ClickPipePatchRequest" - } - } - } - }, "responses": { "200": { "description": "Successful response", @@ -7441,10 +7546,10 @@ "ClickPipes" ] }, - "delete": { - "summary": "Delete ClickPipe", - "description": "Delete the specified ClickPipe.", - "operationId": "clickPipeDelete", + "patch": { + "summary": "Update ClickPipe", + "description": "Update the specified ClickPipe. Source fields not present in the per-source update schemas are immutable after creation. For Kafka sources, values submitted for immutable fields (type, format, brokers, topics, consumerGroup, offset, schemaRegistry, exactlyOnce) are not applied, except schema registry credentials, which are rejected.", + "operationId": "clickPipeUpdate", "parameters": [ { "in": "path", @@ -7459,7 +7564,7 @@ { "in": "path", "name": "serviceId", - "description": "ID of the service that owns the ClickPipe.", + "description": "ID of the service to create the ClickPipe for.", "required": true, "schema": { "type": "string", @@ -7469,7 +7574,7 @@ { "in": "path", "name": "clickPipeId", - "description": "ID of the ClickPipe to delete.", + "description": "ID of the requested ClickPipe.", "required": true, "schema": { "type": "string", @@ -7477,6 +7582,15 @@ } } ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ClickPipePatchRequest" + } + } + } + }, "responses": { "200": { "description": "Successful response", @@ -7494,6 +7608,9 @@ "type": "string", "description": "Unique id assigned to every request. UUIDv4", "format": "uuid" + }, + "result": { + "$ref": "#/components/schemas/ClickPipe" } } } @@ -7556,13 +7673,11 @@ "tags": [ "ClickPipes" ] - } - }, - "/v1/organizations/{organizationId}/services/{serviceId}/clickpipes/{clickPipeId}/settings": { - "get": { - "summary": "Get ClickPipe settings", - "description": "Returns the advanced settings for the specified ClickPipe.", - "operationId": "clickPipeSettingsGet", + }, + "delete": { + "summary": "Delete ClickPipe", + "description": "Delete the specified ClickPipe.", + "operationId": "clickPipeDelete", "parameters": [ { "in": "path", @@ -7587,7 +7702,7 @@ { "in": "path", "name": "clickPipeId", - "description": "ID of the ClickPipe to get settings for.", + "description": "ID of the ClickPipe to delete.", "required": true, "schema": { "type": "string", @@ -7612,9 +7727,127 @@ "type": "string", "description": "Unique id assigned to every request. UUIDv4", "format": "uuid" - }, - "result": { - "$ref": "#/components/schemas/ClickPipeSettings" + } + } + } + } + } + }, + "400": { + "description": "The request cannot be processed due to a client error. Please verify your request parameters and try again.", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "status": { + "type": "number", + "description": "HTTP status code.", + "example": 400 + }, + "error": { + "type": "string", + "description": "Detailed error description." + }, + "requestId": { + "type": "string", + "description": "Unique id assigned to every request. UUIDv4", + "format": "uuid" + } + } + } + } + } + }, + "500": { + "description": "An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance.", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "status": { + "type": "integer", + "description": "HTTP status code.", + "example": 500 + }, + "error": { + "type": "string", + "description": "Detailed error description." + }, + "requestId": { + "type": "string", + "description": "Unique id assigned to every request. UUIDv4", + "format": "uuid" + } + } + } + } + } + } + }, + "tags": [ + "ClickPipes" + ] + } + }, + "/v1/organizations/{organizationId}/services/{serviceId}/clickpipes/{clickPipeId}/settings": { + "get": { + "summary": "Get ClickPipe settings", + "description": "Returns the advanced settings for the specified ClickPipe.", + "operationId": "clickPipeSettingsGet", + "parameters": [ + { + "in": "path", + "name": "organizationId", + "description": "ID of the organization that owns the service.", + "required": true, + "schema": { + "type": "string", + "format": "uuid" + } + }, + { + "in": "path", + "name": "serviceId", + "description": "ID of the service that owns the ClickPipe.", + "required": true, + "schema": { + "type": "string", + "format": "uuid" + } + }, + { + "in": "path", + "name": "clickPipeId", + "description": "ID of the ClickPipe to get settings for.", + "required": true, + "schema": { + "type": "string", + "format": "uuid" + } + } + ], + "responses": { + "200": { + "description": "Successful response", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "status": { + "type": "number", + "description": "HTTP status code.", + "example": 200 + }, + "requestId": { + "type": "string", + "description": "Unique id assigned to every request. UUIDv4", + "format": "uuid" + }, + "result": { + "$ref": "#/components/schemas/ClickPipeSettings" } } } @@ -7810,7 +8043,7 @@ "/v1/organizations/{organizationId}/services/{serviceId}/clickpipes/schemaDiscovery": { "post": { "summary": "Discover ClickPipe source schema", - "description": "**This endpoint is in beta.** API contract is stable, and no breaking changes are expected in the future.

Infers the schema (field names and ClickHouse data types) of a streaming ClickPipe source without creating a pipe. Supported for Kafka, Kinesis sources.", + "description": "**This endpoint is in beta.** API contract is stable, and no breaking changes are expected in the future.

Infers the schema (field names and ClickHouse data types) of a ClickPipe source without creating a pipe. Supported for Kafka, Kinesis, Pub/Sub, and object storage sources. Object storage inference runs on the destination service, which must be running.", "operationId": "clickPipeSchemaDiscovery", "parameters": [ { @@ -9175,7 +9408,7 @@ "/v1/organizations/{organizationId}/services/{serviceId}/clickstack/alerts": { "get": { "summary": "ClickStack: List Alerts", - "description": "**This endpoint is in beta.** API contract is stable, and no breaking changes are expected in the future.

ClickStack: Retrieves alerts for the authenticated team (paginated). Results are capped at `limit` (default and maximum 1000). When more records exist than are returned, `meta.total` exceeds `data.length`; clients with large collections must page with `limit`/`offset` to retrieve them all.", + "description": "**This endpoint is in beta.** API contract is stable, and no breaking changes are expected in the future.

ClickStack: Retrieves alerts for the authenticated team (paginated). Results are capped at `limit` (default and maximum 1000). When `totalCount` exceeds the number of returned items, page with `limit`/`offset` to retrieve them all.", "operationId": "clickStackListAlerts", "parameters": [ { @@ -9197,6 +9430,27 @@ "type": "string", "format": "uuid" } + }, + { + "in": "query", + "name": "limit", + "description": "Maximum number of results to return.", + "schema": { + "type": "integer", + "minimum": 1, + "maximum": 1000, + "default": 1000 + } + }, + { + "in": "query", + "name": "offset", + "description": "Number of results to skip before returning.", + "schema": { + "type": "integer", + "minimum": 0, + "default": 0 + } } ], "responses": { @@ -10423,7 +10677,7 @@ "/v1/organizations/{organizationId}/services/{serviceId}/clickstack/webhooks": { "get": { "summary": "ClickStack: List Webhooks", - "description": "**This endpoint is in beta.** API contract is stable, and no breaking changes are expected in the future.

ClickStack: Retrieves webhooks for the authenticated team (paginated). Results are capped at `limit` (default and maximum 1000). When more records exist than are returned, `meta.total` exceeds `data.length`; clients with large collections must page with `limit`/`offset` to retrieve them all.", + "description": "**This endpoint is in beta.** API contract is stable, and no breaking changes are expected in the future.

ClickStack: Retrieves webhooks for the authenticated team (paginated). Results are capped at `limit` (default and maximum 1000). When `totalCount` exceeds the number of returned items, page with `limit`/`offset` to retrieve them all.", "operationId": "clickStackListWebhooks", "parameters": [ { @@ -10445,6 +10699,27 @@ "type": "string", "format": "uuid" } + }, + { + "in": "query", + "name": "limit", + "description": "Maximum number of results to return.", + "schema": { + "type": "integer", + "minimum": 1, + "maximum": 1000, + "default": 1000 + } + }, + { + "in": "query", + "name": "offset", + "description": "Number of results to skip before returning.", + "schema": { + "type": "integer", + "minimum": 0, + "default": 0 + } } ], "responses": { @@ -11547,7 +11822,7 @@ "/v1/organizations/{organizationId}/services/{serviceId}/clickstack/saved-searches": { "get": { "summary": "ClickStack: List Saved Searches", - "description": "**This endpoint is in beta.** API contract is stable, and no breaking changes are expected in the future.

ClickStack: Retrieves saved searches for the authenticated team (paginated). Results are capped at `limit` (default and maximum 1000). When more records exist than are returned, `meta.total` exceeds `data.length`; clients with large collections must page with `limit`/`offset` to retrieve them all.", + "description": "**This endpoint is in beta.** API contract is stable, and no breaking changes are expected in the future.

ClickStack: Retrieves saved searches for the authenticated team (paginated). Results are capped at `limit` (default and maximum 1000). When `totalCount` exceeds the number of returned items, page with `limit`/`offset` to retrieve them all.", "operationId": "clickStackListSavedSearches", "parameters": [ { @@ -11569,6 +11844,27 @@ "type": "string", "format": "uuid" } + }, + { + "in": "query", + "name": "limit", + "description": "Maximum number of results to return.", + "schema": { + "type": "integer", + "minimum": 1, + "maximum": 1000, + "default": 1000 + } + }, + { + "in": "query", + "name": "offset", + "description": "Number of results to skip before returning.", + "schema": { + "type": "integer", + "minimum": 0, + "default": 0 + } } ], "responses": { @@ -14464,6 +14760,196 @@ ] } }, + "/v1/organizations/{organizationId}/postgres/{postgresId}/logs": { + "get": { + "summary": "List Postgres server logs", + "description": "**This endpoint is in beta.** API contract is stable, and no breaking changes are expected in the future.

Returns PostgreSQL server log entries for a Postgres service within the given time window, most recent first by default (override with `sort_order`). Results are paginated with `limit`/`offset`; advance `offset` until a page returns fewer than `limit` entries to read the full window. The time range must not exceed 30 days, and `to_date` must be after `from_date`.", + "operationId": "postgresLogsGetList", + "parameters": [ + { + "in": "path", + "name": "organizationId", + "description": "ID of the organization that owns the Postgres service.", + "required": true, + "schema": { + "type": "string", + "format": "uuid" + } + }, + { + "in": "path", + "name": "postgresId", + "description": "ID of the requested Postgres service.", + "required": true, + "schema": { + "type": "string", + "format": "uuid" + } + }, + { + "in": "query", + "name": "from_date", + "description": "Inclusive start of the time window (RFC 3339 date-time).", + "schema": { + "type": "string", + "format": "date-time" + }, + "required": true + }, + { + "in": "query", + "name": "to_date", + "description": "Inclusive end of the time window (RFC 3339 date-time).", + "schema": { + "type": "string", + "format": "date-time" + }, + "required": true + }, + { + "in": "query", + "name": "body_contains", + "description": "Case-sensitive substring the log body must contain.", + "schema": { + "type": "string" + } + }, + { + "in": "query", + "name": "severity", + "description": "Filter to log entries with this PostgreSQL severity (for example, ERROR, WARNING, LOG).", + "schema": { + "type": "string" + } + }, + { + "in": "query", + "name": "sort_order", + "description": "Sort order. One of `asc` or `desc`.", + "schema": { + "type": "string", + "enum": [ + "asc", + "desc" + ], + "default": "desc" + } + }, + { + "in": "query", + "name": "limit", + "description": "Maximum number of results to return.", + "schema": { + "type": "integer", + "minimum": 1, + "maximum": 2000, + "default": 50 + } + }, + { + "in": "query", + "name": "offset", + "description": "Number of results to skip before returning.", + "schema": { + "type": "integer", + "minimum": 0, + "default": 0 + } + } + ], + "responses": { + "200": { + "description": "Successful response", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "status": { + "type": "number", + "description": "HTTP status code.", + "example": 200 + }, + "requestId": { + "type": "string", + "description": "Unique id assigned to every request. UUIDv4", + "format": "uuid" + }, + "result": { + "type": "array", + "items": { + "$ref": "#/components/schemas/PostgresLogEntry" + } + } + } + } + } + } + }, + "400": { + "description": "The request cannot be processed due to a client error. Please verify your request parameters and try again.", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "status": { + "type": "number", + "description": "HTTP status code.", + "example": 400 + }, + "error": { + "type": "string", + "description": "Detailed error description." + }, + "requestId": { + "type": "string", + "description": "Unique id assigned to every request. UUIDv4", + "format": "uuid" + } + } + } + } + } + }, + "500": { + "description": "An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance.", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "status": { + "type": "integer", + "description": "HTTP status code.", + "example": 500 + }, + "error": { + "type": "string", + "description": "Detailed error description." + }, + "requestId": { + "type": "string", + "description": "Unique id assigned to every request. UUIDv4", + "format": "uuid" + } + } + } + } + } + } + }, + "tags": [ + "Postgres" + ], + "x-badges": [ + { + "name": "Beta", + "position": "after" + } + ] + } + }, "/v1/organizations/{organizationId}/privateEndpointConfig": { "get": { "summary": "Get private endpoint configuration for region within cloud provider for an organization", @@ -19416,6 +19902,51 @@ } } }, + "PrometheusDiscoveryLabels": { + "properties": { + "__scheme__": { + "description": "URL scheme Prometheus must scrape the target with.", + "type": "string" + }, + "__metrics_path__": { + "description": "Path of the per-service Prometheus metrics endpoint.", + "type": "string" + }, + "__param_filtered_metrics": { + "description": "Value passed as the filtered_metrics query parameter on each scrape.", + "type": "string", + "format": "boolean" + }, + "clickhouse_org_id": { + "description": "Organization ID the service belongs to.", + "type": "string", + "format": "uuid" + }, + "clickhouse_service_id": { + "description": "Service ID.", + "type": "string", + "format": "uuid" + }, + "clickhouse_discovery_service_name": { + "description": "Service name.", + "type": "string" + } + } + }, + "PrometheusDiscoveryTargetGroup": { + "properties": { + "targets": { + "type": "array", + "description": "Host (and port) of the ClickHouse Cloud API.", + "items": { + "type": "string" + } + }, + "labels": { + "$ref": "#/components/schemas/PrometheusDiscoveryLabels" + } + } + }, "OrganizationPatchPrivateEndpoint": { "properties": { "id": { @@ -22263,7 +22794,7 @@ "ClickPipeMongoDBSource": { "properties": { "uri": { - "description": "MongoDB connection URI. Supports both standard URIs (mongodb://...) and SRV URIs (mongodb+srv://...).", + "description": "MongoDB connection URI. Supports both standard URIs (mongodb://...) and SRV URIs (mongodb+srv://...). Embedded credentials are redacted from API responses, so the returned value can differ from what was submitted.", "type": "string", "example": "mongodb+srv://cluster0.example.mongodb.net/mydb" }, @@ -22321,7 +22852,7 @@ "$ref": "#/components/schemas/PLAIN" }, "uri": { - "description": "MongoDB connection URI. Supports both standard URIs (mongodb://...) and SRV URIs (mongodb+srv://...).", + "description": "MongoDB connection URI. Supports both standard URIs (mongodb://...) and SRV URIs (mongodb+srv://...). Embedded credentials are redacted from API responses, so the returned value can differ from what was submitted.", "type": "string", "example": "mongodb+srv://cluster0.example.mongodb.net/mydb" }, @@ -22381,7 +22912,7 @@ "$ref": "#/components/schemas/PLAIN" }, "uri": { - "description": "MongoDB connection URI. Supports both standard URIs (mongodb://...) and SRV URIs (mongodb+srv://...).", + "description": "MongoDB connection URI. Supports both standard URIs (mongodb://...) and SRV URIs (mongodb+srv://...). Embedded credentials are redacted from API responses, so the returned value can differ from what was submitted.", "type": [ "string", "null" @@ -22979,6 +23510,11 @@ "maximum": 2, "example": 2 }, + "kafka_read_committed": { + "description": "Kafka Read Committed. Whether Kafka consumers read only committed messages", + "type": "boolean", + "example": false + }, "object_storage_use_cluster_function": { "description": "use cluster function. Whether to use ClickHouse cluster function for distributed processing", "type": [ @@ -23387,6 +23923,11 @@ "maximum": 2, "example": 2 }, + "kafka_read_committed": { + "description": "Kafka Read Committed. Whether Kafka consumers read only committed messages", + "type": "boolean", + "example": false + }, "object_storage_use_cluster_function": { "description": "use cluster function. Whether to use ClickHouse cluster function for distributed processing", "type": [ @@ -23426,6 +23967,26 @@ "type": "null" } ] + }, + "pubsub": { + "oneOf": [ + { + "$ref": "#/components/schemas/ClickPipePostPubSubSource" + }, + { + "type": "null" + } + ] + }, + "objectStorage": { + "oneOf": [ + { + "$ref": "#/components/schemas/ClickPipePostObjectStorageSource" + }, + { + "type": "null" + } + ] } } }, @@ -23494,6 +24055,7 @@ "type": "string", "enum": [ "create_organization", + "delete_organization", "organization_update_name", "transfer_service_in", "transfer_service_out", @@ -23535,7 +24097,6 @@ "service_update_autoscaling_replicas", "service_update_max_allowable_replicas", "service_update_backup_configuration", - "service_update_snapshot_configuration", "service_restore_backup", "service_update_release_channel", "service_update_gpt_usage_consent", @@ -23545,7 +24106,72 @@ "service_maintenance_start", "service_maintenance_end", "service_update_core_dump", - "backup_delete" + "service_update_autoscaling_schedule", + "service_update_query_endpoints", + "service_update_direct_connection", + "service_update_sql_console_jwt_auth", + "service_update_snapshot_configuration", + "service_update_collector_ip_access_list", + "service_update_mysql_interface", + "service_update_upgrade_window", + "service_delete_upgrade_window", + "service_trigger_failover", + "service_trigger_recovery", + "service_mcp_enabled", + "service_mcp_disabled", + "service_upgrade", + "service_scaled_down_for_tier_change", + "service_encryption_key_check_failed", + "service_encryption_key_rotation_failed", + "service_encryption_key_rotated", + "service_stop_encryption_key_inaccessible", + "service_restart_encryption_key_rotation", + "backup_delete", + "backup_bucket_create", + "backup_bucket_update", + "backup_bucket_delete", + "backup_bucket_archive", + "warehouse_update_name", + "warehouse_update_release_channel", + "role_create", + "role_update", + "role_delete", + "role_resources_delete", + "organization_member_remove_roles", + "scim_user_profile_update", + "scim_group_create", + "scim_group_update", + "scim_group_delete", + "organization_saml_connection_delete", + "datadog_integration_create", + "datadog_integration_delete", + "organization_update_spend_alert", + "organization_update_core_dumps", + "organization_update_private_endpoints", + "organization_update_pci_compliance", + "organization_update_hipaa_status", + "transfer_credits_in", + "transfer_credits_out", + "promo_code_claim", + "schema_advisor_seed", + "schema_advisor_generate_plan", + "schema_advisor_approve_plan", + "schema_advisor_start_deployment", + "schema_advisor_start_benchmark", + "schema_advisor_run_benchmark", + "schema_advisor_start_promotion", + "schema_advisor_exchange_tables", + "schema_advisor_drop_sandbox", + "udf_create", + "udf_update", + "udf_delete", + "udf_version_create", + "udf_version_delete", + "udf_attach", + "udf_detach", + "udf_update_services", + "udf_redeploy", + "udf_rebuild" ] }, "actorType": { @@ -23602,6 +24228,34 @@ "service-role-changed", "roles-v2-changed" ] + }, + "targetRoleIds": { + "type": "array", + "description": "For role and actor-role activities: IDs of the affected roles.", + "items": { + "type": "string" + } + }, + "targetRoleNames": { + "type": "array", + "description": "For role and actor-role activities: names of the affected roles, when recorded.", + "items": { + "type": "string" + } + }, + "targetActorIds": { + "type": "array", + "description": "For 'organization_member_update_roles' and 'organization_member_remove_roles' activities: IDs of the affected actors (e.g. 'user/').", + "items": { + "type": "string" + } + }, + "targetResourceIds": { + "type": "array", + "description": "For 'role_resources_delete' activities: IDs of the deleted resources the roles referenced.", + "items": { + "type": "string" + } } } }, @@ -30358,7 +31012,8 @@ "enum": [ "services-per-organization", "postgres-services-per-organization", - "replicas-per-warehouse" + "replicas-per-warehouse", + "api-keys-per-organization" ], "example": "services-per-organization" }, @@ -30406,6 +31061,52 @@ "adjustable" ] }, + "ActiveBalance": { + "properties": { + "id": { + "description": "Unique ID of the prepaid balance.", + "type": "string", + "format": "uuid" + }, + "remainingPrepaidCredits": { + "description": "Remaining credits available on this balance, in ClickHouse Credits (CHCs).", + "type": "number" + }, + "totalAmount": { + "description": "Total credits granted on this balance, in ClickHouse Credits (CHCs).", + "type": "number" + }, + "amountSpent": { + "description": "Credits spent from this balance, in ClickHouse Credits (CHCs).", + "type": "number" + }, + "startDate": { + "description": "Date the balance became active. ISO-8601, based on the UTC timezone.", + "type": "string", + "format": "date-time" + }, + "expirationDate": { + "description": "Date the balance expires. ISO-8601, based on the UTC timezone.", + "type": "string", + "format": "date-time" + } + } + }, + "ActiveBalances": { + "properties": { + "totalRemainingPrepaidCredits": { + "description": "Total remaining credits across all active prepaid balances, in ClickHouse Credits (CHCs).", + "type": "number" + }, + "prepaidBalances": { + "type": "array", + "description": "List of active prepaid balances for the organization.", + "items": { + "$ref": "#/components/schemas/ActiveBalance" + } + } + } + }, "ServiceClickhouseSetting": { "properties": { "name": { @@ -31810,6 +32511,28 @@ "recentExecutions" ] }, + "PostgresLogEntry": { + "properties": { + "timestamp": { + "description": "Time the entry was logged (RFC 3339).", + "type": "string", + "format": "date-time" + }, + "severity": { + "description": "PostgreSQL severity of the entry (for example, LOG, WARNING, ERROR, FATAL, PANIC).", + "type": "string" + }, + "body": { + "description": "Raw log entry body as emitted by PostgreSQL. Structured bodies are returned as a JSON-encoded string.", + "type": "string" + } + }, + "required": [ + "timestamp", + "severity", + "body" + ] + }, "UpgradeWindow": { "properties": { "weekday": { @@ -33125,6 +33848,19 @@ "type": "integer", "exclusiveMinimum": 0 }, + "memoryLimitMib": { + "default": null, + "anyOf": [ + { + "type": "integer", + "minimum": 1, + "maximum": 1048576 + }, + { + "type": "null" + } + ] + }, "sendChunkHeader": { "default": false, "type": "boolean" @@ -33246,6 +33982,19 @@ "type": "integer", "exclusiveMinimum": 0 }, + "memoryLimitMib": { + "default": null, + "anyOf": [ + { + "type": "integer", + "minimum": 1, + "maximum": 1048576 + }, + { + "type": "null" + } + ] + }, "sendChunkHeader": { "default": false, "type": "boolean" @@ -33366,6 +34115,19 @@ "type": "integer", "exclusiveMinimum": 0 }, + "memoryLimitMib": { + "default": null, + "anyOf": [ + { + "type": "integer", + "minimum": 1, + "maximum": 1048576 + }, + { + "type": "null" + } + ] + }, "sendChunkHeader": { "default": false, "type": "boolean" @@ -33482,6 +34244,19 @@ "type": "integer", "exclusiveMinimum": 0 }, + "memoryLimitMib": { + "default": null, + "anyOf": [ + { + "type": "integer", + "minimum": 1, + "maximum": 1048576 + }, + { + "type": "null" + } + ] + }, "sendChunkHeader": { "default": false, "type": "boolean" @@ -33743,6 +34518,20 @@ } ] }, + "memoryLimitMib": { + "description": "Maximum memory, in MiB, available to each UDF sandbox process. Null uses the sandbox default.", + "default": null, + "anyOf": [ + { + "type": "integer", + "minimum": 1, + "maximum": 1048576 + }, + { + "type": "null" + } + ] + }, "sendChunkHeader": { "description": "Whether ClickHouse sends a row-count chunk header.", "type": "boolean" @@ -33803,6 +34592,7 @@ "commandReadTimeout", "commandWriteTimeout", "maxCommandExecutionTime", + "memoryLimitMib", "sendChunkHeader", "format", "sandboxType", diff --git a/crates/clickhouse-cloud-api/src/client/organizations.rs b/crates/clickhouse-cloud-api/src/client/organizations.rs index 66098e9..ae9c17d 100644 --- a/crates/clickhouse-cloud-api/src/client/organizations.rs +++ b/crates/clickhouse-cloud-api/src/client/organizations.rs @@ -3,6 +3,36 @@ use crate::error::Error; use crate::models::*; impl Client { + /// Get organization active prepaid balances + pub async fn active_balances_get( + &self, + organization_id: &str, + limit: Option, + offset: Option, + ) -> Result, Error> { + let path = format!("/v1/organizations/{organization_id}/activeBalances"); + let mut req = self.request(reqwest::Method::GET, &path); + if let Some(v) = limit { + req = req.query(&[("limit", v)]); + } + if let Some(v) = offset { + req = req.query(&[("offset", v)]); + } + let resp = req.send().await?; + let status = resp.status(); + let body_text = resp.text().await?; + if !status.is_success() { + return Err(Error::Api { + status: status.as_u16(), + message: serde_json::from_str::>(&body_text) + .ok() + .and_then(|r| r.error) + .unwrap_or(body_text.clone()), + }); + } + Ok(serde_json::from_str(&body_text)?) + } + /// Get list of available organizations pub async fn organization_get_list(&self) -> Result>, Error> { let path = "/v1/organizations".to_string(); @@ -544,6 +574,32 @@ impl Client { Ok(resp.text().await?) } + /// Discover Prometheus scrape targets for an organization + pub async fn organization_prometheus_discovery_get( + &self, + organization_id: &str, + filtered_metrics: Option<&str>, + ) -> Result, Error> { + let path = format!("/v1/organizations/{organization_id}/prometheus/discovery"); + let mut req = self.request(reqwest::Method::GET, &path); + if let Some(v) = filtered_metrics { + req = req.query(&[("filtered_metrics", v)]); + } + let resp = req.send().await?; + let status = resp.status(); + let body_text = resp.text().await?; + if !status.is_success() { + return Err(Error::Api { + status: status.as_u16(), + message: serde_json::from_str::>(&body_text) + .ok() + .and_then(|r| r.error) + .unwrap_or(body_text.clone()), + }); + } + Ok(serde_json::from_str(&body_text)?) + } + /// Get organization usage costs pub async fn usage_cost_get( &self, diff --git a/crates/clickhouse-cloud-api/src/client/postgres.rs b/crates/clickhouse-cloud-api/src/client/postgres.rs index b0de939..3326e97 100644 --- a/crates/clickhouse-cloud-api/src/client/postgres.rs +++ b/crates/clickhouse-cloud-api/src/client/postgres.rs @@ -363,6 +363,53 @@ impl Client { Ok(serde_json::from_str(&body_text)?) } + /// List Postgres logs + #[allow(clippy::too_many_arguments)] + pub async fn postgres_logs_get_list( + &self, + organization_id: &str, + postgres_id: &str, + from_date: &str, + to_date: &str, + body_contains: Option<&str>, + severity: Option<&str>, + sort_order: Option<&PostgresLogsGetListSortorder>, + limit: Option, + offset: Option, + ) -> Result>, Error> { + let path = format!("/v1/organizations/{organization_id}/postgres/{postgres_id}/logs"); + let mut req = self.request(reqwest::Method::GET, &path); + req = req.query(&[("from_date", from_date), ("to_date", to_date)]); + if let Some(v) = body_contains { + req = req.query(&[("body_contains", v)]); + } + if let Some(v) = severity { + req = req.query(&[("severity", v)]); + } + if let Some(v) = sort_order { + req = req.query(&[("sort_order", v)]); + } + if let Some(v) = limit { + req = req.query(&[("limit", v)]); + } + if let Some(v) = offset { + req = req.query(&[("offset", v)]); + } + let resp = req.send().await?; + let status = resp.status(); + let body_text = resp.text().await?; + if !status.is_success() { + return Err(Error::Api { + status: status.as_u16(), + message: serde_json::from_str::>(&body_text) + .ok() + .and_then(|r| r.error) + .unwrap_or(body_text.clone()), + }); + } + Ok(serde_json::from_str(&body_text)?) + } + /// Get Postgres metrics #[allow(clippy::too_many_arguments)] pub async fn postgres_instance_metrics_get( diff --git a/crates/clickhouse-cloud-api/src/meta.rs b/crates/clickhouse-cloud-api/src/meta.rs index 663a929..4417076 100644 --- a/crates/clickhouse-cloud-api/src/meta.rs +++ b/crates/clickhouse-cloud-api/src/meta.rs @@ -20,6 +20,7 @@ /// Snake-case operation IDs (matching [`crate::client::Client`] method names) /// that the OpenAPI spec marks Beta via `x-badges`. pub const BETA_OPERATIONS: &[&str] = &[ + "active_balances_get", "backup_bucket_create", "backup_bucket_delete", "backup_bucket_get", @@ -55,6 +56,7 @@ pub const BETA_OPERATIONS: &[&str] = &[ "click_stack_update_source", "click_stack_update_webhook", "click_stack_validate_dashboard", + "organization_prometheus_discovery_get", "organization_quota_get", "organization_quotas_get_list", "postgres_instance_config_get", @@ -64,6 +66,7 @@ pub const BETA_OPERATIONS: &[&str] = &[ "postgres_instance_metrics_get", "postgres_instance_prometheus_get", "postgres_instance_restore", + "postgres_logs_get_list", "postgres_org_prometheus_get", "postgres_service_certs_get", "postgres_service_create", diff --git a/crates/clickhouse-cloud-api/src/models.rs b/crates/clickhouse-cloud-api/src/models.rs index ea8054c..5458fb3 100644 --- a/crates/clickhouse-cloud-api/src/models.rs +++ b/crates/clickhouse-cloud-api/src/models.rs @@ -181,19 +181,23 @@ pub use organization_private_endpoints::{ OrganizationPrivateEndpoint, OrganizationPrivateEndpointCloudprovider, OrganizationPrivateEndpointRegion, OrganizationPrivateEndpointsPatch, }; -pub use organizations::{Organization, OrganizationPatchRequest}; +pub use organizations::{ + ActiveBalance, ActiveBalances, Organization, OrganizationPatchRequest, + PrometheusDiscoveryLabels, PrometheusDiscoveryTargetGroup, +}; pub use postgres::{ BasePostgresService, PgBouncerConfig, PgBouncerConfigResponse, PgConfig, PgConfigDefaultTransactionIsolation, PgConfigResponse, PgConfigSslMinProtocolVersion, PgConfigWalCompression, PgCreatedAtProperty, PgHaType, PgIdProperty, PgIsPrimaryProperty, PgNameProperty, PgPassword, PgPitrRestoreTargetProperty, PgProvider, PgRegion, PgSize, PgStateProperty, PgStorageSize, PgTags, PgTagsResponse, PgVersion, PostgresInstanceConfig, - PostgresInstanceConfigResponse, PostgresInstanceUpdateConfigResponse, PostgresMetric, - PostgresMetricDataPoint, PostgresMetricSeries, PostgresMetrics, PostgresQueryExecution, - PostgresService, PostgresServiceListItem, PostgresServicePasswordResource, - PostgresServicePatchRequest, PostgresServicePostRequest, PostgresServiceReadReplicaRequest, - PostgresServiceRestoreRequest, PostgresServiceSetPassword, PostgresServiceSetState, - PostgresServiceSetStateCommand, PostgresSlowQueryPattern, PostgresSlowQueryPatternDetail, + PostgresInstanceConfigResponse, PostgresInstanceUpdateConfigResponse, PostgresLogEntry, + PostgresLogsGetListSortorder, PostgresMetric, PostgresMetricDataPoint, PostgresMetricSeries, + PostgresMetrics, PostgresQueryExecution, PostgresService, PostgresServiceListItem, + PostgresServicePasswordResource, PostgresServicePatchRequest, PostgresServicePostRequest, + PostgresServiceReadReplicaRequest, PostgresServiceRestoreRequest, PostgresServiceSetPassword, + PostgresServiceSetState, PostgresServiceSetStateCommand, PostgresSlowQueryPattern, + PostgresSlowQueryPatternDetail, }; pub use quotas::{OrganizationQuota, OrganizationQuotaQuotacode, OrganizationQuotaScope}; pub use rbac::{ diff --git a/crates/clickhouse-cloud-api/src/models/activity.rs b/crates/clickhouse-cloud-api/src/models/activity.rs index c2bcd93..3851fb6 100644 --- a/crates/clickhouse-cloud-api/src/models/activity.rs +++ b/crates/clickhouse-cloud-api/src/models/activity.rs @@ -188,6 +188,136 @@ pub enum ActivityType { Service_update_core_dump, #[serde(rename = "backup_delete")] Backup_delete, + #[serde(rename = "backup_bucket_archive")] + Backup_bucket_archive, + #[serde(rename = "backup_bucket_create")] + Backup_bucket_create, + #[serde(rename = "backup_bucket_delete")] + Backup_bucket_delete, + #[serde(rename = "backup_bucket_update")] + Backup_bucket_update, + #[serde(rename = "datadog_integration_create")] + Datadog_integration_create, + #[serde(rename = "datadog_integration_delete")] + Datadog_integration_delete, + #[serde(rename = "delete_organization")] + Delete_organization, + #[serde(rename = "organization_member_remove_roles")] + Organization_member_remove_roles, + #[serde(rename = "organization_saml_connection_delete")] + Organization_saml_connection_delete, + #[serde(rename = "organization_update_core_dumps")] + Organization_update_core_dumps, + #[serde(rename = "organization_update_hipaa_status")] + Organization_update_hipaa_status, + #[serde(rename = "organization_update_pci_compliance")] + Organization_update_pci_compliance, + #[serde(rename = "organization_update_private_endpoints")] + Organization_update_private_endpoints, + #[serde(rename = "organization_update_spend_alert")] + Organization_update_spend_alert, + #[serde(rename = "promo_code_claim")] + Promo_code_claim, + #[serde(rename = "role_create")] + Role_create, + #[serde(rename = "role_delete")] + Role_delete, + #[serde(rename = "role_resources_delete")] + Role_resources_delete, + #[serde(rename = "role_update")] + Role_update, + #[serde(rename = "schema_advisor_approve_plan")] + Schema_advisor_approve_plan, + #[serde(rename = "schema_advisor_drop_sandbox")] + Schema_advisor_drop_sandbox, + #[serde(rename = "schema_advisor_exchange_tables")] + Schema_advisor_exchange_tables, + #[serde(rename = "schema_advisor_generate_plan")] + Schema_advisor_generate_plan, + #[serde(rename = "schema_advisor_run_benchmark")] + Schema_advisor_run_benchmark, + #[serde(rename = "schema_advisor_seed")] + Schema_advisor_seed, + #[serde(rename = "schema_advisor_start_benchmark")] + Schema_advisor_start_benchmark, + #[serde(rename = "schema_advisor_start_deployment")] + Schema_advisor_start_deployment, + #[serde(rename = "schema_advisor_start_promotion")] + Schema_advisor_start_promotion, + #[serde(rename = "scim_group_create")] + Scim_group_create, + #[serde(rename = "scim_group_delete")] + Scim_group_delete, + #[serde(rename = "scim_group_update")] + Scim_group_update, + #[serde(rename = "scim_user_profile_update")] + Scim_user_profile_update, + #[serde(rename = "service_delete_upgrade_window")] + Service_delete_upgrade_window, + #[serde(rename = "service_encryption_key_check_failed")] + Service_encryption_key_check_failed, + #[serde(rename = "service_encryption_key_rotated")] + Service_encryption_key_rotated, + #[serde(rename = "service_encryption_key_rotation_failed")] + Service_encryption_key_rotation_failed, + #[serde(rename = "service_mcp_disabled")] + Service_mcp_disabled, + #[serde(rename = "service_mcp_enabled")] + Service_mcp_enabled, + #[serde(rename = "service_restart_encryption_key_rotation")] + Service_restart_encryption_key_rotation, + #[serde(rename = "service_scaled_down_for_tier_change")] + Service_scaled_down_for_tier_change, + #[serde(rename = "service_stop_encryption_key_inaccessible")] + Service_stop_encryption_key_inaccessible, + #[serde(rename = "service_trigger_failover")] + Service_trigger_failover, + #[serde(rename = "service_trigger_recovery")] + Service_trigger_recovery, + #[serde(rename = "service_update_autoscaling_schedule")] + Service_update_autoscaling_schedule, + #[serde(rename = "service_update_collector_ip_access_list")] + Service_update_collector_ip_access_list, + #[serde(rename = "service_update_direct_connection")] + Service_update_direct_connection, + #[serde(rename = "service_update_mysql_interface")] + Service_update_mysql_interface, + #[serde(rename = "service_update_query_endpoints")] + Service_update_query_endpoints, + #[serde(rename = "service_update_sql_console_jwt_auth")] + Service_update_sql_console_jwt_auth, + #[serde(rename = "service_update_upgrade_window")] + Service_update_upgrade_window, + #[serde(rename = "service_upgrade")] + Service_upgrade, + #[serde(rename = "transfer_credits_in")] + Transfer_credits_in, + #[serde(rename = "transfer_credits_out")] + Transfer_credits_out, + #[serde(rename = "udf_attach")] + Udf_attach, + #[serde(rename = "udf_create")] + Udf_create, + #[serde(rename = "udf_delete")] + Udf_delete, + #[serde(rename = "udf_detach")] + Udf_detach, + #[serde(rename = "udf_rebuild")] + Udf_rebuild, + #[serde(rename = "udf_redeploy")] + Udf_redeploy, + #[serde(rename = "udf_update")] + Udf_update, + #[serde(rename = "udf_update_services")] + Udf_update_services, + #[serde(rename = "udf_version_create")] + Udf_version_create, + #[serde(rename = "udf_version_delete")] + Udf_version_delete, + #[serde(rename = "warehouse_update_name")] + Warehouse_update_name, + #[serde(rename = "warehouse_update_release_channel")] + Warehouse_update_release_channel, /// Catch-all for unknown or newly-added values. #[serde(untagged)] Unknown(String), @@ -273,6 +403,125 @@ impl std::fmt::Display for ActivityType { Self::Service_maintenance_end => write!(f, "service_maintenance_end"), Self::Service_update_core_dump => write!(f, "service_update_core_dump"), Self::Backup_delete => write!(f, "backup_delete"), + Self::Backup_bucket_archive => write!(f, "backup_bucket_archive"), + Self::Backup_bucket_create => write!(f, "backup_bucket_create"), + Self::Backup_bucket_delete => write!(f, "backup_bucket_delete"), + Self::Backup_bucket_update => write!(f, "backup_bucket_update"), + Self::Datadog_integration_create => write!(f, "datadog_integration_create"), + Self::Datadog_integration_delete => write!(f, "datadog_integration_delete"), + Self::Delete_organization => write!(f, "delete_organization"), + Self::Organization_member_remove_roles => { + write!(f, "organization_member_remove_roles") + } + Self::Organization_saml_connection_delete => { + write!(f, "organization_saml_connection_delete") + } + Self::Organization_update_core_dumps => { + write!(f, "organization_update_core_dumps") + } + Self::Organization_update_hipaa_status => { + write!(f, "organization_update_hipaa_status") + } + Self::Organization_update_pci_compliance => { + write!(f, "organization_update_pci_compliance") + } + Self::Organization_update_private_endpoints => { + write!(f, "organization_update_private_endpoints") + } + Self::Organization_update_spend_alert => { + write!(f, "organization_update_spend_alert") + } + Self::Promo_code_claim => write!(f, "promo_code_claim"), + Self::Role_create => write!(f, "role_create"), + Self::Role_delete => write!(f, "role_delete"), + Self::Role_resources_delete => write!(f, "role_resources_delete"), + Self::Role_update => write!(f, "role_update"), + Self::Schema_advisor_approve_plan => write!(f, "schema_advisor_approve_plan"), + Self::Schema_advisor_drop_sandbox => write!(f, "schema_advisor_drop_sandbox"), + Self::Schema_advisor_exchange_tables => { + write!(f, "schema_advisor_exchange_tables") + } + Self::Schema_advisor_generate_plan => write!(f, "schema_advisor_generate_plan"), + Self::Schema_advisor_run_benchmark => { + write!(f, "schema_advisor_run_benchmark") + } + Self::Schema_advisor_seed => write!(f, "schema_advisor_seed"), + Self::Schema_advisor_start_benchmark => { + write!(f, "schema_advisor_start_benchmark") + } + Self::Schema_advisor_start_deployment => { + write!(f, "schema_advisor_start_deployment") + } + Self::Schema_advisor_start_promotion => { + write!(f, "schema_advisor_start_promotion") + } + Self::Scim_group_create => write!(f, "scim_group_create"), + Self::Scim_group_delete => write!(f, "scim_group_delete"), + Self::Scim_group_update => write!(f, "scim_group_update"), + Self::Scim_user_profile_update => write!(f, "scim_user_profile_update"), + Self::Service_delete_upgrade_window => { + write!(f, "service_delete_upgrade_window") + } + Self::Service_encryption_key_check_failed => { + write!(f, "service_encryption_key_check_failed") + } + Self::Service_encryption_key_rotated => { + write!(f, "service_encryption_key_rotated") + } + Self::Service_encryption_key_rotation_failed => { + write!(f, "service_encryption_key_rotation_failed") + } + Self::Service_mcp_disabled => write!(f, "service_mcp_disabled"), + Self::Service_mcp_enabled => write!(f, "service_mcp_enabled"), + Self::Service_restart_encryption_key_rotation => { + write!(f, "service_restart_encryption_key_rotation") + } + Self::Service_scaled_down_for_tier_change => { + write!(f, "service_scaled_down_for_tier_change") + } + Self::Service_stop_encryption_key_inaccessible => { + write!(f, "service_stop_encryption_key_inaccessible") + } + Self::Service_trigger_failover => write!(f, "service_trigger_failover"), + Self::Service_trigger_recovery => write!(f, "service_trigger_recovery"), + Self::Service_update_autoscaling_schedule => { + write!(f, "service_update_autoscaling_schedule") + } + Self::Service_update_collector_ip_access_list => { + write!(f, "service_update_collector_ip_access_list") + } + Self::Service_update_direct_connection => { + write!(f, "service_update_direct_connection") + } + Self::Service_update_mysql_interface => { + write!(f, "service_update_mysql_interface") + } + Self::Service_update_query_endpoints => { + write!(f, "service_update_query_endpoints") + } + Self::Service_update_sql_console_jwt_auth => { + write!(f, "service_update_sql_console_jwt_auth") + } + Self::Service_update_upgrade_window => { + write!(f, "service_update_upgrade_window") + } + Self::Service_upgrade => write!(f, "service_upgrade"), + Self::Transfer_credits_in => write!(f, "transfer_credits_in"), + Self::Transfer_credits_out => write!(f, "transfer_credits_out"), + Self::Udf_attach => write!(f, "udf_attach"), + Self::Udf_create => write!(f, "udf_create"), + Self::Udf_delete => write!(f, "udf_delete"), + Self::Udf_detach => write!(f, "udf_detach"), + Self::Udf_rebuild => write!(f, "udf_rebuild"), + Self::Udf_redeploy => write!(f, "udf_redeploy"), + Self::Udf_update => write!(f, "udf_update"), + Self::Udf_update_services => write!(f, "udf_update_services"), + Self::Udf_version_create => write!(f, "udf_version_create"), + Self::Udf_version_delete => write!(f, "udf_version_delete"), + Self::Warehouse_update_name => write!(f, "warehouse_update_name"), + Self::Warehouse_update_release_channel => { + write!(f, "warehouse_update_release_channel") + } Self::Unknown(s) => write!(f, "{s}"), } } @@ -299,8 +548,16 @@ pub struct Activity { pub organization_id: Option, #[serde(rename = "serviceId", skip_serializing_if = "Option::is_none")] pub service_id: Option, + #[serde(rename = "targetActorIds", skip_serializing_if = "Option::is_none")] + pub target_actor_ids: Option>, #[serde(rename = "targetKeyId", skip_serializing_if = "Option::is_none")] pub target_key_id: Option, + #[serde(rename = "targetResourceIds", skip_serializing_if = "Option::is_none")] + pub target_resource_ids: Option>, + #[serde(rename = "targetRoleIds", skip_serializing_if = "Option::is_none")] + pub target_role_ids: Option>, + #[serde(rename = "targetRoleNames", skip_serializing_if = "Option::is_none")] + pub target_role_names: Option>, #[serde(skip_serializing_if = "Option::is_none")] pub r#type: Option, #[serde(rename = "userAgent", skip_serializing_if = "Option::is_none")] diff --git a/crates/clickhouse-cloud-api/src/models/clickpipes.rs b/crates/clickhouse-cloud-api/src/models/clickpipes.rs index a3d7c19..7fc1848 100644 --- a/crates/clickhouse-cloud-api/src/models/clickpipes.rs +++ b/crates/clickhouse-cloud-api/src/models/clickpipes.rs @@ -2945,6 +2945,10 @@ pub struct ClickPipeSchemaDiscoverySource { pub kafka: Option, #[serde(skip_serializing_if = "Option::is_none")] pub kinesis: Option, + #[serde(rename = "objectStorage", skip_serializing_if = "Option::is_none")] + pub object_storage: Option, + #[serde(rename = "pubsub", skip_serializing_if = "Option::is_none")] + pub pubsub: Option, } /// `ClickPipePostObjectStorageSource` from the ClickHouse Cloud API. @@ -3315,6 +3319,8 @@ pub struct ClickPipeSettings { pub clickhouse_parallel_distributed_insert_select: Option, #[serde(skip_serializing_if = "Option::is_none")] pub clickhouse_parallel_view_processing: Option, + #[serde(rename = "kafka_read_committed")] + pub kafka_read_committed: bool, #[serde(skip_serializing_if = "Option::is_none")] pub object_storage_concurrency: Option, #[serde(skip_serializing_if = "Option::is_none")] @@ -3348,6 +3354,11 @@ pub struct ClickPipeSettingsResponse { pub clickhouse_parallel_distributed_insert_select: Option, #[serde(skip_serializing_if = "Option::is_none")] pub clickhouse_parallel_view_processing: Option, + #[serde( + rename = "kafka_read_committed", + skip_serializing_if = "Option::is_none" + )] + pub kafka_read_committed: Option, #[serde(skip_serializing_if = "Option::is_none")] pub object_storage_concurrency: Option, #[serde(skip_serializing_if = "Option::is_none")] @@ -3377,6 +3388,8 @@ pub struct ClickPipeSettingsPutRequest { pub clickhouse_parallel_distributed_insert_select: Option, #[serde(skip_serializing_if = "Option::is_none")] pub clickhouse_parallel_view_processing: Option, + #[serde(rename = "kafka_read_committed")] + pub kafka_read_committed: bool, #[serde(skip_serializing_if = "Option::is_none")] pub object_storage_concurrency: Option, #[serde(skip_serializing_if = "Option::is_none")] diff --git a/crates/clickhouse-cloud-api/src/models/organizations.rs b/crates/clickhouse-cloud-api/src/models/organizations.rs index f3c3eaf..a78cf34 100644 --- a/crates/clickhouse-cloud-api/src/models/organizations.rs +++ b/crates/clickhouse-cloud-api/src/models/organizations.rs @@ -1,5 +1,73 @@ use super::{ByocConfig, OrganizationPrivateEndpoint, OrganizationPrivateEndpointsPatch}; use serde::{Deserialize, Serialize}; + +/// `ActiveBalance` from the ClickHouse Cloud API. +#[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize)] +pub struct ActiveBalance { + #[serde(rename = "amountSpent", skip_serializing_if = "Option::is_none")] + pub amount_spent: Option, + #[serde(rename = "expirationDate", skip_serializing_if = "Option::is_none")] + pub expiration_date: Option>, + #[serde(rename = "id", skip_serializing_if = "Option::is_none")] + pub id: Option, + #[serde( + rename = "remainingPrepaidCredits", + skip_serializing_if = "Option::is_none" + )] + pub remaining_prepaid_credits: Option, + #[serde(rename = "startDate", skip_serializing_if = "Option::is_none")] + pub start_date: Option>, + #[serde(rename = "totalAmount", skip_serializing_if = "Option::is_none")] + pub total_amount: Option, +} + +/// `ActiveBalances` from the ClickHouse Cloud API. +#[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize)] +pub struct ActiveBalances { + #[serde(rename = "prepaidBalances", skip_serializing_if = "Option::is_none")] + pub prepaid_balances: Option>, + #[serde( + rename = "totalRemainingPrepaidCredits", + skip_serializing_if = "Option::is_none" + )] + pub total_remaining_prepaid_credits: Option, +} + +/// `PrometheusDiscoveryLabels` from the ClickHouse Cloud API. +#[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize)] +pub struct PrometheusDiscoveryLabels { + #[serde(rename = "__metrics_path__", skip_serializing_if = "Option::is_none")] + pub metrics_path: Option, + #[serde( + rename = "__param_filtered_metrics", + skip_serializing_if = "Option::is_none" + )] + pub param_filtered_metrics: Option, + #[serde(rename = "__scheme__", skip_serializing_if = "Option::is_none")] + pub scheme: Option, + #[serde( + rename = "clickhouse_discovery_service_name", + skip_serializing_if = "Option::is_none" + )] + pub clickhouse_discovery_service_name: Option, + #[serde(rename = "clickhouse_org_id", skip_serializing_if = "Option::is_none")] + pub clickhouse_org_id: Option, + #[serde( + rename = "clickhouse_service_id", + skip_serializing_if = "Option::is_none" + )] + pub clickhouse_service_id: Option, +} + +/// `PrometheusDiscoveryTargetGroup` from the ClickHouse Cloud API. +#[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize)] +pub struct PrometheusDiscoveryTargetGroup { + #[serde(rename = "labels", skip_serializing_if = "Option::is_none")] + pub labels: Option, + #[serde(rename = "targets", skip_serializing_if = "Option::is_none")] + pub targets: Option>, +} + /// `Organization` from the ClickHouse Cloud API. #[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize)] pub struct Organization { diff --git a/crates/clickhouse-cloud-api/src/models/postgres.rs b/crates/clickhouse-cloud-api/src/models/postgres.rs index 6044677..2443655 100644 --- a/crates/clickhouse-cloud-api/src/models/postgres.rs +++ b/crates/clickhouse-cloud-api/src/models/postgres.rs @@ -57,6 +57,29 @@ impl PgProvider { pub const VALUES: &'static [&'static str] = &["aws"]; } +/// Inline enum for `postgresLogsGetList.sort_order`. +#[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize)] +pub enum PostgresLogsGetListSortorder { + #[serde(rename = "asc")] + Asc, + #[serde(rename = "desc")] + #[default] + Desc, + /// Catch-all for unknown or newly-added values. + #[serde(untagged)] + Unknown(String), +} + +impl std::fmt::Display for PostgresLogsGetListSortorder { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::Asc => write!(f, "asc"), + Self::Desc => write!(f, "desc"), + Self::Unknown(s) => write!(f, "{s}"), + } + } +} + /// `pgSize` enum from the ClickHouse Cloud API. #[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize)] pub enum PgSize { @@ -688,6 +711,17 @@ pub struct PostgresServiceSetState { pub command: PostgresServiceSetStateCommand, } +/// `PostgresLogEntry` from the ClickHouse Cloud API. +#[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize)] +pub struct PostgresLogEntry { + #[serde(rename = "timestamp", skip_serializing_if = "Option::is_none")] + pub timestamp: Option>, + #[serde(rename = "severity", skip_serializing_if = "Option::is_none")] + pub severity: Option, + #[serde(rename = "body", skip_serializing_if = "Option::is_none")] + pub body: Option, +} + /// `PostgresMetricDataPoint` from the ClickHouse Cloud API. #[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize)] pub struct PostgresMetricDataPoint { diff --git a/crates/clickhouse-cloud-api/src/models/quotas.rs b/crates/clickhouse-cloud-api/src/models/quotas.rs index 4155f79..12af47f 100644 --- a/crates/clickhouse-cloud-api/src/models/quotas.rs +++ b/crates/clickhouse-cloud-api/src/models/quotas.rs @@ -7,6 +7,8 @@ pub enum OrganizationQuotaQuotacode { Services_per_organization, #[serde(rename = "postgres-services-per-organization")] Postgres_services_per_organization, + #[serde(rename = "api-keys-per-organization")] + Api_keys_per_organization, #[serde(rename = "replicas-per-warehouse")] Replicas_per_warehouse, /// Catch-all for unknown or newly-added values. @@ -21,6 +23,7 @@ impl std::fmt::Display for OrganizationQuotaQuotacode { Self::Postgres_services_per_organization => { write!(f, "postgres-services-per-organization") } + Self::Api_keys_per_organization => write!(f, "api-keys-per-organization"), Self::Replicas_per_warehouse => write!(f, "replicas-per-warehouse"), Self::Unknown(s) => write!(f, "{s}"), } diff --git a/crates/clickhouse-cloud-api/src/models/udfs.rs b/crates/clickhouse-cloud-api/src/models/udfs.rs index a056b7e..41fce9b 100644 --- a/crates/clickhouse-cloud-api/src/models/udfs.rs +++ b/crates/clickhouse-cloud-api/src/models/udfs.rs @@ -110,6 +110,8 @@ pub struct Udf { skip_serializing_if = "Option::is_none" )] pub max_command_execution_time: Option, + #[serde(rename = "memoryLimitMib", skip_serializing_if = "Option::is_none")] + pub memory_limit_mib: Option, #[serde(rename = "poolSize", skip_serializing_if = "Option::is_none")] pub pool_size: Option, #[serde(rename = "returnName", skip_serializing_if = "Option::is_none")] diff --git a/crates/clickhouse-cloud-api/tests/client_test.rs b/crates/clickhouse-cloud-api/tests/client_test.rs index 644592b..7ac81dd 100644 --- a/crates/clickhouse-cloud-api/tests/client_test.rs +++ b/crates/clickhouse-cloud-api/tests/client_test.rs @@ -94,6 +94,35 @@ async fn get_organization() { assert_eq!(org.name.as_deref(), Some("My Org")); } +#[tokio::test] +async fn get_active_balances_with_pagination() { + let (s, c) = setup().await; + + Mock::given(method("GET")) + .and(path("/v1/organizations/org-1/activeBalances")) + .and(query_param("limit", "25")) + .and(query_param("offset", "50")) + .respond_with(ok_json(serde_json::json!({ + "totalRemainingPrepaidCredits": 12.5, + "prepaidBalances": [{ + "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", + "remainingPrepaidCredits": 12.5, + "expirationDate": "2027-01-01T00:00:00Z" + }] + }))) + .mount(&s) + .await; + + let balances = c + .active_balances_get("org-1", Some(25), Some(50)) + .await + .unwrap() + .result + .unwrap(); + assert_eq!(balances.total_remaining_prepaid_credits, Some(12.5)); + assert_eq!(balances.prepaid_balances.unwrap().len(), 1); +} + #[tokio::test] async fn update_organization() { let (s, c) = setup().await; @@ -167,6 +196,43 @@ async fn get_prometheus_metrics() { assert!(resp.contains("ch_metric")); } +#[tokio::test] +async fn discover_organization_prometheus_targets() { + let (s, c) = setup().await; + + Mock::given(method("GET")) + .and(path("/v1/organizations/org-1/prometheus/discovery")) + .and(query_param("filtered_metrics", "false")) + .respond_with( + ResponseTemplate::new(200).set_body_json(serde_json::json!([{ + "targets": ["api.clickhouse.cloud"], + "labels": { + "__scheme__": "https", + "__metrics_path__": "/v1/organizations/org-1/services/svc-1/prometheus", + "__param_filtered_metrics": "false", + "clickhouse_org_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", + "clickhouse_service_id": "b1b2c3d4-e5f6-7890-abcd-ef1234567890", + "clickhouse_discovery_service_name": "analytics" + } + }])), + ) + .mount(&s) + .await; + + let groups = c + .organization_prometheus_discovery_get("org-1", Some("false")) + .await + .unwrap(); + assert_eq!(groups.len(), 1); + assert_eq!( + groups[0] + .labels + .as_ref() + .and_then(|labels| labels.scheme.as_deref()), + Some("https") + ); +} + #[tokio::test] #[allow(deprecated)] async fn get_private_endpoint_config() { @@ -1586,6 +1652,7 @@ async fn update_click_pipe_settings() { .await; let body = ClickPipeSettingsPutRequest { + kafka_read_committed: true, ..Default::default() }; let resp = c @@ -1626,6 +1693,8 @@ async fn click_pipe_schema_discovery_kafka() { ..Default::default() }), kinesis: None, + object_storage: None, + pubsub: None, }, }; let resp = c @@ -3106,6 +3175,47 @@ async fn get_quota() { // PostgreSQL Services // =========================================================================== +#[tokio::test] +async fn list_postgres_logs_with_filters() { + let (s, c) = setup().await; + + Mock::given(method("GET")) + .and(path("/v1/organizations/org-1/postgres/pg-1/logs")) + .and(query_param("from_date", "2026-08-01T00:00:00Z")) + .and(query_param("to_date", "2026-08-02T00:00:00Z")) + .and(query_param("body_contains", "checkpoint")) + .and(query_param("severity", "LOG")) + .and(query_param("sort_order", "asc")) + .and(query_param("limit", "100")) + .and(query_param("offset", "20")) + .respond_with(ok_json(serde_json::json!([{ + "timestamp": "2026-08-01T12:00:00Z", + "severity": "LOG", + "body": "checkpoint complete" + }]))) + .mount(&s) + .await; + + let logs = c + .postgres_logs_get_list( + "org-1", + "pg-1", + "2026-08-01T00:00:00Z", + "2026-08-02T00:00:00Z", + Some("checkpoint"), + Some("LOG"), + Some(&PostgresLogsGetListSortorder::Asc), + Some(100), + Some(20), + ) + .await + .unwrap() + .result + .unwrap(); + assert_eq!(logs.len(), 1); + assert_eq!(logs[0].body.as_deref(), Some("checkpoint complete")); +} + #[tokio::test] async fn create_postgres_service() { let (s, c) = setup().await; diff --git a/crates/clickhouse-cloud-api/tests/model_facade_test.rs b/crates/clickhouse-cloud-api/tests/model_facade_test.rs index b70358e..220d75c 100644 --- a/crates/clickhouse-cloud-api/tests/model_facade_test.rs +++ b/crates/clickhouse-cloud-api/tests/model_facade_test.rs @@ -4,6 +4,10 @@ fn assert_same_type(_: T, _: T) {} #[test] fn extracted_models_keep_root_and_models_paths() { + assert_same_type( + api::ActiveBalances::default(), + api::models::ActiveBalances::default(), + ); assert_same_type(api::Activity::default(), api::models::Activity::default()); assert_same_type(api::ApiKey::default(), api::models::ApiKey::default()); assert_same_type( @@ -49,6 +53,18 @@ fn extracted_models_keep_root_and_models_paths() { api::PostgresInstanceConfig::default(), api::models::PostgresInstanceConfig::default(), ); + assert_same_type( + api::PostgresLogEntry::default(), + api::models::PostgresLogEntry::default(), + ); + assert_same_type( + api::PostgresLogsGetListSortorder::default(), + api::models::PostgresLogsGetListSortorder::default(), + ); + assert_same_type( + api::PrometheusDiscoveryTargetGroup::default(), + api::models::PrometheusDiscoveryTargetGroup::default(), + ); assert_same_type(api::RBACRole::default(), api::models::RBACRole::default()); assert_same_type(api::ScimUser::default(), api::models::ScimUser::default()); assert_same_type( diff --git a/crates/clickhouse-cloud-api/tests/models_test.rs b/crates/clickhouse-cloud-api/tests/models_test.rs index 09086e7..5294a9e 100644 --- a/crates/clickhouse-cloud-api/tests/models_test.rs +++ b/crates/clickhouse-cloud-api/tests/models_test.rs @@ -382,12 +382,14 @@ fn deserialize_clickpipe_settings() { let json = r#"{ "streaming_max_insert_wait_ms": 5000, "object_storage_concurrency": null, - "clickhouse_max_threads": 4 + "clickhouse_max_threads": 4, + "kafka_read_committed": true }"#; let settings: ClickPipeSettings = serde_json::from_str(json).unwrap(); assert_eq!(settings.streaming_max_insert_wait_ms, Some(5000)); assert_eq!(settings.object_storage_concurrency, None); assert_eq!(settings.clickhouse_max_threads, Some(4)); + assert!(settings.kafka_read_committed); } #[test] @@ -1328,7 +1330,7 @@ fn schema_discovery_response_tolerates_dropped_and_null_meta() { fn udf_responses_tolerate_dropped_and_null_fields() { let dropped: Udf = serde_json::from_str("{}").unwrap(); let nulled: Udf = serde_json::from_str( - r#"{"functionName":null,"runtime":null,"arguments":null,"createdAt":null}"#, + r#"{"functionName":null,"runtime":null,"arguments":null,"createdAt":null,"memoryLimitMib":null}"#, ) .unwrap(); @@ -3458,11 +3460,34 @@ fn click_pipe_schema_discovery_request_kafka_source() { source: ClickPipeSchemaDiscoverySource { kafka: Some(ClickPipePostKafkaSource::default()), kinesis: None, + object_storage: None, + pubsub: None, }, }; let v = serde_json::to_value(&req).unwrap(); assert!(v["source"]["kafka"].is_object()); assert!(v["source"].get("kinesis").is_none()); + assert!(v["source"].get("objectStorage").is_none()); + assert!(v["source"].get("pubsub").is_none()); +} + +#[test] +fn click_pipe_schema_discovery_request_supports_new_sources() { + let object_storage = ClickPipeSchemaDiscoveryRequest { + source: ClickPipeSchemaDiscoverySource { + object_storage: Some(ClickPipePostObjectStorageSource::default()), + ..Default::default() + }, + }; + let pubsub = ClickPipeSchemaDiscoveryRequest { + source: ClickPipeSchemaDiscoverySource { + pubsub: Some(ClickPipePostPubSubSource::default()), + ..Default::default() + }, + }; + + assert!(serde_json::to_value(object_storage).unwrap()["source"]["objectStorage"].is_object()); + assert!(serde_json::to_value(pubsub).unwrap()["source"]["pubsub"].is_object()); } #[test] @@ -3748,8 +3773,10 @@ fn shared_clickpipe_nested_types_stay_strict_on_the_request_side() { serde_json::from_str::("{}").unwrap(), ClickPipePostgresPipeTableMappingResponse::default() ); - // `ClickPipeSettings` is an all-optional schema in both directions, so the - // split is visible only in the type name the settings endpoints return. + // The new non-nullable Kafka setting is required in requests while the + // response variant remains tolerant of a dropped key. + assert!(serde_json::from_str::("{}").is_err()); + assert!(serde_json::from_str::("{}").is_err()); assert_eq!( serde_json::from_str::("{}").unwrap(), ClickPipeSettingsResponse::default() @@ -3787,6 +3814,10 @@ fn activity_type_new_wire_values_deserialize_to_typed_variants() { "service_update_snapshot_configuration", ActivityType::Service_update_snapshot_configuration, ), + ("backup_bucket_create", ActivityType::Backup_bucket_create), + ("role_update", ActivityType::Role_update), + ("service_mcp_enabled", ActivityType::Service_mcp_enabled), + ("udf_create", ActivityType::Udf_create), ]; for (wire, expected) in cases { let parsed: ActivityType = serde_json::from_str(&format!("\"{wire}\"")).unwrap(); @@ -5249,6 +5280,51 @@ fn organization_quota_typed_enums_round_trip() { assert_eq!(back, quota); } +#[test] +fn organization_api_key_quota_code_round_trips() { + let parsed: OrganizationQuotaQuotacode = + serde_json::from_str("\"api-keys-per-organization\"").unwrap(); + assert_eq!( + parsed, + OrganizationQuotaQuotacode::Api_keys_per_organization + ); + assert_eq!(parsed.to_string(), "api-keys-per-organization"); + assert_eq!( + serde_json::to_value(parsed).unwrap(), + "api-keys-per-organization" + ); +} + +#[test] +fn new_response_models_tolerate_absent_and_null_fields() { + let balances: ActiveBalances = + serde_json::from_str(r#"{"totalRemainingPrepaidCredits":null,"prepaidBalances":null}"#) + .unwrap(); + let balance: ActiveBalance = serde_json::from_str( + r#"{"id":null,"remainingPrepaidCredits":null,"totalAmount":null,"amountSpent":null,"startDate":null,"expirationDate":null}"#, + ) + .unwrap(); + let labels: PrometheusDiscoveryLabels = serde_json::from_str( + r#"{"__scheme__":null,"__metrics_path__":null,"__param_filtered_metrics":null,"clickhouse_org_id":null,"clickhouse_service_id":null,"clickhouse_discovery_service_name":null}"#, + ) + .unwrap(); + let group: PrometheusDiscoveryTargetGroup = + serde_json::from_str(r#"{"targets":null,"labels":null}"#).unwrap(); + let log: PostgresLogEntry = + serde_json::from_str(r#"{"timestamp":null,"severity":null,"body":null}"#).unwrap(); + + assert_eq!(balances, ActiveBalances::default()); + assert_eq!(balance, ActiveBalance::default()); + assert_eq!(labels, PrometheusDiscoveryLabels::default()); + assert_eq!(group, PrometheusDiscoveryTargetGroup::default()); + assert_eq!(log, PostgresLogEntry::default()); + assert_eq!( + serde_json::to_value(balances).unwrap(), + serde_json::json!({}) + ); + assert_eq!(serde_json::to_value(log).unwrap(), serde_json::json!({})); +} + #[test] fn organization_quota_usage_optional_omitted() { let json = r#"{ diff --git a/crates/clickhousectl/src/cloud/clickpipes.rs b/crates/clickhousectl/src/cloud/clickpipes.rs index ab56ad9..af5518c 100644 --- a/crates/clickhousectl/src/cloud/clickpipes.rs +++ b/crates/clickhousectl/src/cloud/clickpipes.rs @@ -1440,10 +1440,14 @@ async fn clickpipe_schema_discover( ClickPipeSchemaDiscoverCommands::Kafka(args) => ClickPipeSchemaDiscoverySource { kafka: Some(build_kafka_source(args)?), kinesis: None, + object_storage: None, + pubsub: None, }, ClickPipeSchemaDiscoverCommands::Kinesis(args) => ClickPipeSchemaDiscoverySource { kafka: None, kinesis: Some(build_kinesis_source(args)?), + object_storage: None, + pubsub: None, }, }; @@ -1647,6 +1651,7 @@ async fn clickpipe_settings_update( clickhouse_max_insert_threads: clickhouse_max_insert_threads.map(i64::from), object_storage_use_cluster_function, clickhouse_parallel_view_processing, + kafka_read_committed: false, clickhouse_max_download_threads: None, clickhouse_min_insert_block_size_bytes: None, clickhouse_parallel_distributed_insert_select: None,