feat(pricing): add AWS Price List Service support#821
Conversation
|
For reviewer context — this is PR 1 of the 3-PR scope discussed in #791. Tracking issues for the remaining two:
Each PR is independently reviewable; this one does not depend on the others landing. |
|
Marked this ready for review. CI workflows are sitting in |
Implements the `AWSPriceListService.*` JSON 1.1 surface backed by a
bundled static snapshot on the classpath. Covers the five public
operations: `DescribeServices`, `GetAttributeValues`, `GetProducts`,
`ListPriceLists`, and `GetPriceListFileUrl`, with pagination (Base64
offset tokens) and the array-of-JSON-strings `PriceList` shape the
real AWS SDKs expect.
Rationale: Floci today cannot back FinOps tooling (cost detection,
budget guards, CUR parsers, etc.) because the billing APIs are
absent. Pricing is the stateless prerequisite for the Cost Explorer
and CUR services that follow — cost lines are synthesized as
`resource state x pricing snapshot`, so the snapshot data path is the
foundation.
Snapshot layout on the classpath (and mirrored on the filesystem
when `FLOCI_SERVICES_PRICING_SNAPSHOT_PATH` is set):
pricing-snapshots/services.json
pricing-snapshots/attribute-values/<ServiceCode>/<Attr>.json
pricing-snapshots/products/<ServiceCode>/<Region>.json
pricing-snapshots/price-lists/<ServiceCode>.json
The bundled snapshot is intentionally minimal — `AmazonEC2`,
`AmazonS3`, and `AWSLambda` in `us-east-1` — to exercise SDK parsing
and filter logic without bloating the image. Users needing broader
coverage point the override env var at a full snapshot generated
from the AWS Price List bulk API.
Wiring follows the existing JSON 1.1 stateless pattern (Textract,
Secrets Manager): descriptor in `ResolvedServiceCatalog`, handler
injected into `AwsJson11Controller`, service config on
`EmulatorConfig.ServicesConfig`. No new protocol, no changes to
existing services, no new runtime dependencies.
Tests: `PricingIntegrationTest` covers all five operations via
RestAssured JSON 1.1 wire-format assertions, including validation
errors, unknown-service errors, filter matching, and pagination
tokens (17 tests, all green).
Refs: https://github.com/orgs/floci-io/discussions/791
c4cbd9b to
9ed87c0
Compare
|
I tried to use this service from testcontainers and there seems to be an issue with the |
|
Thanks for the report @cfranzen — confirmed: the bundled Hotfix up at #836: registers the tree under |
|
@cfranzen — verified the fix locally against the native build (full log on #836). |
|
@cfranzen — thanks again for catching this. The original PR was working on the JVM image (and all the tests in Two GraalVM-only failures combined:
Fix in #836:
Verified locally with |
Implements the `AWSInsightsIndexService.*` JSON 1.1 surface backed by synthesizing cost and usage from Floci's existing resource state multiplied by the bundled AWS Pricing snapshot (floci-io#821). Covers the nine Cost Explorer operations the FinOps tooling ecosystem most commonly exercises: - `GetCostAndUsage` / `GetCostAndUsageWithResources` — full `TimePeriod`, `Granularity` (DAILY / MONTHLY / HOURLY), `Metrics` (UnblendedCost / BlendedCost / AmortizedCost / NetUnblendedCost / NetAmortizedCost / UsageQuantity / NormalizedUsageAmount), `GroupBy` (DIMENSION / TAG / COST_CATEGORY), and the recursive `Filter` expression tree (`And` / `Or` / `Not` / `Dimensions` / `Tags` / `CostCategories`). - `GetDimensionValues` — returns the dimension values present in the synthesized data; honors filters and search strings. - `GetTags` — returns tag keys / values across enumerated resources. - `GetReservationCoverage` / `GetReservationUtilization` — zero-totalled stubs so callers that hit these endpoints during smoke tests don't fail on `UnknownOperationException`. - `GetSavingsPlansCoverage` / `GetSavingsPlansUtilization` — same stubbed shape. - `GetCostCategories` — empty list (cost-category management is out-of-scope for this PR). `GROUP_BY=RECORD_TYPE` distinguishes `Usage`, `Credit`, `Tax`, `Refund`, `DiscountedUsage`, and the four `SavingsPlan*` variants. `Tax` / `Refund` / `DiscountedUsage` / `SavingsPlan*` are reserved for follow-up PRs; `Credit` is emitted when `FLOCI_SERVICES_CE_CREDIT_USD_MONTHLY > 0`, capped at the synthesized monthly usage so net cost never goes below zero. ## Architecture A new SPI lives in `core/common/`: public interface ResourceUsageEnumerator { Stream<UsageLine> enumerate(Instant start, Instant end, String region); } CDI auto-discovers all `@ApplicationScoped` beans implementing it. Cost Explorer iterates `Instance<ResourceUsageEnumerator>` per request, so adding a new Floci service with cost data is a matter of dropping a new enumerator bean next to the service — zero changes to `CostExplorerService`. The same enumerators will feed the upcoming CUR and BCM Data Exports Parquet writer (floci-io#823) without duplication. The bundled enumerators are: - `Ec2UsageEnumerator` — `BoxUsage:<instanceType>` * hours per running instance, priced from the Pricing snapshot. - `S3UsageEnumerator` — `TimedStorage-ByteHrs` * GB-month per bucket, priced from the snapshot. - `LambdaUsageEnumerator` — catalog-only line per function (zero quantity until invocation tracking lands in a follow-up). - `UnpricedServicesEnumerator` — emits zero-quantity catalog rows for the remaining ~30 Floci services so they appear in `GetDimensionValues SERVICE` without contributing billed cost. Each priced enumerator additionally emits a zero-quantity catalog line so the service shows up in `GetDimensionValues SERVICE` even when no resources exist. ## Tests `CostExplorerIntegrationTest` covers all nine operations via RestAssured wire-format assertions: granularity bucketing, filter-expression evaluation, group-by combinations, dimension and tag listings, RI / SP stubs, validation errors. 20 tests, all green. The full Floci test suite stays green with the additions (4030 tests run, 0 failures, 0 errors, excluding the pre-existing local-env flakes documented during PR floci-io#821). Refs: https://github.com/orgs/floci-io/discussions/791 Refs: floci-io#822
Summary
Adds the AWS Price List Service (
pricing:*/AWSPriceListService.*) emulation — the stateless prerequisite for the Cost Explorer and CUR services proposed in #791.DescribeServices,GetAttributeValues,GetProducts,ListPriceLists,GetPriceListFileUrlAwsJson11ControllerdispatchFLOCI_SERVICES_PRICING_SNAPSHOT_PATHfor custom datasetsPriceListreturned as array-of-JSON-strings — matches real wire format so SDKs parse offers directlyWhy
Per scope discussion in #791, this is PR 1 of 3. FinOps tooling (cost detection, budget guards, CUR parsers, anomaly detectors) can't currently run against Floci because the billing APIs are absent. Pricing is the smallest stateless piece and unblocks the next two PRs (
ce:*,cur:*).Design choices
AmazonEC2,AmazonS3,AWSLambdainus-east-1only. Keeps the image small; users wanting full coverage drop a Price List bulk download underFLOCI_SERVICES_PRICING_SNAPSHOT_PATH.ServiceCode,AttributeName,regionCode), plus root-containment check inSnapshotLoaderso the override directory can't be escaped.Test plan
./mvnw test -Dtest=PricingIntegrationTest— 26/26 green locally (Java 25)@QuarkusTestinstanceValidationException,InvalidParameterException,UnknownOperationException)AttributeNameandregionCodeScope boundaries (out of scope for this PR)
GetPriceListFileUrlreturns a stub URL).Refs: https://github.com/orgs/floci-io/discussions/791