Skip to content

feat(pricing): add AWS Price List Service support#821

Merged
hectorvent merged 1 commit into
floci-io:mainfrom
ShubhamDX:feat/pricing-api
May 13, 2026
Merged

feat(pricing): add AWS Price List Service support#821
hectorvent merged 1 commit into
floci-io:mainfrom
ShubhamDX:feat/pricing-api

Conversation

@ShubhamDX
Copy link
Copy Markdown
Contributor

Summary

Adds the AWS Price List Service (pricing:* / AWSPriceListService.*) emulation — the stateless prerequisite for the Cost Explorer and CUR services proposed in #791.

  • Five operations: DescribeServices, GetAttributeValues, GetProducts, ListPriceLists, GetPriceListFileUrl
  • JSON 1.1 protocol, reuses existing AwsJson11Controller dispatch
  • Backed by a bundled classpath snapshot; override via FLOCI_SERVICES_PRICING_SNAPSHOT_PATH for custom datasets
  • Pagination with Base64 offset tokens (AWS-compatible shape)
  • PriceList returned as array-of-JSON-strings — matches real wire format so SDKs parse offers directly

Why

Per scope discussion in #791, this is PR 1 of 3. FinOps tooling (cost detection, budget guards, CUR parsers, anomaly detectors) can't currently run against Floci because the billing APIs are absent. Pricing is the smallest stateless piece and unblocks the next two PRs (ce:*, cur:*).

Design choices

  • Bundled snapshot (minimal). AmazonEC2, AmazonS3, AWSLambda in us-east-1 only. Keeps the image small; users wanting full coverage drop a Price List bulk download under FLOCI_SERVICES_PRICING_SNAPSHOT_PATH.
  • No new dependencies. Snapshot is plain JSON, parsed with the existing Jackson mapper.
  • No SPI. Cost-synthesis / pluggability deferred until a second consumer emerges — keeps scope minimal per the existing stateless pattern.
  • Path-safe snapshot loader. Segment-level validation on any input used to build a filesystem path (ServiceCode, AttributeName, regionCode), plus root-containment check in SnapshotLoader so the override directory can't be escaped.
  • EffectiveDate accepts both wire shapes. AWS JSON spec uses Unix epoch seconds (number); the handler also accepts ISO-8601 strings for SDK/CLI variance.

Test plan

  • ./mvnw test -Dtest=PricingIntegrationTest — 26/26 green locally (Java 25)
  • Wire-format assertions for each operation via RestAssured against a booted @QuarkusTest instance
  • Validation-error paths (ValidationException, InvalidParameterException, UnknownOperationException)
  • Filter matching, filter input validation (missing Field/Value), pagination round-trip
  • EffectiveDate version selection (numeric epoch + ISO-8601, with multi-version AmazonEC2 us-east-1 fixture)
  • Path-traversal rejection on AttributeName and regionCode
  • CI to run the full suite

Scope boundaries (out of scope for this PR)

Refs: https://github.com/orgs/floci-io/discussions/791

@ShubhamDX
Copy link
Copy Markdown
Contributor Author

For reviewer context — this is PR 1 of the 3-PR scope discussed in #791. Tracking issues for the remaining two:

Each PR is independently reviewable; this one does not depend on the others landing.

@ShubhamDX ShubhamDX marked this pull request as ready for review May 12, 2026 18:18
@ShubhamDX
Copy link
Copy Markdown
Contributor Author

ShubhamDX commented May 12, 2026

Marked this ready for review. CI workflows are sitting in action_required state - looks like the first-time-contributor approval gate. Waiting on a maintainer to click "Approve and run". Happy to address anything CI flags.

Comment thread src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java Outdated
Implements the `AWSPriceListService.*` JSON 1.1 surface backed by a
bundled static snapshot on the classpath. Covers the five public
operations: `DescribeServices`, `GetAttributeValues`, `GetProducts`,
`ListPriceLists`, and `GetPriceListFileUrl`, with pagination (Base64
offset tokens) and the array-of-JSON-strings `PriceList` shape the
real AWS SDKs expect.

Rationale: Floci today cannot back FinOps tooling (cost detection,
budget guards, CUR parsers, etc.) because the billing APIs are
absent. Pricing is the stateless prerequisite for the Cost Explorer
and CUR services that follow — cost lines are synthesized as
`resource state x pricing snapshot`, so the snapshot data path is the
foundation.

Snapshot layout on the classpath (and mirrored on the filesystem
when `FLOCI_SERVICES_PRICING_SNAPSHOT_PATH` is set):

    pricing-snapshots/services.json
    pricing-snapshots/attribute-values/<ServiceCode>/<Attr>.json
    pricing-snapshots/products/<ServiceCode>/<Region>.json
    pricing-snapshots/price-lists/<ServiceCode>.json

The bundled snapshot is intentionally minimal — `AmazonEC2`,
`AmazonS3`, and `AWSLambda` in `us-east-1` — to exercise SDK parsing
and filter logic without bloating the image. Users needing broader
coverage point the override env var at a full snapshot generated
from the AWS Price List bulk API.

Wiring follows the existing JSON 1.1 stateless pattern (Textract,
Secrets Manager): descriptor in `ResolvedServiceCatalog`, handler
injected into `AwsJson11Controller`, service config on
`EmulatorConfig.ServicesConfig`. No new protocol, no changes to
existing services, no new runtime dependencies.

Tests: `PricingIntegrationTest` covers all five operations via
RestAssured JSON 1.1 wire-format assertions, including validation
errors, unknown-service errors, filter matching, and pagination
tokens (17 tests, all green).

Refs: https://github.com/orgs/floci-io/discussions/791
Copy link
Copy Markdown
Collaborator

@hectorvent hectorvent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ShubhamDX

@hectorvent hectorvent merged commit e0224e9 into floci-io:main May 13, 2026
10 checks passed
@ShubhamDX ShubhamDX deleted the feat/pricing-api branch May 13, 2026 07:12
@cfranzen
Copy link
Copy Markdown
Contributor

cfranzen commented May 13, 2026

I tried to use this service from testcontainers and there seems to be an issue with the PricingService.load() method. It looks to me as if this method is never called when running Floci as container. I can't see any log output from the PricingService in the logs and the load() method should always log some information. When calling the pricing service via the API, there seem to be no service codes available.

@ShubhamDX
Copy link
Copy Markdown
Contributor Author

Thanks for the report @cfranzen — confirmed: the bundled pricing-snapshots/** tree was missing from the native-image build, so the standard container had no JSON files to read. load() returns silently before logging because the classpath lookup returns null rather than going down the WARN path.

Hotfix up at #836: registers the tree under quarkus.native.resources.includes and switches to the class's own classloader (the thread-context loader can be the bootstrap loader in a native image and miss application resources). Should restore the service-code list in the container build.

@ShubhamDX
Copy link
Copy Markdown
Contributor Author

@cfranzen — verified the fix locally against the native build (full log on #836). DescribeServices now returns the expected three services (AmazonEC2, AmazonS3, AWSLambda) and GetProducts returns rate data. The missing Pricing snapshot loaded log line is now present in the container output, confirming @PostConstruct runs as intended.

@ShubhamDX
Copy link
Copy Markdown
Contributor Author

@cfranzen — thanks again for catching this. The original PR was working on the JVM image (and all the tests in PricingIntegrationTest pass under ./mvnw test, which boots Quarkus on the JVM), so the snapshot loader looked fine in CI. The bug was specific to the GraalVM native binary that ships in the standard floci/floci:latest image, which is what Testcontainers pulls by default.

Two GraalVM-only failures combined:

  1. Resource bundling. Quarkus's static resource analysis only includes classpath resources it can prove are read. The pricing snapshot path is built dynamically ("pricing-snapshots/" + relativePath), so GraalVM stripped every JSON file out of the native image. On the JVM the files are still on the classpath, so the loader worked.
  2. Classloader. Thread.currentThread().getContextClassLoader() returned the bootstrap loader inside the native image, which doesn't see application resources. On the JVM it returns the application loader, so even without (1) the read would have succeeded.

Fix in #836:

  • Register pricing-snapshots/** under quarkus.native.resources.includes so GraalVM bundles the tree.
  • Read via PricingService.class.getClassLoader() so the same code path works on JVM and native.

Verified locally with docker build -f docker/Dockerfile.native + container run — DescribeServices now returns the expected three services and the missing Pricing snapshot loaded log line shows up. Full evidence on #836.

ShubhamDX added a commit to ShubhamDX/floci that referenced this pull request May 14, 2026
Implements the `AWSInsightsIndexService.*` JSON 1.1 surface backed by
synthesizing cost and usage from Floci's existing resource state
multiplied by the bundled AWS Pricing snapshot (floci-io#821). Covers the nine
Cost Explorer operations the FinOps tooling ecosystem most commonly
exercises:

- `GetCostAndUsage` / `GetCostAndUsageWithResources` — full
  `TimePeriod`, `Granularity` (DAILY / MONTHLY / HOURLY),
  `Metrics` (UnblendedCost / BlendedCost / AmortizedCost /
  NetUnblendedCost / NetAmortizedCost / UsageQuantity /
  NormalizedUsageAmount), `GroupBy` (DIMENSION / TAG /
  COST_CATEGORY), and the recursive `Filter` expression tree
  (`And` / `Or` / `Not` / `Dimensions` / `Tags` / `CostCategories`).
- `GetDimensionValues` — returns the dimension values present in the
  synthesized data; honors filters and search strings.
- `GetTags` — returns tag keys / values across enumerated resources.
- `GetReservationCoverage` / `GetReservationUtilization` — zero-totalled
  stubs so callers that hit these endpoints during smoke tests don't
  fail on `UnknownOperationException`.
- `GetSavingsPlansCoverage` / `GetSavingsPlansUtilization` — same
  stubbed shape.
- `GetCostCategories` — empty list (cost-category management is
  out-of-scope for this PR).

`GROUP_BY=RECORD_TYPE` distinguishes `Usage`, `Credit`, `Tax`,
`Refund`, `DiscountedUsage`, and the four `SavingsPlan*` variants.
`Tax` / `Refund` / `DiscountedUsage` / `SavingsPlan*` are reserved for
follow-up PRs; `Credit` is emitted when
`FLOCI_SERVICES_CE_CREDIT_USD_MONTHLY > 0`, capped at the synthesized
monthly usage so net cost never goes below zero.

## Architecture

A new SPI lives in `core/common/`:

    public interface ResourceUsageEnumerator {
        Stream<UsageLine> enumerate(Instant start, Instant end, String region);
    }

CDI auto-discovers all `@ApplicationScoped` beans implementing it.
Cost Explorer iterates `Instance<ResourceUsageEnumerator>` per request,
so adding a new Floci service with cost data is a matter of dropping a
new enumerator bean next to the service — zero changes to
`CostExplorerService`. The same enumerators will feed the upcoming CUR
and BCM Data Exports Parquet writer (floci-io#823) without duplication.

The bundled enumerators are:

- `Ec2UsageEnumerator` — `BoxUsage:<instanceType>` * hours per running
  instance, priced from the Pricing snapshot.
- `S3UsageEnumerator` — `TimedStorage-ByteHrs` * GB-month per bucket,
  priced from the snapshot.
- `LambdaUsageEnumerator` — catalog-only line per function (zero
  quantity until invocation tracking lands in a follow-up).
- `UnpricedServicesEnumerator` — emits zero-quantity catalog rows for
  the remaining ~30 Floci services so they appear in
  `GetDimensionValues SERVICE` without contributing billed cost.

Each priced enumerator additionally emits a zero-quantity catalog
line so the service shows up in `GetDimensionValues SERVICE` even
when no resources exist.

## Tests

`CostExplorerIntegrationTest` covers all nine operations via
RestAssured wire-format assertions: granularity bucketing,
filter-expression evaluation, group-by combinations, dimension and
tag listings, RI / SP stubs, validation errors. 20 tests, all green.

The full Floci test suite stays green with the additions
(4030 tests run, 0 failures, 0 errors, excluding the pre-existing
local-env flakes documented during PR floci-io#821).

Refs: https://github.com/orgs/floci-io/discussions/791
Refs: floci-io#822
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants