Skip to content

Add deterministic local indexing smoke flow #241

Merged
IbrahimIjai merged 6 commits into
SO4-Markets:mainfrom
0xsteins:s03-indexer-local-smoke
Jun 24, 2026
Merged

Add deterministic local indexing smoke flow #241
IbrahimIjai merged 6 commits into
SO4-Markets:mainfrom
0xsteins:s03-indexer-local-smoke

Conversation

@0xsteins

@0xsteins 0xsteins commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Summary

closes #184
Adds smoke:local — one documented command that proves the SO4 contracts,
SubQuery indexer, and GraphQL layer work together end to end. It drives real
protocol actions against a local deployment and then asserts the indexer turned
those on-chain events into GraphQL entities, writing a JSON report whose first
failure names the broken layer.

bun run --cwd apps/s03-indexer build
bun run --cwd apps/s03-indexer smoke:local

What it does
Layered, timed steps recorded in .smoke/report.json:

preflight — checks stellar (+ make/docker for fresh deploys), locates the contracts repo, ensures local-only signing keys (generated + friendbot-funded).
services — verifies Soroban RPC, Horizon, and GraphQL are reachable.
contracts-deploy — validates an existing .deployed/local.env, or deploys test tokens + faucet + protocol contracts and bootstraps a market via the contract repo make targets.
frontend-config — runs sync:contracts:local so the manifest matches the deployed IDs.
indexer-runtime — rebuilds for local config and starts the Docker stack.
contract-action — oracle prices → faucet → deposit → MarketIncrease (open) → MarketDecrease (close) → optional referral.
graphql-query — waits for the indexer to catch up and asserts market/deposit/order/position/transfer counts.
Run modes: --mode auto (default), --mode fresh, --mode existing.
Reset between runs with smoke:clean (drops the Postgres volume + clears reports).

Implementation checklist
 Documented local scenario script (scripts/local-smoke.ts)
 Configurable contracts repo path (--contracts-repo / SO4_CONTRACTS_REPO)
 Fresh local network path (deploy/bootstrap run by the script)
 Pre-existing local deployment path discovered from .deployed/local.env
 Validate required local services are reachable before protocol actions
 Deploy contracts or validate an existing deployment
 Create or validate test tokens
 Bootstrap at least one market
 Submit initial oracle prices
 Create and execute a deposit
 Create and execute an order that opens a position
 Close the position (MarketDecrease)
 Optionally register and set a referral code
 Query SubQuery GraphQL and assert expected entities exist
 Save a JSON smoke report with ledgers, tx hashes, entity counts, and failures
Acceptance checklist
 One documented command tells a contributor whether indexing works
 No private keys outside local/test keys (CLI keystore + friendbot only)
 Failures identify the broken layer (deploy / action / indexer / GraphQL / config)
 Report includes market, deposit, order, position, and transfer counts
 Rerunnable without manual DB cleanup (smoke:clean, documented)
Notes
project.ts now honors INDEXER_START_LEDGER so a fresh local network backfills from an early ledger instead of the hardcoded testnet start block.
Protocol argument JSON is built directly in TypeScript (no python dependency).
Verification
 bun run --cwd apps/s03-indexer build
 bun run --cwd apps/s03-indexer smoke:local

0xsteins added 6 commits June 24, 2026 19:28
Introduce the shared scaffolding for the SO4 local indexing smoke flow:

- types.ts defines the layered step/report model so a failing run names the
  broken layer (contracts deploy, contract action, indexer runtime, GraphQL,
  or frontend config) rather than just dumping a stack trace.
- process.ts wraps child_process with capture and streaming modes plus a
  PATH binary probe used by preflight checks.
- config.ts resolves smoke configuration from CLI flags, env vars, and
  local-friendly defaults (configurable contracts repo path, source/keeper
  keys, endpoints, and run mode).
- stellar.ts wraps the stellar CLI for key resolution, friendbot funding, and
  contract invokes, plus Soroban RPC / Horizon reachability probes. All signing
  stays in the CLI keystore so no raw secret keys touch the runner.
- graphql.ts queries SubQuery for indexer metadata and entity counts, and waits
  for the indexer to catch up to a target ledger within a lag tolerance.
- report.ts provides the step runner that times each action, tags it with a
  layer, promotes the first failure, and serializes the JSON smoke report.
- deployment.ts reads and validates the contract repo's .deployed/<network>.env
  and tokens-<network>.env outputs: it checks required core contract IDs, resolves
  a market token triplet for a long/short code pair, and resolves test tokens by
  ticker. This is what lets the smoke flow detect a usable existing deployment.
- protocol.ts implements the on-chain composites the smoke flow exercises:
  fixed local oracle price submission (set_prices_simple), faucet claims,
  approvals, deposit create/execute, MarketIncrease open, MarketDecrease close,
  and optional referral register/set. Argument JSON is built directly in TS,
  removing the python dependency the shell scripts carried.
local-smoke.ts sequences the full smoke flow as layered, timed steps:
preflight (binaries, contracts repo, local signing keys with friendbot funding),
service reachability (Soroban RPC, Horizon, GraphQL), deployment (validate an
existing local deployment or deploy + bootstrap a fresh one via the contract
repo make targets), manifest sync, indexer rebuild/restart, the on-chain action
sequence (oracle prices, faucet, deposit, open, close, optional referral), and a
GraphQL assertion that the expected entities were indexed. Every phase records
artifacts and the first failure names the broken layer.

project.ts now honors INDEXER_START_LEDGER so a fresh local network can backfill
from an early ledger instead of the hardcoded testnet start block.
- Add smoke:local and smoke:clean package scripts so the flow and its reset run
  through the documented bun commands.
- clean-local.ts tears down the indexer stack, drops the Postgres volume (with a
  container fallback for root-owned bind-mount files), and clears reports so the
  smoke flow is rerunnable without manual database cleanup. --keep-db and
  --reports variants are supported.
- Ignore the .smoke report directory and the generated local contract manifest.
Add a Local Smoke Flow section covering the one-command entrypoint, the layered
steps and which protocol actions run, the auto/fresh/existing run modes, common
flags and SMOKE_* env equivalents, the JSON report (including the failure block
that names the broken layer), and how to rerun cleanly with smoke:clean.
@vercel

vercel Bot commented Jun 24, 2026

Copy link
Copy Markdown

@0xsteins is attempting to deploy a commit to the Ijai's projects Team on Vercel.

A member of the Team first needs to authorize it.

@drips-wave

drips-wave Bot commented Jun 24, 2026

Copy link
Copy Markdown

@0xsteins Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@IbrahimIjai

Copy link
Copy Markdown
Contributor

It is a god one, you passed... Link

@IbrahimIjai IbrahimIjai merged commit 2e07c7f into SO4-Markets:main Jun 24, 2026
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

#7. Add a Deterministic Local Indexing Scenario

2 participants