Merged
Conversation
…ST manipulation The previous implementation used string position matching (finding WHERE, ORDER BY, LIMIT keywords) and format! interpolation to splice block_num filters into user-provided SQL. This was vulnerable to structural SQL injection where crafted queries could exploit the naive keyword matching (e.g. WHERE inside string literals, UNION bypasses). Replace with sqlparser AST parsing and manipulation: - Parse user SQL into AST, requiring a single simple SELECT statement - Determine filter column from the FROM table (num for blocks, block_num for others) - Safely AND the block filter into the existing WHERE clause (or add one) - Serialize modified AST back to SQL Also: - Reject UNION/INTERSECT/set operations in live mode (ambiguous filtering) - Return Result<String, ApiError> instead of String for proper error handling - Add Display impl for ApiError - Add tests for UNION rejection, non-SELECT rejection, and WHERE keyword in string literals Amp-Thread-ID: https://ampcode.com/threads/T-019c3272-f632-763c-8078-504a90852a67 Co-authored-by: Amp <amp@ampcode.com>
Replace the blocklist-only approach with a table allowlist so API users can only query: blocks, txs, logs, receipts, token_holders, token_balances, and CTE-defined tables. Previously, users could query sync_state (internal), pg_tables (schema enumeration), or any other table accessible to the tidx DB user. Changes: - Allowlist in validator: only permitted tables + CTE-defined names pass - Block dblink function family (cross-database access) - Add db/api_role.sql migration creating a tidx_api read-only role with SELECT-only grants on indexed tables (defense-in-depth) - Thread CTE names through all validate_* functions - 6 new tests: sync_state rejected, pg_tables rejected, unknown table rejected, CTE tables allowed, dblink blocked, analytics tables allowed Amp-Thread-ID: https://ampcode.com/threads/T-019c3272-f632-763c-8078-504a90852a67 Co-authored-by: Amp <amp@ampcode.com>
Block three categories of attacks:
1. DoS via resource exhaustion:
- Reject WITH RECURSIVE (endless loop CTEs)
- Block generate_series() (billion-row generation)
- Block SELECT INTO (object creation)
2. Privilege escalation / validator bypass:
- Validate expressions inside VALUES rows (previously VALUES(pg_sleep(10))
bypassed the entire function blocklist)
- Reject TABLE statement (TABLE pg_shadow bypassed table allowlist)
- Validate GROUP BY, HAVING, JOIN ON expressions (could hide function calls)
- Walk IsNull/IsNotNull/IsTrue/IsFalse/Like expressions recursively
3. File read hardening:
- Block lo_get/lo_open/lo_close/loread/lo_creat/lo_create/lo_unlink/lo_put
- Block pg_file_read/pg_file_write/pg_file_rename/pg_file_unlink/pg_logdir_ls
- VALUES bypass closure prevents pg_read_file via VALUES(...)
11 new tests covering all vectors.
Amp-Thread-ID: https://ampcode.com/threads/T-019c3272-f632-763c-8078-504a90852a67
Co-authored-by: Amp <amp@ampcode.com>
Switch from a function blocklist (reject known-bad) to an allowlist (permit known-good only). This eliminates the risk of missing dangerous functions as PostgreSQL adds new ones. Validator changes: - ALLOWED_FUNCTIONS allowlist: ABI helpers, aggregates, scalars, string, numeric, time, window functions, and type casting - Reject ALL table functions (FROM func(...)) unconditionally - Reject unsupported TableFactor variants (catch-all _ => Err) - Remove is_dangerous_function() and is_dangerous_table_function() API role hardening (db/api_role.sql): - Deny-by-default: REVOKE ALL on tables, sequences, and functions before granting specific access - Add token_holders and token_balances to SELECT grants - CONNECTION LIMIT 64 - statement_timeout = 30s, work_mem = 64MB, temp_file_limit = 256MB 5 new tests, all 147 lib tests passing. Amp-Thread-ID: https://ampcode.com/threads/T-019c499a-f07e-73a9-9526-6c18fd511372 Co-authored-by: Amp <amp@ampcode.com>
Switch validate_expr to reject-by-default: only explicitly allowed
expression types are permitted (identifiers, literals, binary/unary ops,
CASE, CAST, BETWEEN, IN, LIKE, subqueries, SQL builtins like EXTRACT,
SUBSTRING, TRIM, etc). Unknown expression variants are rejected.
Query structure hardening:
- Reject FOR UPDATE/SHARE locking clauses
- Validate ORDER BY expressions through validate_expr
- Validate LIMIT/OFFSET: must be numeric literals, capped at 10,000
- Reject LIMIT BY (ClickHouse-specific)
- Subquery depth limit: max 4 levels of nesting
- Query size limit: max 64KB
- Validate window function OVER clause (PARTITION BY, ORDER BY)
Service layer:
- Replace string-based LIMIT detection (contains("LIMIT")) with
AST-based detection via append_limit_if_missing()
- Use HARD_LIMIT_MAX constant (10,000) across validator, service, and API
- API param clamping uses HARD_LIMIT_MAX instead of hardcoded 100,000
15 new tests, all 162 lib tests passing.
Amp-Thread-ID: https://ampcode.com/threads/T-019c499a-f07e-73a9-9526-6c18fd511372
Co-authored-by: Amp <amp@ampcode.com>
Fixes found during oracle review: - Validate function FILTER (WHERE ...) clause: previously COUNT(*) FILTER (WHERE pg_sleep(1) IS NOT NULL) bypassed the function allowlist entirely - Validate WITHIN GROUP (ORDER BY ...) expressions - Reject LIMIT NULL (effectively means no limit, bypasses cap) - Reject negative LIMIT/OFFSET values - Reject FETCH clause (FETCH FIRST N ROWS ONLY bypasses LIMIT cap) 4 new tests, all 166 lib tests passing. Amp-Thread-ID: https://ampcode.com/threads/T-019c499a-f07e-73a9-9526-6c18fd511372 Co-authored-by: Amp <amp@ampcode.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Hardens the SQL query API against injection, privilege escalation, and DoS attacks. Builds on PR #75 which rewrote
inject_block_filterand added the initial table allowlist.Continues work from Slack thread: https://tempoxyz.slack.com/archives/C0A87C21805/p1770372917363549
Changes
Function allowlist (
src/query/validator.rs)FROM func(...))_ => Err)Reject-by-default expression validation (
src/query/validator.rs)validate_exprnow rejects unknown Expr variants instead of silently acceptingLIMIT / depth / size caps
contains("LIMIT")in service layerQuery structure hardening
validate_exprAPI role hardening (
db/api_role.sql)token_holdersandtoken_balancesadded to SELECT grantsCONNECTION LIMIT 64,statement_timeout = 30s,work_mem = 64MB,temp_file_limit = 256MBTesting
24 new tests covering: function allowlist, reject-by-default expressions, LIMIT caps, depth limits, FOR UPDATE rejection, FILTER clause bypass, LIMIT NULL/negative bypass, FETCH rejection, ORDER BY validation.