v0.11.6

[sergeliatko]

ADR: Response Schema $ref Complexity Limit for fast-json-stringify

Date: 2026-02-18

Status: Implemented

Affects: Gateway code generation (emitOperationSchemas, emitRouteFiles, emitRouteFilesWithHandlers)

Related: v0.9.2 flattenAllOf (#46), #44 Circular type refs (#44)

Context

Fastify uses fast-json-stringify (fjs) to compile JSON Schema response definitions into optimized serializer functions. During compilation, fjs recursively resolves every $ref in the schema graph. For most SOAP or WSDL-derived schemas this works fine, but OTA (OpenTravel Alliance) specification types have deeply nested, wide type hierarchies where a single response type can transitively reference hundreds of sub-schemas.

The failure

When generating a Fastify gateway from the Escapia EVRN WSDL (15 operations, 368 compiled types), three operations caused RangeError: Maximum call stack size exceeded during Fastify's app.ready().

Operation	Response type	Unique $ref targets	Max depth	Result
unit-res	EVRN_UnitResRS	237	17	Stack overflow
unit-read	EVRN_UnitReadRS	237	17	Stack overflow
unit-descriptive-info	EVRN_UnitDescriptiveInfoRS	200+	16	Stack overflow
reservation-update	EVRN_ReservationRS	55	14	OK
unit-hold	EVRN_UnitHoldRS	15	6	OK

The stack overflow occurs during fjs schema compilation, not during data serialization. Even an empty response object {} triggers it because fjs compiles the full schema graph eagerly at route registration time.

History of fjs compatibility fixes in this project

This is the third time fast-json-stringify limitations have required gateway generator adaptations.

v0.8.17 to v0.8.18, Issue #44

Compiler stack overflow on circular type references during WSDL compilation. Fixed by moving cycle detection before pascal() in getOrCompileComplex(). This affected the compiler subsystem, not the gateway runtime.

v0.9.2, PR #46

allOf compositions in gateway model schemas caused fjs compilation failures. Fixed by adding flattenAllOf() in src/gateway/helpers.ts, which merges inherited schemas into flat objects with direct properties. This function runs during emitModelSchemas() and eliminates all allOf from generated gateway schemas.

This fix

Pure $ref chain depth and breadth exceeding fjs stack limits. flattenAllOf already ran correctly (zero allOf in gateway schemas), so the issue is solely about the number of distinct schemas reachable through $ref resolution.

Decision

When a response schema's $ref graph references 150 or more unique schema components, strip the response key from the Fastify route registration. The operation schema JSON file retains the full response definition for documentation and tooling. Only the generated route TypeScript code omits it, causing Fastify to fall back to JSON.stringify for response serialization.

Implementation

Three files changed.

src/gateway/helpers.ts

New measureSchemaRefComplexity(startSchemaName, allSchemas, limit) function. Walks the $ref graph from a starting OpenAPI component schema, counting unique referenced schemas. Stops early once the limit is reached. This is O(n) with early termination and no repeated visits.

src/gateway/generators.ts

emitOperationSchemas() calls the measurement for each operation's response schemas. If any response status code's schema exceeds the threshold, skipResponseSchema: true is set on OperationMetadata. Both emitRouteFiles() (stub handlers) and emitRouteFilesWithHandlers() (full handlers) check this flag and emit a destructured schema binding that strips response.

// Generated route file for complex operations:
const { response: _response, ...routeSchema } = schema as Record<string, unknown>;
fastify.route({
  schema: routeSchema,
  // ...
});

OperationMetadata interface

New optional skipResponseSchema boolean field.

Threshold rationale

The threshold of 150 unique refs was chosen based on empirical testing.

• 55 unique refs (reservation-update): compiles instantly, no issues
• 200+ unique refs (unit-descriptive-info): stack overflow
• 237 unique refs (unit-res, unit-read): stack overflow

150 provides a safety margin of approximately 3x above the largest known working schema (55) and catches all observed failures. The threshold is a named constant (REF_COMPLEXITY_LIMIT) that can be adjusted if needed.

Alternatives considered

Increase Node.js stack size (--stack-size)

Approach: run Node.js with --stack-size=4194304 (4 MB) or similar.

Pros:

• Zero code changes
• Works immediately

Rejected because:

• Fragile. The right value depends on schema complexity, which varies per WSDL. A WSDL with 500+ referenced types would need an even larger stack.
• Not portable. Requires users to configure Node.js flags in their deployment, which generated gateway code cannot control.
• Masks the symptom without addressing the structural mismatch between OTA schema complexity and fjs's recursive design.
• Generated tests would also need the flag, breaking the zero-config Vitest experience.

Inline all $ref chains into a single self-contained schema

Approach: resolve all $ref in the response schema to produce a single JSON Schema document with $defs or fully inlined definitions. fjs would not need to do recursive $ref resolution because everything is already inline.

Pros:

• Preserves fjs response serialization (performance benefit)
• Schema file is fully self-contained

Rejected because:

• Produces enormous schema files. EVRN_UnitResRS inlined would be tens of thousands of lines since many types are referenced multiple times, for example AddressType appears in dozens of contexts.
• May still exceed fjs stack limits during compilation of deeply nested inline schemas. The recursion depth would be similar, just without $ref lookups.
• Contradicts the project's design of one-schema-per-type with URN-based $ref for Fastify's addSchema() store, which is the documented Fastify pattern.
• Would require a separate inlining pass after emitModelSchemas(), adding build-time complexity.

Custom serializerCompiler per route

Approach: use Fastify's serializerCompiler route option to provide a custom serializer for complex routes, for example one based on ajv standalone mode or a depth-limited fjs compilation.

Pros:

• Per-route granularity
• Could still validate response structure

Rejected because:

• fast-json-stringify does not expose a depth-limit or iterative compilation mode. The stack overflow is an internal implementation detail.
• A custom compiler based on ajv standalone serialization would add a new dependency and significant complexity to the generated code.
• The performance benefit of optimized serialization is marginal for SOAP gateway responses. Network round-trip to the SOAP backend dominates latency, not JSON serialization.

Strip response schema for all operations unconditionally

Approach: never include response in any Fastify route schema. Always use JSON.stringify.

Pros:

• Simplest implementation. No complexity analysis needed.
• Eliminates the class of fjs issues entirely.

Rejected because:

• Loses response validation for operations where it works fine, which is the majority. Response schema validation catches bugs in envelope construction and type mismatches at development time.
• Loses fjs serialization performance for all routes, including simple ones where the overhead is meaningful relative to fast local operations.
• Overly broad. Penalizes all WSDLs because of limitations that only affect extreme OTA-style schemas.

Use fjs v6 or Fastify v5 improvements

Approach: wait for upstream fixes. fjs has had ongoing work on iterative schema compilation.

Investigated:

• Checked fast-json-stringify changelog and open issues. As of Fastify 5.x (which the generated gateway targets), fjs v6 is included. The recursive $ref resolution remains fundamentally recursive.
• No open PR or roadmap item for iterative $ref resolution was found.

Rejected because:

• No concrete timeline for upstream fix
• Users need a working solution now
• The chosen approach (selective stripping) is forward-compatible. If fjs adds iterative compilation, the threshold can be raised or removed.

Consequences

Positive

• All 159 out of 159 EVRN generated tests pass. Previously 157 out of 159.
• No user-visible behavior change for routes below the threshold. The vast majority of operations continue to use fjs optimized serialization.
• Graceful degradation. Affected routes still work correctly, just using JSON.stringify instead of fjs. Response data integrity is unchanged.
• Self-documenting. The generated route file includes a comment explaining why the response schema was omitted.
• Forward-compatible. If fjs adds iterative compilation or depth control, the threshold can be adjusted without changing the architecture.

Negative

• No response schema validation for affected routes. Malformed responses from the SOAP backend will be serialized as-is without fjs catching structural issues. Mitigated by: the gateway's buildSuccessEnvelope() wrapper enforces envelope structure, and AJV request validation still runs.
• Slight performance loss for affected routes. JSON.stringify is approximately 2x to 5x slower than fjs for large objects. Mitigated by: SOAP backend latency (100ms+) dominates total response time, making serialization overhead negligible.
• Threshold is empirical, not computed from fjs internals. A future fjs version change could shift the safe boundary. Mitigated by: the threshold (150) has a 3x safety margin over known working schemas.

Neutral

• Operation schema JSON files remain complete, including response. Only the TypeScript route registration strips it. Tools that read the JSON files (OpenAPI viewers, test generators) see the full schema.
• The OperationMetadata interface gains an optional skipResponseSchema boolean. Consumers that do not check it are unaffected.

Verification

Check	Result
Project unit tests (262)	All pass
EVRN generated tests (159)	All pass (was 157/159)
Snapshot tests	All pass (no snapshot change for this fix)
unit-res route	200 SUCCESS (was 500 stack overflow)
unit-read route	200 SUCCESS (was 500 stack overflow)
unit-descriptive-info route	200 SUCCESS (was 500 stack overflow)
Passing routes (12 others)	Unchanged, still use fjs serialization
Generated route file inspection	Comment present, routeSchema used
Operation schema JSON	Full response object preserved

Files changed

File	Change
src/gateway/helpers.ts	Added measureSchemaRefComplexity()
src/gateway/generators.ts	Added complexity check in emitOperationSchemas(), conditional schema stripping in emitRouteFiles() and emitRoute FilesWithHandlers(), added skipResponseSchema to OperationMetadata

Related work

This fix is part of a larger effort to fix the --test-dir generator (v0.11.1+) for complex real-world WSDLs. The full set of bugs discovered and fixed during the EVRN WSDL testing is listed in the table.

Bug	Summary	Fix location
1	Missing attributes, for example Version, in mock data	schemaCompiler.ts (attrType), mockData.ts
2	SOAP wrapper format in request payloads	catalogMeta.ts (flattenMockPayload), pipeline threading
3	Wrong ARRAY_WRAPPERS vs CHILDREN_TYPES detection	Consolidated into src/util/catalogMeta.ts
4	Mock client Errors+Success (no code change needed)	N/A
5	vitest.config.ts uses wrong API	generators.ts (defineProject)
6	Enum types not detected in mock primitives	mockData.ts
7	Alias types not resolved in attrType	schemaCompiler.ts
8	Array-typed attributes	mockData.ts
9	unwrapArrayWrappers shallow recursion	generators.ts (runtime template)
10	flattenMockPayload shallow recursion	catalogMeta.ts
11	fast-json-stringify stack overflow on complex schemas	helpers.ts, generators.ts (this ADR)

Full Changelog: v0.11.5...v0.11.6

---

This discussion was created from the release v0.11.6.