Repository navigation
astgen: cancel a type write TypeScript cannot bring to an end - #25
Merged
Merged
Conversation
Once it is truncating, TypeScript's node builder still expands substitution, conditional and template literal types, so a type that recurses through them runs on until the heap is gone. One prisma.article.findMany (Prisma 4.16) in node-express-realworld-example-app reached 17 million characters over 300,000 builder calls and took 4 GB with it; astgen exited without an AST for any file. The builder asks the checker's cancellation token once per node, so a write now runs under a token that cancels past 16384 calls, and is then a type too long to write, as when the length limit holds. Across 37 projects no written type needed more than 305 builder calls, and every AST and type map is byte for byte as before. ASTGEN_TYPE_TIMING reports the most a file's written types needed and the writes cancelled. astgen 4.5.2, so an empty parse cached from the crash is not reused. Signed-off-by: Team AppThreat <cloud@appthreat.com>
prabhu
added
bug
prabhu
enabled auto-merge
October 26, 2026 03:24
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
astgen ran out of memory on node-express-realworld-example-app and exited without writing an AST for most files. chen then built an atom with almost nothing in it, and every reachables flow for the project disappeared.
The cause is in TypeScript's node builder. Once a write passes astgen's 4096-character limit the builder marks itself as truncating, but in several places it keeps going regardless: substitution, conditional and template literal types are still expanded. Prisma 4's client result types recurse through exactly those. Writing the type of one
prisma.article.findManyreached 17 million characters over about 300,000 builder calls and used up 4 GB.The builder asks the checker's cancellation token once per node it builds, so each write now runs under
runWithCancellationTokenwith a token that cancels after 16,384 calls. A cancelled write is handled exactly like one that ran over the length limit: astgen tries the alias name, then falls back toany. After a cancellation I checked the checker's instantiation depth, active mapper stack and resolution stack, and all three are back at zero, so later writes aren't affected.On the budget: I ran main and this branch over 37 projects, including juice-shop, taxonomy and the Angular, React, Vue, Svelte and Solid realworld apps. Every AST and type map came out byte for byte the same, no write was cancelled, and no written type needed more than 305 builder calls. On node-express-realworld, main dies after 7 files; this branch writes all 28 in 3 seconds.
ASTGEN_TYPE_TIMINGnow also reports, per file, the most builder calls a written type needed and how many writes were cancelled.The new
astgen-recursive-types-regression.jsuses a small fixture in the same shape as an ORM client's result types. It's my own code, not Prisma's. Without the change the fixture needs about 2 GB; the test runs it under 512 MB, where main fails and this branch passes. The fullnpm testpasses.ASTGEN_VERSIONgoes to 4.5.2, because chen keys its parse cache on it and would otherwise keep reusing the empty parse from the crash. The package version is left for the release.