Ability to disable sourcemap generation in dev for large files, i.e Astro 5's data layer

[KianNH]

Description

Astro 5's Content Layer leverages a data store snapshot, implemented as a virtual module called astro:data-layer-content.

When editing a file in astro dev, the content of the data store changes and the virtual module is transformed & run as an inline module by Vite:

A lot of the time is spent generating a sourcemap for this module (which takes the shape of a large export default [...]). In large sites, this module can be upwards of 20 megabytes.

With a module that is 25.49 MB, a 54.23 MB sourcemap is created with MagicString#generateMap({ hires: true }) and 132.99 MB of code is eventually evaluated.

Sourcemaps in ssrTransformScript

Patch

diff --git a/node_modules/vite/dist/node/chunks/dep-BJP6rrE_.js b/node_modules/vite/dist/node/chunks/dep-BJP6rrE_.js
index a7b3e26..f6fd7a9 100644
--- a/node_modules/vite/dist/node/chunks/dep-BJP6rrE_.js
+++ b/node_modules/vite/dist/node/chunks/dep-BJP6rrE_.js
@@ -31343,7 +31343,18 @@ Object.defineProperty(${ssrModuleExportsKey}, "default", { enumerable: true, con
       }
     }
   });
+
+  const isAstroContentModule = url.includes("astro:data-layer-content")
+
+  if (isAstroContentModule) {
+    console.log(`size of MagicString: ${s.toString().length / 1_000_000} MB`)
+    console.time("generating astro:data-layer-content sourcemap")
+  }
   let map = s.generateMap({ hires: "boundary" });
+  if (isAstroContentModule) {
+    console.timeEnd("generating astro:data-layer-content sourcemap")
+    console.log(`size of sourcemap: ${map.toString().length / 1_000_000} MB`)
+  }
   map.sources = [path$d.basename(url)];
   map.sourcesContent = [originalCode];
   if (inMap && inMap.mappings && "sources" in inMap && inMap.sources.length > 0) {

Output

size of MagicString: 25.501066 MB
generating astro:data-layer-content sourcemap: 1.342s
size of sourcemap: 54.235023 MB

Inlining sourcemap in inlineSourceMap

Patch

diff --git a/node_modules/vite/dist/node/chunks/dep-BJP6rrE_.js b/node_modules/vite/dist/node/chunks/dep-BJP6rrE_.js
index a7b3e26..f6fd7a9 100644
--- a/node_modules/vite/dist/node/chunks/dep-BJP6rrE_.js
+++ b/node_modules/vite/dist/node/chunks/dep-BJP6rrE_.js
@@ -51508,6 +51519,10 @@ const OTHER_SOURCE_MAP_REGEXP = new RegExp(
   "gm"
 );
 function inlineSourceMap(mod, result, startOffset) {
+  const isAstroContentModule = mod.id.includes("astro:data-layer-content");
+  if (isAstroContentModule) {
+    console.time("inling astro:data-layer-content sourcemap")
+  }
   const map = result.map;
   let code = result.code;
   if (!map || !("version" in map) || code.includes(MODULE_RUNNER_SOURCEMAPPING_SOURCE))
@@ -51523,6 +51538,10 @@ function inlineSourceMap(mod, result, startOffset) {
 ${MODULE_RUNNER_SOURCEMAPPING_SOURCE}
 //# ${SOURCEMAPPING_URL}=${genSourceMapUrl(sourceMap)}
 `;
+
+  if (isAstroContentModule) {
+    console.timeEnd("inling astro:data-layer-content sourcemap")
+  }
   return result;
 }

Output

inling astro:data-layer-content sourcemap: 350.641ms

Evaluating code in runInlinedModule.

Patch

diff --git a/node_modules/vite/dist/node/module-runner.js b/node_modules/vite/dist/node/module-runner.js
index d862bb2..7c0ddd1 100644
--- a/node_modules/vite/dist/node/module-runner.js
+++ b/node_modules/vite/dist/node/module-runner.js
@@ -1046,6 +1046,14 @@ function enableSourceMapSupport(runner) {
 class ESModulesEvaluator {
   startOffset = getAsyncFunctionDeclarationPaddingLineCount();
   async runInlinedModule(context, code) {
+    const isAstroContentModule = context[ssrImportMetaKey].url.includes("astro:data-layer-content");
+
+    if (isAstroContentModule) {
+      console.log(`astro:data-layer-content is ${code.length / 1_000_000} MB.`)
+
+      console.time(`run module`)
+    }
+
     await new AsyncFunction(
       ssrModuleExportsKey,
       ssrImportMetaKey,
@@ -1061,6 +1069,10 @@ class ESModulesEvaluator {
       context[ssrDynamicImportKey],
       context[ssrExportAllKey]
     ), Object.seal(context[ssrModuleExportsKey]);
+
+    if (isAstroContentModule) {
+      console.timeEnd(`run module`)
+    }
   }
   runExternalModule(filepath) {
     return import(filepath);

Output

astro:data-layer-content is 132.995481 MB.
run module: 1.057s

Combined

Output

size of MagicString: 25.501066 MB
generating astro:data-layer-content sourcemap: 1.342s
size of sourcemap: 54.235023 MB
inling astro:data-layer-content sourcemap: 350.641ms
astro:data-layer-content is 132.995481 MB.
run module: 1.057s

Generating the sourcemap, inlining it and then executing the code takes ~2.75 seconds - which represents the majority of the ~3.5 seconds to reload a page in SSR.

Suggested solution

A configuration option to, if possible, avoid creating sourcemaps for specific files.

I am not well-versed in JavaScript sourcemaps so there may be caveats, or that it simply isn't possible even if a file is not importing any other modules, to this suggestion.

Whilst this module will always need to be evaluated, the slowdown in runInlinedModule seems to be attributed to the size of the inlined sourcemap.

Alternative

Astro itself had a Vite plugin generating sourcemaps for this file when it wasn't required, but that has been fixed in withastro/astro#13001 so the only remaining generation is being done by Vite.

Additional context

No response

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

[hi-ogawa]

Can you also reference astro:data-layer-content implementation? Do you have return { code, map: { mappings: '' } } to indicate not to generate source map? https://rollupjs.org/plugin-development/#source-code-transformations

This may be considered as a perf bug. One solution might be that we can skip ssr transform magic-string generation when inMap.mappings === '' since they should probably get collapsed to empty mapping in the end:
(This might not be only a perf issue, but also it's not collapsing properly)

vite/packages/vite/src/node/ssr/ssrTransform.ts

Lines 422 to 437 in 4f5845a

	let map = s.generateMap({ hires: 'boundary' })
	map.sources = [path.basename(url)]
	// needs to use originalCode instead of code
	// because code might be already transformed even if map is null
	map.sourcesContent = [originalCode]
	if (
	inMap &&
	inMap.mappings &&
	'sources' in inMap &&
	inMap.sources.length > 0
	) {
	map = combineSourcemaps(url, [
	map as RawSourceMap,
	inMap as RawSourceMap,
	]) as SourceMap
	}

[KianNH]

Hi Hiroshi - thanks for taking a look at this so quickly!

It looks like mappings: '' is being set: https://github.com/withastro/astro/blob/main/packages/astro/src/content/vite-plugin-content-virtual-mod.ts#L154