feat(website): export blog markdown for agents#1934
Open
rohitg00 wants to merge 7 commits into
Open
Conversation
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Terraform plan —
|
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds blog markdown export for coding agents, threads generated blog links into ChangesBlog Markdown Export for Coding Agents
Estimated code review effort: 3 (Moderate) | ~25 minutes Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
rohitg00
marked this pull request as draft
July 6, 2026 09:13
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (4)
website/scripts/generate-blog-md.test.ts (1)
14-43: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winConsider adding an
.mdxsource test case.These tests only exercise
.mdsource files. GivenreadBlogPostssupports.mdxtoo, a test writing an.mdxfixture would catch the extension-mismatch issue flagged ingenerate-blog-md.ts.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@website/scripts/generate-blog-md.test.ts` around lines 14 - 43, Add coverage for the .mdx path in exportBlogMarkdown/readBlogPosts by extending the generate-blog-md.test.ts fixture setup to include an .mdx blog source file and asserting it is exported correctly. Use the existing exportBlogMarkdown test and related symbols to mirror the current .md case, but ensure the new case verifies the extension handling so the mismatch in generate-blog-md.ts is caught.website/scripts/generate-blog-md.ts (2)
50-53: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low valueUnescaped title/description can break markdown link syntax.
post.title/post.descriptionare interpolated raw into[title](url)links. A title or description containing]or)would produce malformed markdown inindex.md,llms.txt, andAGENTS.md. Since content is authored internally, this is low risk today but worth a defensive check for frontmatter authors.Also applies to: 77-80
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@website/scripts/generate-blog-md.ts` around lines 50 - 53, The generated blog index entries in generate-blog-md.ts interpolate post.title and post.description directly into markdown link text, which can break the syntax if either contains ] or ). Update the post mapping used for indexLines (and the same logic in the other referenced output paths) to escape or sanitize these fields before building the [title](url) string, using the existing post.slug-based URL as-is and ensuring the rendered markdown stays valid for all authored content.
50-53: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDuplicate link-line formatting logic.
The markdown link template
`- [${post.title}](${SITE_ORIGIN}/blog/${post.slug}.md) — ${post.description}`is duplicated verbatim inexportBlogMarkdownandbuildBlogLinksSection.♻️ Proposed refactor
+function formatBlogLink(post: { title: string; slug: string; description: string }): string { + return `- [${post.title}](${SITE_ORIGIN}/blog/${post.slug}.md) — ${post.description}` +} + export async function exportBlogMarkdown( ... - const indexLines = posts.map( - (post) => - `- [${post.title}](${SITE_ORIGIN}/blog/${post.slug}.md) — ${post.description}`, - ) + const indexLines = posts.map(formatBlogLink) ... export async function buildBlogLinksSection(): Promise<string> { ... - const lines = posts.map( - (post) => - `- [${post.title}](${SITE_ORIGIN}/blog/${post.slug}.md) — ${post.description}`, - ) + const lines = posts.map(formatBlogLink)Also applies to: 73-80
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@website/scripts/generate-blog-md.ts` around lines 50 - 53, The blog markdown link-line template is duplicated in both exportBlogMarkdown and buildBlogLinksSection, so extract the shared formatting into a single helper and reuse it from both places. Update the logic around posts.map in generate-blog-md.ts so the link text, URL, and description are built in one symbol (for example a dedicated formatter function) and both sections call that instead of repeating the same template.website/scripts/generate-llms-agents.ts (1)
153-160: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueExtract the duplicated optional-section insertion pattern.
The
[section.trimEnd(), section ? '' : undefined]pattern is duplicated verbatim betweenbuildLlmsTxt(Lines 153-154) andbuildAgentsMd(Lines 179-180). Worth extracting into a small shared helper, e.g.optionalSection(section: string): (string | undefined)[], to avoid divergence if this logic needs to change later.Also note: when
blogSectionis empty (no visible posts), the array becomes[..., home.trimEnd(), '', '', tail.trimEnd()]after filtering — two consecutive empty strings produce an extra blank line compared to whenblogSectionis populated ([..., home.trimEnd(), '', content, '', tail.trimEnd()]). This is a harmless cosmetic inconsistency in the generatedllms.txt/AGENTS.mdoutput.♻️ Proposed helper extraction
+function optionalSection(section: string): (string | undefined)[] { + return section ? [section.trimEnd(), ''] : [] +} + export function buildLlmsTxt(html: string, blogSection = ''): string { ... const body = [ '# iii', '', `> ${LLMS_TAGLINE}`, '', overview.trimEnd(), '', home.trimEnd(), '', - blogSection.trimEnd(), - blogSection ? '' : undefined, + ...optionalSection(blogSection), tail.trimEnd(), '', ].filter((chunk): chunk is string => chunk !== undefined).join('\n')Also applies to: 179-188
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@website/scripts/generate-llms-agents.ts` around lines 153 - 160, The duplicated optional-section pattern in buildLlmsTxt and buildAgentsMd should be extracted into a shared helper to keep the empty-section handling consistent. Introduce a small helper such as optionalSection and use it at both call sites so the trimEnd/conditional-empty-string logic lives in one place, and keep the resulting output behavior unchanged aside from the noted extra blank line when blogSection is empty.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@website/scripts/generate-blog-md.test.ts`:
- Around line 14-43: Add coverage for the .mdx path in
exportBlogMarkdown/readBlogPosts by extending the generate-blog-md.test.ts
fixture setup to include an .mdx blog source file and asserting it is exported
correctly. Use the existing exportBlogMarkdown test and related symbols to
mirror the current .md case, but ensure the new case verifies the extension
handling so the mismatch in generate-blog-md.ts is caught.
In `@website/scripts/generate-blog-md.ts`:
- Around line 50-53: The generated blog index entries in generate-blog-md.ts
interpolate post.title and post.description directly into markdown link text,
which can break the syntax if either contains ] or ). Update the post mapping
used for indexLines (and the same logic in the other referenced output paths) to
escape or sanitize these fields before building the [title](url) string, using
the existing post.slug-based URL as-is and ensuring the rendered markdown stays
valid for all authored content.
- Around line 50-53: The blog markdown link-line template is duplicated in both
exportBlogMarkdown and buildBlogLinksSection, so extract the shared formatting
into a single helper and reuse it from both places. Update the logic around
posts.map in generate-blog-md.ts so the link text, URL, and description are
built in one symbol (for example a dedicated formatter function) and both
sections call that instead of repeating the same template.
In `@website/scripts/generate-llms-agents.ts`:
- Around line 153-160: The duplicated optional-section pattern in buildLlmsTxt
and buildAgentsMd should be extracted into a shared helper to keep the
empty-section handling consistent. Introduce a small helper such as
optionalSection and use it at both call sites so the
trimEnd/conditional-empty-string logic lives in one place, and keep the
resulting output behavior unchanged aside from the noted extra blank line when
blogSection is empty.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 0f638eeb-29bd-43ab-9745-baa396b43f22
📒 Files selected for processing (18)
.github/workflows/deploy-website.ymlAGENTS.mdblog/README.mdinfra/terraform/website/cloudfront_functions/redirects.jsinfra/terraform/website/cloudfront_functions/redirects.test.jswebsite/index.htmlwebsite/privacy.htmlwebsite/scripts/agents-appendix.mdwebsite/scripts/blog-posts.test.tswebsite/scripts/blog-posts.tswebsite/scripts/generate-blog-md.test.tswebsite/scripts/generate-blog-md.tswebsite/scripts/generate-llms-agents.test.tswebsite/scripts/generate-llms-agents.tswebsite/scripts/generate-sitemap.tswebsite/scripts/routes-kvs.test.tswebsite/scripts/routes.tswebsite/sitemap.xml
rohitg00
marked this pull request as ready for review
July 6, 2026 09:21
rohitg00
changed the title
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (2)
website/scripts/generate-blog-md.test.ts (1)
60-88: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider extracting shared tmp-dir setup for the two
exportBlogMarkdowntests.Both the
.md(Line 29) and.mdx(Line 60) tests repeat the samemkdtemp/mkdirboilerplate. A small helper (e.g.,setupTempBlogDirs()) would reduce duplication, though current tests remain readable as-is.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@website/scripts/generate-blog-md.test.ts` around lines 60 - 88, The two exportBlogMarkdown tests duplicate the same temporary directory setup, so extract the shared mkdtemp/mkdir boilerplate into a small helper such as setupTempBlogDirs and reuse it from both the .md and .mdx test cases. Keep the helper close to exportBlogMarkdown in generate-blog-md.test.ts so the test intent stays clear while reducing repeated setup code.website/scripts/generate-blog-md.ts (1)
53-59: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winSequential await in loop; consider parallelizing.
Each post's read/rewrite/write happens sequentially. Since these are independent I/O operations,
Promise.allcould reduce build time as the number of posts grows.Proposed refactor
- for (const post of posts) { - const sourcePath = path.join(contentDir, post.sourceFile) - const raw = await fs.readFile(sourcePath, 'utf8') - const exported = rewriteBlogImagePaths(raw, astroAssets) - await fs.writeFile(path.join(distDir, `${post.slug}.md`), exported, 'utf8') - count++ - } + await Promise.all( + posts.map(async (post) => { + const sourcePath = path.join(contentDir, post.sourceFile) + const raw = await fs.readFile(sourcePath, 'utf8') + const exported = rewriteBlogImagePaths(raw, astroAssets) + await fs.writeFile(path.join(distDir, `${post.slug}.md`), exported, 'utf8') + }), + ) + count = posts.length🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@website/scripts/generate-blog-md.ts` around lines 53 - 59, The blog generation loop is performing independent read/rewrite/write work sequentially in the generateBlogMd flow, which slows builds as post count grows. Refactor the posts processing in generateBlogMd to run each post’s fs.readFile, rewriteBlogImagePaths, and fs.writeFile concurrently with Promise.all (or an equivalent parallel map), while keeping the same output path logic and count tracking behavior in the surrounding function.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@website/scripts/generate-blog-md.test.ts`:
- Around line 60-88: The two exportBlogMarkdown tests duplicate the same
temporary directory setup, so extract the shared mkdtemp/mkdir boilerplate into
a small helper such as setupTempBlogDirs and reuse it from both the .md and .mdx
test cases. Keep the helper close to exportBlogMarkdown in
generate-blog-md.test.ts so the test intent stays clear while reducing repeated
setup code.
In `@website/scripts/generate-blog-md.ts`:
- Around line 53-59: The blog generation loop is performing independent
read/rewrite/write work sequentially in the generateBlogMd flow, which slows
builds as post count grows. Refactor the posts processing in generateBlogMd to
run each post’s fs.readFile, rewriteBlogImagePaths, and fs.writeFile
concurrently with Promise.all (or an equivalent parallel map), while keeping the
same output path logic and count tracking behavior in the surrounding function.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 280e3e92-3206-459d-83e4-4858cd2b1bcc
📒 Files selected for processing (4)
website/scripts/blog-posts.tswebsite/scripts/generate-blog-md.test.tswebsite/scripts/generate-blog-md.tswebsite/scripts/generate-llms-agents.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- website/scripts/generate-llms-agents.ts
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
website/scripts/generate-blog-md.ts (1)
63-80: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winValidate
slug !== 'index'before starting concurrent I/O.The
slug === 'index'check is performed inside thePromise.allmap (Line 71-73), after other posts' async reads/writes may already be in flight. SincePromise.allrejects on the first failure without cancelling already-started promises, other posts'<slug>.mdfiles can still get written todistDireven though the export ultimately throws — leaving a partially-exporteddistDiron failure.Move the collision check to a synchronous pass over
postsbefore thePromise.allcall to fail fast and avoid partial writes.♻️ Proposed fix
const astroAssets = await listAstroAssets(distDir) + const indexCollision = posts.find((post) => post.slug === 'index') + if (indexCollision) { + throw new Error(`blog post slug "index" collides with the generated blog/index.md`) + } + await Promise.all( posts.map(async (post) => { - if (post.slug === 'index') { - throw new Error(`blog post slug "index" collides with the generated blog/index.md`) - } const sourcePath = path.join(contentDir, post.sourceFile)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@website/scripts/generate-blog-md.ts` around lines 63 - 80, The blog export in generate-blog-md.ts is starting async reads/writes in Promise.all before validating that no post has the reserved slug "index", which can leave partial files in distDir if one task fails. Add a synchronous pre-check over posts before the Promise.all block in generate-blog-md.ts (near readBlogPosts, listAstroAssets, and rewriteBlogImagePaths usage) to reject any post with slug === 'index' up front, then keep the per-post export logic unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@website/scripts/generate-blog-md.ts`:
- Around line 63-80: The blog export in generate-blog-md.ts is starting async
reads/writes in Promise.all before validating that no post has the reserved slug
"index", which can leave partial files in distDir if one task fails. Add a
synchronous pre-check over posts before the Promise.all block in
generate-blog-md.ts (near readBlogPosts, listAstroAssets, and
rewriteBlogImagePaths usage) to reject any post with slug === 'index' up front,
then keep the per-post export logic unchanged.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: ac2f846e-7177-460d-a671-e482ffb724a5
📒 Files selected for processing (2)
website/scripts/generate-blog-md.test.tswebsite/scripts/generate-blog-md.ts
guibeira
previously approved these changes
Jul 8, 2026
rohitg00
force-pushed
the
docs/blog-md-llms-index
branch
from
July 13, 2026 10:37
883d55b to
9fd807c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
https://iii.dev/blog/<slug>.md, plushttps://iii.dev/blog/index.md./blog/_astro/URLs.llms.txtandAGENTS.md(deploy-time), repoAGENTS.md, sitemap, andagents-appendix.md.Test plan
pnpm --filter iii-website test— 67/67 pass (includes redirect + blog-md export tests)pnpm --filter iii-blog build+tsx scripts/generate-blog-md.ts— 7 posts + index.md exportedpnpm --filter iii-website build— llms.txt / sitemap include blog .md URLs and/privacySummary by CodeRabbit
blog/index.md./blog/<slug>.mdURLs (and adjusted priorities).