Give exported diagrams their labels back, and say when an export is working - #1390
Open
Paul Lizer (paullizer) wants to merge 1 commit into
Open
Give exported diagrams their labels back, and say when an export is working#1390Paul Lizer (paullizer) wants to merge 1 commit into
Paul Lizer (paullizer) wants to merge 1 commit into
Conversation
…orking An exported Mermaid diagram came out with every shape and arrow in place and no text in any of the boxes, while the same diagram read correctly on screen. The image ships no scalable Latin typeface: the Dockerfile installed only the legacy X11 Type1 and bitmap packages, behind a `|| true` that hid the case where they did not resolve at all. The server renderer then asked for `Arial, Helvetica, sans-serif`, which resolves to nothing there, so Chromium measured every label as zero-width, Mermaid fell back to its minimum node size, and no glyphs were painted. The giveaway was every box coming out the same width when on screen they varied with their labels. The renderer now embeds DejaVu Sans from matplotlib, already required for TeX export, so it no longer depends on what the host happens to carry. It also captures each diagram with an element screenshot rather than serializing the SVG and repainting it through an `<img>` onto a canvas: it already has a real browser page open, and the isolated image context silently drops anything it cannot reproduce. Scalable fonts are added to the image as well, for everything else that renders in it. V2 sent only the message and conversation id, so every one of its exports took the server path. It now sends the diagram already on screen, which skips the browser launch entirely and keeps the colours the reader chose. Neither interface said anything while an export ran, and a PowerPoint waits on a model planning its slides, so it read as hung. Both now raise a notice on click that is replaced by the result, and disable the menu entry while it runs. The existing label test asserted only `painted_ratio > 0.01`, which boxes and arrows clear on their own; it now compares a labelled graph against the same graph with empty labels on dark-pixel count and width. Investigated and rejected: htmlLabels and `<foreignObject>` were never involved. The rendered SVG carries 11 `<text>` elements and no `<foreignObject>`, and a diagram using classDef produces identical colour histograms through the canvas path and a live screenshot, so the browser rasterizers were left alone. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Exporting a message containing a Mermaid diagram produced the right shapes, arrows and layout with no text in any of the boxes, while the same diagram read correctly on screen. The export was also slow enough to look like it had hung.
Root cause
Two problems that presented as one.
1. The container has no scalable font. The Dockerfile installed only
xorg-x11-fonts-Type1andxorg-x11-fonts-misc— legacy X11 Type1 and bitmap packages, no scalable TrueType Latin face — behind a|| truethat hid the case where they did not resolve at all. Azure Linux ships no fonts by default. The server renderer then asked Mermaid forArial, Helvetica, sans-serif, none of which exist there, and the genericsans-seriffallback resolved to nothing. Chromium measured every label as zero-width, Mermaid fell back to its minimum node size, and no glyphs were painted.The giveaway in the reported files is that every box came out the same width, while on screen the same diagram had boxes of visibly different widths. That is zero-width text measurement, not a colour, theme or
<foreignObject>problem.2. V2 never sent the diagram it had already drawn.
downloadMessageExport()posted onlymessage_idandconversation_id. The classic chat sendsvisual_assets; V2 did not, so every V2 export took the server path and hit the font problem — even though a correct picture already existed in the page.Investigated and rejected
I chased
htmlLabels/<foreignObject>first and was wrong. The rendered SVG carries 11<text>elements and zero<foreignObject>elements. I also checked whether the browser's serialize-and-repaint rasterizer loses the styles Mermaid adds viainsertRule: a diagram usingclassDefandstyleproduces identical colour histograms through the canvas path and a live screenshot. The browser rasterizers were correct and are unchanged rather than "fixed" speculatively.Changes
functions_mermaid_server_render.py@font-facefrom matplotlib (already a TeX-export dependency); capture each diagram with a Playwright element screenshot instead of SVG →<img>→ canvas; block only real network schemes so the font'sdata:URI resolves; reportembedded_font_availableDockerfilev2_ui/src/lib/exportVisuals.ts(new)visual_assetsMermaidDiagram.tsx,endpoints.tsvisual_assetson the export requesttoastStore.ts,Toaster.tsx,MessageActions.tsxpendingtone that does not auto-dismiss and settles in place; spinner and disabled menu entry while an export runstoast.js,chat-toast.js,chat-message-export.js,chat-messages.js{ autohide: false }plus a{ dismiss }handle, a progress toast always cleared infinally, and thedata-pending-labelits inline buttons already hadWhy an element screenshot
The renderer was serializing its SVG, reloading it through an
<img>and painting it onto a canvas — a reconstruction of a browser render, performed inside a real browser that was already open. That isolated context drops<foreignObject>content and will not load a font that is not already present. Screenshotting the mounted element renders exactly what a browser would show, and is less code.Why this shipped broken
test_server_renders_diagrams_with_visible_labelsclaimed to verify that "a rendered diagram keeps its label text" but asserted onlypainted_ratio > 0.01. Boxes and arrows clear that easily, so a diagram with no text at all passed.It now renders the same graph with and without label text and requires the labelled one to have materially more dark pixels and to be wider. On the fixture: 12,295 dark pixels vs 0, and 945px wide vs 372px.
Validation
test_export_mermaid_server_render.py— 13/13test_message_export_progress_feedback.py— 8/8, including a parity check that executes the browser's ownnormalizeVisualSourcein Node against Python'snormalize_visual_source. A mismatch there does not fail loudly: every client asset is silently discarded and every diagram quietly re-rendered server-side.test_export_mermaid_browser_rasterizer.py— 3/3test_conversation_export_mermaid_tex_images.py— 18/18test_deep_research_chromium_build_opt_out.py— 3/3test_docs_app_surface_coverage.py— 7/7,test_docs_site_quality.py— 6/6route_tests/— 3/3npm run buildinapplication/v2_ui— cleanEnd to end: a real
.docxgenerated through the actual export route embeds a 945×549 diagram with full label text and correctly varying box widths.Notes for review
test_deep_research_chromium_build_opt_out.pyasserted an exact deployer version of1.0.4and had been failing since the deployer moved past it (now1.0.26). It covers the Dockerfile changed here, so a red test would have masked real regressions; it now uses the repo's ownassert_version_at_leasthelper, which the versioning instructions require.deployers/changes, sodeployers/version.txtis not bumped. The Dockerfile lives underapplication/and introduces no new build argument or parameter.gunicorn.conf.pyalready allows 900s per request, so nothing was cutting exports off server-side. The practical ceiling is the Azure App Service front-end idle timeout of roughly 230s, which cannot be configured from this repo. Skipping the browser launch and reporting progress were the available mitigations; PowerPoint remains inherently slow because a model plans the slides.Version bumped to
0.261.034. Fix doc atdocs/explanation/fixes/MERMAID_EXPORT_LABEL_TEXT_FIX.md; feature doc and release notes updated.