Repository navigation
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
lexical
@lexical/a11y
@lexical/clipboard
@lexical/code
@lexical/code-core
@lexical/code-prism
@lexical/code-shiki
@lexical/compiler
@lexical/devtools-core
@lexical/dragon
@lexical/eslint-plugin
@lexical/extension
@lexical/fast-check
@lexical/file
@lexical/hashtag
@lexical/headless
@lexical/history
@lexical/html
@lexical/internal
@lexical/link
@lexical/list
@lexical/mark
@lexical/markdown
@lexical/mdast
@lexical/offset
@lexical/overflow
@lexical/plain-text
@lexical/react
@lexical/rich-text
@lexical/selection
@lexical/table
@lexical/tailwind
@lexical/text
@lexical/utils
@lexical/y
@lexical/yjs
commit: |
There was a problem hiding this comment.
Looks good: thanks for tracking down both ranges. The new test fails on main and passes here. Everything below is optional and non-blocking.
- Lines that start with a flag or an astral letter now resolve the other way (
packages/lexical/src/LexicalConstants.ts:121). Regional-indicator flags (U+1F1E6–U+1F1FF) and letters such as U+20000 (CJK Extension B) are Bidi_Class L. Browsers therefore lay out🇮🇱 שלוםunderdir="auto"left-to-right, butgetTextDirectionnow returns'rtl'for it. In a code block,$getCodeLineDirectionpasses that result toMOVE_TO_START/MOVE_TO_END, so Cmd+Left/Right goes to the opposite side on such a line. This is the limitation the description already names. To close it, regexes with theuflag could list the astral L ranges. To keep it, a test asserting the current result would make it deliberate. - Test the code-block path. Inside Lexical, the result reaches users through
$getCodeLineDirection(Cmd+Left/Right in code blocks). Paragraphs already renderdir="auto", and onmainthe browser already picks rtl for😀 שלום. A case next to thertl code linestests inpackages/lexical-code/src/__tests__/unit/LexicalCodeNode.test.tswould cover the behaviour this fixes: aplaintextline😀 تجربة, caret at offset 3, thenMOVE_TO_ENDshould land at offset 0 (it lands at 8 onmain). The description could say this too, since it currently says the paragraph is laid out left-to-right. - JSDoc (
packages/lexical/src/LexicalUtils.ts:320):getTextDirectionnow also returnsnullfor text made only of characters outside the BMP. A clause such as "characters outside the Basic Multilingual Plane are treated as neutral" would tell callers. - Test plan: AGENTS.md ("Commits and Pull Requests") asks for the pasted Before and After output of the command you ran. The After results here were run together with other open fixes, not on this branch alone.
| 'A-Za-z\u00C0-\u00D6\u00D8-\u00F6' + | ||
| '\u00F8-\u02B8\u0300-\u0590\u0800-\u1FFF\u200E\u2C00-\uFB1C' + | ||
| '\uFE00-\uFE6F\uFEFD-\uFFFF'; | ||
| '\u00F8-\u02B8\u0300-\u0590\u0800-\u1FFF\u200E\u2C00-\uD7FF' + |
There was a problem hiding this comment.
Optional: dropping the whole surrogate range also makes strong-LTR characters outside the BMP neutral. Regional-indicator flags (U+1F1E6–U+1F1FF) and CJK Extension B (U+20000) are Bidi_Class L, so browsers lay out 🇮🇱 שלום under dir="auto" left-to-right, while getTextDirection now returns 'rtl'. Cmd+Left/Right in a code block is then inverted on such a line. Regexes with the u flag that list those astral L ranges would close the gap; otherwise, a test pinning the current result would make the limitation deliberate.
| expect(getTextDirection(`\uFEFC`)).toBe('rtl'); | ||
| }); | ||
|
|
||
| test('getTextDirection() skips emoji: surrogate pairs and variation selectors (#5260)', () => { |
There was a problem hiding this comment.
Optional: users see this change in code-block Cmd+Left/Right, through $getCodeLineDirection. A case beside the rtl code lines tests in packages/lexical-code/src/__tests__/unit/LexicalCodeNode.test.ts would cover it: a plaintext line 😀 تجربة, caret at offset 3, then MOVE_TO_END should land at offset 0 (it lands at 8 on main).
The LTR ranges used by getTextDirection() included the UTF-16 surrogates \uD800-\uDFFF and the variation selectors \uFE00-\uFE0F, so a leading emoji made text such as "😀 שלום" or "❤️ שלום" left-to-right. Leave both out of the LTR ranges, so the direction comes from the next strong character. Fixes meta#5260 Signed-off-by: Ved Thakar <111605341+Vedthakar@users.noreply.github.com> Co-authored-by: Ved Thakar <111605341+Vedthakar@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…neutral Characters outside the Basic Multilingual Plane no longer count as left-to-right, so say so in the JSDoc. Co-authored-by: Ved Thakar <111605341+Vedthakar@users.noreply.github.com> Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-authored-by: Ved Thakar <111605341+Vedthakar@users.noreply.github.com> Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
a8d5f3c to
6170ba6
Compare
|
Thanks for the review. The — Ved Thakar |
There was a problem hiding this comment.
Thanks, this looks good. Treating surrogates and U+FE00–U+FE0F as neutral fixes the emoji-first case from #5260: getTextDirection('😀 שלום') now returns 'rtl', and in a code block Home/End (Cmd+Left/Right on macOS) on such a line now moves the caret the same way as on a line of only Hebrew or Arabic. The new test fails on main and passes here.
A few optional, non-blocking suggestions:
- Flags now read as right-to-left (
packages/lexical/src/LexicalConstants.ts:119-123). Regional-indicator symbols (U+1F1E6–U+1F1FF, i.e. every flag emoji), the Mathematical Alphanumeric letters and CJK Extension B have Unicode Bidi class L, and a browser lays out🇮🇱 שלוםunderdir="auto"left-to-right. With the whole surrogate range neutral,getTextDirection('🇮🇱 שלום')now returns'rtl'(it returned'ltr'before), so Home/End in a code block is inverted on that line. Au-flagLTRthat lists the astral L ranges while leaving the emoji blocks U+1F300–U+1FAFF neutral would keep the fix without this; otherwise a test that pins the flag result, plus a sentence in the JSDoc, would make the limitation deliberate. The description mentions CJK Extension B but not flags, which are the case people are most likely to type. - Cover the path users see. The only caller in the repository is
$getCodeLineDirectionin@lexical/code-core. A case beside thertl code linestests inpackages/lexical-code/src/__tests__/unit/LexicalCodeNode.test.tswould guard it: aplaintextline😀 تجربة, caret at offset 4, thenMOVE_TO_ENDshould land at offset 0 (it lands at 8 onmain). - Add a
Verify:paragraph to the changeset (.changeset/text-direction-ignores-emoji.md). CONTRIBUTING.md (Changesets) asks for one when a fix changes observable behaviour, for example:Verify: getTextDirection('😀 שלום') returns 'rtl'; in a code block, Cmd+Left/Right (Home/End outside macOS) on a line that starts with an emoji followed by Hebrew or Arabic moves the caret the way it does on a line of only Hebrew or Arabic.
| 'lexical': patch | ||
| --- | ||
|
|
||
| [fix] `getTextDirection` no longer reads an emoji or another character outside the Basic Multilingual Plane as left-to-right, so a line that starts with an emoji followed by right-to-left text is right-to-left |
There was a problem hiding this comment.
Optional: this fix changes what getTextDirection returns and how Home/End behaves in a code block, so CONTRIBUTING.md (Changesets) asks for a Verify: paragraph, which lists the entry under "Behavior changes". For example:
Verify: getTextDirection('😀 שלום') returns 'rtl'; in a code block, Cmd+Left/Right (Home/End outside macOS) on a line that starts with an emoji followed by Hebrew or Arabic moves the caret the way it does on a line of only Hebrew or Arabic.
Description
This solves issue #5260: text that starts with an emoji is given the wrong direction.
getTextDirection('😀 שלום')returns'ltr', so a Hebrew or Arabic paragraph that begins with an emoji is laid out left-to-right.Cause.
RTL_REGEXandLTR_REGEXinLexicalConstants.tsskip characters that aren't strongly directional, then look at the first one that is. They match UTF-16 code units (nouflag), and theLTRtable has two kinds of code unit that emoji are made of:\uD800-\uDFFF, inside\u2C00-\uFB1C. An emoji such as 😀 is the surrogate pair\uD83D\uDE00;\uFE00-\uFE0F, inside\uFE00-\uFE6F.\uFE0Ffollows many emoji written with a BMP character, such as ❤️ (\u2764\uFE0F), ✔️ and keycaps like 1️⃣ (1\uFE0F\u20E3).What this changes. The
LTRranges become\u2C00-\uD7FF,\uE000-\uFB1Cand\uFE10-\uFE6F. Surrogates and variation selectors are now neutral, so the direction comes from the next strong character. Nothing else in the tables changes. The existing assertion that\uFE00is'ltr'now checks\uFE10, the new start of that range.Limitation. Astral-plane letters (e.g. CJK Extension B) are also written as surrogate pairs, so they're now neutral too. Classifying them properly would need
u-flag regexes with astral ranges, which this PR doesn't attempt.Closes #5260
Test plan
New test in
packages/lexical/src/__tests__/unit/LexicalUtils.test.ts:getTextDirection() skips emoji: surrogate pairs and variation selectors (#5260). It checks:😀and\uFE0F→null😀 שלום,😀 مرحبا,😀😀 שלום,שלום 😀→'rtl'\uFE0F:❤️ שלום,1️⃣ שלום,✔️ مرحبا→'rtl'😀 hello→'ltr'\uD7FFand\uE000, either side of the removed surrogate range, stay'ltr'In the existing
getTextDirection()test, the\uFE00→'ltr'assertion is now\uFE10→'ltr'.Before
The updated test file run against
main'sLexicalConstants.ts:Tests 1 failed | 75 passed (76). The failing test isgetTextDirection() skips emoji: surrogate pairs and variation selectors (#5260).After
On current
main(tested together with my other open fixes):LexicalUtils.test.ts: 76 passed (the new test fails onmain).pnpm run test-unit: all 10,080 tests passtsc,pnpm run flow(0 errors), and eslint/prettier on the changed files: cleanAuthor: Ved Thakar (@Vedthakar)
🤖 Generated with Claude Code