Skip to content

[lexical] Bug Fix: Don't treat surrogate pairs (emoji) as left-to-right - #9462

Open
Vedthakar wants to merge 3 commits into
meta:mainfrom
Vedthakar:fix/5260-rtl-surrogates
Open

Vedthakar wants to merge 3 commits into
meta:mainfrom
Vedthakar:fix/5260-rtl-surrogates

Conversation

@Vedthakar

Copy link
Copy Markdown
Contributor

Description

This solves issue #5260: text that starts with an emoji is given the wrong direction. getTextDirection('😀 שלום') returns 'ltr', so a Hebrew or Arabic paragraph that begins with an emoji is laid out left-to-right.

Cause. RTL_REGEX and LTR_REGEX in LexicalConstants.ts skip characters that aren't strongly directional, then look at the first one that is. They match UTF-16 code units (no u flag), and the LTR table has two kinds of code unit that emoji are made of:

  • the surrogates \uD800-\uDFFF, inside \u2C00-\uFB1C. An emoji such as 😀 is the surrogate pair \uD83D\uDE00;
  • the variation selectors \uFE00-\uFE0F, inside \uFE00-\uFE6F. \uFE0F follows many emoji written with a BMP character, such as ❤️ (\u2764\uFE0F), ✔️ and keycaps like 1️⃣ (1\uFE0F\u20E3).

What this changes. The LTR ranges become \u2C00-\uD7FF, \uE000-\uFB1C and \uFE10-\uFE6F. Surrogates and variation selectors are now neutral, so the direction comes from the next strong character. Nothing else in the tables changes. The existing assertion that \uFE00 is 'ltr' now checks \uFE10, the new start of that range.

Limitation. Astral-plane letters (e.g. CJK Extension B) are also written as surrogate pairs, so they're now neutral too. Classifying them properly would need u-flag regexes with astral ranges, which this PR doesn't attempt.

Closes #5260

Test plan

New test in packages/lexical/src/__tests__/unit/LexicalUtils.test.ts: getTextDirection() skips emoji: surrogate pairs and variation selectors (#5260). It checks:

  • 😀 and \uFE0F → null
  • 😀 שלום, 😀 مرحبا, 😀😀 שלום, שלום 😀 → 'rtl'
  • emoji followed by \uFE0F: ❤️ שלום, 1️⃣ שלום, ✔️ مرحبا → 'rtl'
  • 😀 hello → 'ltr'
  • \uD7FF and \uE000, either side of the removed surrogate range, stay 'ltr'

In the existing getTextDirection() test, the \uFE00 → 'ltr' assertion is now \uFE10 → 'ltr'.

Before

The updated test file run against main's LexicalConstants.ts: Tests 1 failed | 75 passed (76). The failing test is getTextDirection() skips emoji: surrogate pairs and variation selectors (#5260).

After

On current main (tested together with my other open fixes):

  • LexicalUtils.test.ts: 76 passed (the new test fails on main).
  • pnpm run test-unit: all 10,080 tests pass
  • tsc, pnpm run flow (0 errors), and eslint/prettier on the changed files: clean

Author: Ved Thakar (@Vedthakar)

🤖 Generated with Claude Code

@vercel

vercel Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
lexical Ready Ready Preview Nov 16, 2026 11:57pm UTC
lexical-playground Ready Ready Preview Nov 16, 2026 11:57pm UTC

Request Review

@pkg-pr-new

pkg-pr-new Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
lexical

npm i https://pkg.pr.new/lexical@9462

@lexical/a11y

npm i https://pkg.pr.new/@lexical/a11y@9462

@lexical/clipboard

npm i https://pkg.pr.new/@lexical/clipboard@9462

@lexical/code

npm i https://pkg.pr.new/@lexical/code@9462

@lexical/code-core

npm i https://pkg.pr.new/@lexical/code-core@9462

@lexical/code-prism

npm i https://pkg.pr.new/@lexical/code-prism@9462

@lexical/code-shiki

npm i https://pkg.pr.new/@lexical/code-shiki@9462

@lexical/compiler

npm i https://pkg.pr.new/@lexical/compiler@9462

@lexical/devtools-core

npm i https://pkg.pr.new/@lexical/devtools-core@9462

@lexical/dragon

npm i https://pkg.pr.new/@lexical/dragon@9462

@lexical/eslint-plugin

npm i https://pkg.pr.new/@lexical/eslint-plugin@9462

@lexical/extension

npm i https://pkg.pr.new/@lexical/extension@9462

@lexical/fast-check

npm i https://pkg.pr.new/@lexical/fast-check@9462

@lexical/file

npm i https://pkg.pr.new/@lexical/file@9462

@lexical/hashtag

npm i https://pkg.pr.new/@lexical/hashtag@9462

@lexical/headless

npm i https://pkg.pr.new/@lexical/headless@9462

@lexical/history

npm i https://pkg.pr.new/@lexical/history@9462

@lexical/html

npm i https://pkg.pr.new/@lexical/html@9462

@lexical/internal

npm i https://pkg.pr.new/@lexical/internal@9462

@lexical/link

npm i https://pkg.pr.new/@lexical/link@9462

@lexical/list

npm i https://pkg.pr.new/@lexical/list@9462

@lexical/mark

npm i https://pkg.pr.new/@lexical/mark@9462

@lexical/markdown

npm i https://pkg.pr.new/@lexical/markdown@9462

@lexical/mdast

npm i https://pkg.pr.new/@lexical/mdast@9462

@lexical/offset

npm i https://pkg.pr.new/@lexical/offset@9462

@lexical/overflow

npm i https://pkg.pr.new/@lexical/overflow@9462

@lexical/plain-text

npm i https://pkg.pr.new/@lexical/plain-text@9462

@lexical/react

npm i https://pkg.pr.new/@lexical/react@9462

@lexical/rich-text

npm i https://pkg.pr.new/@lexical/rich-text@9462

@lexical/selection

npm i https://pkg.pr.new/@lexical/selection@9462

@lexical/table

npm i https://pkg.pr.new/@lexical/table@9462

@lexical/tailwind

npm i https://pkg.pr.new/@lexical/tailwind@9462

@lexical/text

npm i https://pkg.pr.new/@lexical/text@9462

@lexical/utils

npm i https://pkg.pr.new/@lexical/utils@9462

@lexical/y

npm i https://pkg.pr.new/@lexical/y@9462

@lexical/yjs

npm i https://pkg.pr.new/@lexical/yjs@9462

commit: 6170ba6

@rootnode-bot rootnode-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good: thanks for tracking down both ranges. The new test fails on main and passes here. Everything below is optional and non-blocking.

  1. Lines that start with a flag or an astral letter now resolve the other way (packages/lexical/src/LexicalConstants.ts:121). Regional-indicator flags (U+1F1E6–U+1F1FF) and letters such as U+20000 (CJK Extension B) are Bidi_Class L. Browsers therefore lay out 🇮🇱 שלום under dir="auto" left-to-right, but getTextDirection now returns 'rtl' for it. In a code block, $getCodeLineDirection passes that result to MOVE_TO_START/MOVE_TO_END, so Cmd+Left/Right goes to the opposite side on such a line. This is the limitation the description already names. To close it, regexes with the u flag could list the astral L ranges. To keep it, a test asserting the current result would make it deliberate.
  2. Test the code-block path. Inside Lexical, the result reaches users through $getCodeLineDirection (Cmd+Left/Right in code blocks). Paragraphs already render dir="auto", and on main the browser already picks rtl for 😀 שלום. A case next to the rtl code lines tests in packages/lexical-code/src/__tests__/unit/LexicalCodeNode.test.ts would cover the behaviour this fixes: a plaintext line 😀 تجربة, caret at offset 3, then MOVE_TO_END should land at offset 0 (it lands at 8 on main). The description could say this too, since it currently says the paragraph is laid out left-to-right.
  3. JSDoc (packages/lexical/src/LexicalUtils.ts:320): getTextDirection now also returns null for text made only of characters outside the BMP. A clause such as "characters outside the Basic Multilingual Plane are treated as neutral" would tell callers.
  4. Test plan: AGENTS.md ("Commits and Pull Requests") asks for the pasted Before and After output of the command you ran. The After results here were run together with other open fixes, not on this branch alone.

'A-Za-z\u00C0-\u00D6\u00D8-\u00F6' +
'\u00F8-\u02B8\u0300-\u0590\u0800-\u1FFF\u200E\u2C00-\uFB1C' +
'\uFE00-\uFE6F\uFEFD-\uFFFF';
'\u00F8-\u02B8\u0300-\u0590\u0800-\u1FFF\u200E\u2C00-\uD7FF' +

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: dropping the whole surrogate range also makes strong-LTR characters outside the BMP neutral. Regional-indicator flags (U+1F1E6–U+1F1FF) and CJK Extension B (U+20000) are Bidi_Class L, so browsers lay out 🇮🇱 שלום under dir="auto" left-to-right, while getTextDirection now returns 'rtl'. Cmd+Left/Right in a code block is then inverted on such a line. Regexes with the u flag that list those astral L ranges would close the gap; otherwise, a test pinning the current result would make the limitation deliberate.

expect(getTextDirection(`\uFEFC`)).toBe('rtl');
});

test('getTextDirection() skips emoji: surrogate pairs and variation selectors (#5260)', () => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: users see this change in code-block Cmd+Left/Right, through $getCodeLineDirection. A case beside the rtl code lines tests in packages/lexical-code/src/__tests__/unit/LexicalCodeNode.test.ts would cover it: a plaintext line 😀 تجربة, caret at offset 3, then MOVE_TO_END should land at offset 0 (it lands at 8 on main).

Vedthakar and others added 3 commits November 16, 2026 18:54
The LTR ranges used by getTextDirection() included the UTF-16 surrogates
\uD800-\uDFFF and the variation selectors \uFE00-\uFE0F, so a leading
emoji made text such as "😀 שלום" or "❤️ שלום" left-to-right. Leave both
out of the LTR ranges, so the direction comes from the next strong
character.

Fixes meta#5260

Signed-off-by: Ved Thakar <111605341+Vedthakar@users.noreply.github.com>
Co-authored-by: Ved Thakar <111605341+Vedthakar@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…neutral

Characters outside the Basic Multilingual Plane no longer count as
left-to-right, so say so in the JSDoc.

Co-authored-by: Ved Thakar <111605341+Vedthakar@users.noreply.github.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-authored-by: Ved Thakar <111605341+Vedthakar@users.noreply.github.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@Vedthakar

Copy link
Copy Markdown
Contributor Author

Thanks for the review. The getTextDirection JSDoc now says characters outside the Basic Multilingual Plane are treated as neutral. I left the astral strong-LTR ranges (flags, CJK Extension B) as the limitation the description names. Also rebased onto main and added a changeset.

— Ved Thakar

@rootnode-bot rootnode-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks good. Treating surrogates and U+FE00–U+FE0F as neutral fixes the emoji-first case from #5260: getTextDirection('😀 שלום') now returns 'rtl', and in a code block Home/End (Cmd+Left/Right on macOS) on such a line now moves the caret the same way as on a line of only Hebrew or Arabic. The new test fails on main and passes here.

A few optional, non-blocking suggestions:

  1. Flags now read as right-to-left (packages/lexical/src/LexicalConstants.ts:119-123). Regional-indicator symbols (U+1F1E6–U+1F1FF, i.e. every flag emoji), the Mathematical Alphanumeric letters and CJK Extension B have Unicode Bidi class L, and a browser lays out 🇮🇱 שלום under dir="auto" left-to-right. With the whole surrogate range neutral, getTextDirection('🇮🇱 שלום') now returns 'rtl' (it returned 'ltr' before), so Home/End in a code block is inverted on that line. A u-flag LTR that lists the astral L ranges while leaving the emoji blocks U+1F300–U+1FAFF neutral would keep the fix without this; otherwise a test that pins the flag result, plus a sentence in the JSDoc, would make the limitation deliberate. The description mentions CJK Extension B but not flags, which are the case people are most likely to type.
  2. Cover the path users see. The only caller in the repository is $getCodeLineDirection in @lexical/code-core. A case beside the rtl code lines tests in packages/lexical-code/src/__tests__/unit/LexicalCodeNode.test.ts would guard it: a plaintext line 😀 تجربة, caret at offset 4, then MOVE_TO_END should land at offset 0 (it lands at 8 on main).
  3. Add a Verify: paragraph to the changeset (.changeset/text-direction-ignores-emoji.md). CONTRIBUTING.md (Changesets) asks for one when a fix changes observable behaviour, for example: Verify: getTextDirection('😀 שלום') returns 'rtl'; in a code block, Cmd+Left/Right (Home/End outside macOS) on a line that starts with an emoji followed by Hebrew or Arabic moves the caret the way it does on a line of only Hebrew or Arabic.

Automated review (maintainer link)

'lexical': patch
---

[fix] `getTextDirection` no longer reads an emoji or another character outside the Basic Multilingual Plane as left-to-right, so a line that starts with an emoji followed by right-to-left text is right-to-left

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: this fix changes what getTextDirection returns and how Home/End behaves in a code block, so CONTRIBUTING.md (Changesets) asks for a Verify: paragraph, which lists the entry under "Behavior changes". For example:

Verify: getTextDirection('😀 שלום') returns 'rtl'; in a code block, Cmd+Left/Right (Home/End outside macOS) on a line that starts with an emoji followed by Hebrew or Arabic moves the caret the way it does on a line of only Hebrew or Arabic.

This branch was successfully deployed

2 active deployments
Preview – lexical — 6170ba6c Deployed Nov 16, 2026 by vercel[bot]
Preview – lexical-playground — 6170ba6c Deployed Nov 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: problem with emojies in rtl texts

1 participant