feat: support Unicode line editing#73
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
Adds Unicode-aware line editing support across the readline stack so terminals can correctly handle wide CJK characters, combining marks, and supplementary code points, while preserving the incremental update fast path for single-width content and improving HTTP input decoding.
Changes:
- Teach
LineBufferto accept non-single-width code points and fall back to a full redraw when wide/combining characters are present. - Fix prompt cursor computations to use Unicode code point length (not UTF-16 char length).
- Add/extend regression tests covering Unicode editing behaviors and charset-aware HTTP decoding; bump snapshot version.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/main/java/io/termd/core/readline/LineBuffer.java | Allow non-single-width code points; choose redraw vs incremental update based on display width properties. |
| src/main/java/io/termd/core/readline/Readline.java | Compute prompt length using code points to support supplementary characters correctly. |
| src/main/java/io/termd/core/http/HttpTtyConnection.java | Decode text input bytes using the connection charset instead of platform default. |
| src/test/java/io/termd/core/readline/TestTerm.java | Update test terminal rendering model to handle wide/combining characters. |
| src/test/java/io/termd/core/readline/ReadlineTest.java | Add regression tests for CJK, combining marks, and supplementary code point prompts/inputs. |
| src/test/java/io/termd/core/readline/LineBufferTest.java | Enable previously-disabled multi-cell cursor position tests. |
| src/test/java/io/termd/core/io/BinaryEncodingTest.java | Extend binary encoding coverage for CJK UTF-8 bytes. |
| src/test/java/io/termd/core/http/HttpTtyConnectionTest.java | New regression test ensuring HTTP terminal input uses the configured charset. |
| pom.xml | Bump version to 1.1.7.16-SNAPSHOT. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+154
to
+159
| private void writeCodePoint(int codePoint) { | ||
| int cellWidth = Wcwidth.of(codePoint); | ||
| if (cellWidth == 0) { | ||
| appendCombiningCodePoint(codePoint); | ||
| return; | ||
| } |
Comment on lines
+323
to
+326
| /** | ||
| * 非单宽字符采用整行重绘。终端显示的基本单位是单元格，整行重绘可以直接处理宽字符和组合字符， | ||
| * 也能在删除组合字符时清除已经附着在基础字符上的旧字形。 | ||
| */ |
Comment on lines
130
to
134
| if ("read".equals(action)) { | ||
| lastAccessedTime = System.currentTimeMillis(); | ||
| String data = obj.getString("data"); | ||
| decoder.write(data.getBytes()); //write back echo | ||
| decoder.write(data.getBytes(charset)); // 按连接字符集进入统一解码链路 | ||
| } else if ("resize".equals(action)) { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
1.1.7.16-SNAPSHOTfor Arthas integrationWhy
The readline implementation assumed every accepted input value occupied exactly one terminal cell. As a result, CJK characters were rejected, and combining or supplementary characters could corrupt cursor positions and line updates. HTTP terminal input also relied on the platform default charset.
Impact
Termd terminals can now edit and submit Unicode command lines while preserving the existing fast path for ASCII input. Arthas consumes this snapshot to enable Chinese input across Telnet and WebSocket terminals.
Checks
mvn -Dtest=LineBufferTest,LineBufferUpdateTest,ReadlineTest,HttpTtyConnectionTest,BinaryEncodingTest,WcwidthTest test(87 tests passed on JDK 8)mvn -DskipTests install