Skip to content

Clean rustc/parse/src/lexer to improve maintainability #144239

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Clean rustc/parse/src/lexer to improve maintainability #144239

wants to merge 1 commit into from
+83 −85

Conversation

xizheyin
Copy link
Contributor

This PR refactors the lexer code to improve maintainability and eliminate code duplication.
In the first commit, I improve the error handling:

  • rename make_unclosed_delims_error to more appropriate make_mismatched_closing_delims_errors
  • changes return type from Option to Vec<Diag> to avoid lengthy vec processing at lex_token_trees
  • use splice instead of extend to make the logic clearer, since errs sounds more generic and better suited as a return value

In the second commit, I replace the magic number 5 with UNCLOSED_DELIMITER_SHOW_LIMIT constant.

In the third commit, I moves eof_err function below parsing logic for better code flow.

In the forth one, I extract calculate_spacing function to eliminate duplicate spacing logic between bump and bump_minimal functions.

r? compiler

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jul 21, 2025
@xizheyin xizheyin changed the title Clean lexer Clean rustc/parse/src/lexer to improve maintainability Jul 21, 2025
fee1-dead
fee1-dead requested changes Jul 21, 2025
View reviewed changes
@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 21, 2025
@xizheyin
Copy link
Contributor Author

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 22, 2025
@fee1-dead
Copy link
Member

Please squash the commits

@xizheyin
Copy link
Contributor Author

ready. :)

@fee1-dead
Copy link
Member

@bors r+ rollup

@bors
Copy link
Collaborator

bors commented Jul 22, 2025

📌 Commit 7a50fe7 has been approved by fee1-dead

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 22, 2025
@fee1-dead
Copy link
Member

@bors r-

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jul 22, 2025
fee1-dead
fee1-dead requested changes Jul 22, 2025
View reviewed changes
@xizheyin
Clean code for rustc_parse/src/lexer
2832517 
1. Rename `make_unclosed_delims_error` and return `Vec<Diag>`
2. change magic number `unclosed_delimiter_show_limit` to const
3. move `eof_err` below parsing logic
4. Add `calculate_spacing` for `bump` and `bump_minimal`

Signed-off-by: xizheyin <[email protected]>
@xizheyin
Copy link
Contributor Author

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fee1-dead fee1-dead fee1-dead requested changes

Assignees

@fee1-dead fee1-dead

Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@xizheyin @fee1-dead @bors @rustbot