Skip to content

gh-151461: Fix encoding-related exception handling in file tokenizer#151462

Merged
serhiy-storchaka merged 13 commits into
python:mainfrom
johnslavik:gh-151461-file-tokenizer-fix-error-handling
Jun 14, 2026
Merged

gh-151461: Fix encoding-related exception handling in file tokenizer#151462
serhiy-storchaka merged 13 commits into
python:mainfrom
johnslavik:gh-151461-file-tokenizer-fix-error-handling

Conversation

@johnslavik

@johnslavik johnslavik commented Jun 14, 2026 •
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Member

@johnslavik johnslavik added needs backport to 3.14 bugs and security fixes needs backport to 3.15 pre-release feature fixes, bugs and security fixes labels Jun 14, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Jun 14, 2026
Closed
@johnslavik johnslavik changed the title gh-151461: Fix exception handling in file tokenizer gh-151461: Fix encoding-related exception handling in file tokenizer Jun 14, 2026
serhiy-storchaka
serhiy-storchaka approved these changes Jun 14, 2026
View reviewed changes

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. 👍

Comment thread Lib/test/test_source_encoding.py Outdated
Comment thread Lib/test/test_source_encoding.py Outdated
@johnslavik

johnslavik commented Jun 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Side note, this PR revealed an edge case we couldn't otherwise have known:

$ ./python.exe <(echo $'# coding: foo')
OSError: [Errno 29] Illegal seek

@serhiy-storchaka serhiy-storchaka enabled auto-merge (squash) June 14, 2026 12:15
@serhiy-storchaka serhiy-storchaka merged commit f1a5f68 into python:main Jun 14, 2026
54 checks passed
@miss-islington-app

miss-islington-app Bot commented Jun 14, 2026

Copy link
Copy Markdown

Thanks @johnslavik for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14, 3.15.
🐍🍒⛏🤖

@miss-islington-app

miss-islington-app Bot commented Jun 14, 2026

Copy link
Copy Markdown

Sorry, @johnslavik and @serhiy-storchaka, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker f1a5f68e3761e010ccd4dda1342500c5ae40bbc4 3.14

@bedevere-app

bedevere-app Bot commented Jun 14, 2026

Copy link
Copy Markdown

GH-151470 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Jun 14, 2026
@bedevere-app

bedevere-app Bot commented Jun 14, 2026

Copy link
Copy Markdown

GH-151474 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Jun 14, 2026
serhiy-storchaka pushed a commit that referenced this pull request Jun 14, 2026
@miss-islington @johnslavik
[3.15] gh-151461: Fix encoding-related exception handling in file tok…
89e178d 
…enizer (GH-151462) (GH-151470)

(cherry picked from commit f1a5f68)

Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
serhiy-storchaka pushed a commit that referenced this pull request Jun 14, 2026
@johnslavik
[3.14] gh-151461: Fix encoding-related exception handling in file tok…
3ad080a 
…enizer (GH-151462) (GH-151474)
@johnslavik johnslavik deleted the gh-151461-file-tokenizer-fix-error-handling branch June 14, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@serhiy-storchaka serhiy-storchaka serhiy-storchaka approved these changes

@pablogsal pablogsal Awaiting requested review from pablogsal pablogsal is a code owner

@lysnikolaou lysnikolaou Awaiting requested review from lysnikolaou lysnikolaou is a code owner

Assignees

@serhiy-storchaka serhiy-storchaka

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@johnslavik @serhiy-storchaka