[mypyc] feat: cache len for iterating over immutable types and expressions with length known at compile time #19503
+1,099
−149
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Looks like my tests failed only on 32-bit python, because the generated IR uses 32-bit size numbers? Not sure what to do here, I don't think this failure is indicative of an issue in the PR. |
BobTheBuidler
changed the title
2 tasks
|
@ilevkivskyi this one is also ready for review if and when you get a chance, though its definitely a bit more involved than #19497 |
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
BobTheBuidler
force-pushed
the
for-loop-len-cache
branch
from
07f40fc to
622f38f
Compare
BobTheBuidler
commented
Aug 3, 2025
| @@ -1147,3 +1187,33 @@ def gen_step(self) -> None: | |||
| def gen_cleanup(self) -> None: | |||
| for gen in self.gens: | |||
| gen.gen_cleanup() | |||
|
|
|||
|
|
|||
| def get_expr_length(expr: Expression) -> int | None: | |||
There was a problem hiding this comment.
These 2 helper functions can be extended to cover more cases and used for other length-based optimizations I have in mind
for more information, see https://pre-commit.ci
…/mypy into for-loop-len-cache
BobTheBuidler
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, if a user has a tuple and uses it as the sequence input for a for loop, the tuple length is checked once at each iteration which, while necessary for some container types such as list and dictionaries, is not necessary for iterating over immutable types tuple, str, and bytes.
This PR modifies the codebase such that the length is only checked at the first iteration, and reused from there.
Also, in cases where a simple genexp is the input argument for a tuple, the length is currently checked one additional time before entering the iteration. In those cases, we don't even need a length check at the first iteration step, and can read the result of that first
lencall instead.Lastly, in cases where a tuple or list is created and the length is knowable at compile time, this PR replaces length checks with constants.