Skip to content

[ty] Optimize enum comparisons in equality evaluation#26340

Merged
charliermarsh merged 16 commits into
mainfrom
charlie/generalize-enum-comparisons
Jun 25, 2026
Merged

[ty] Optimize enum comparisons in equality evaluation#26340
charliermarsh merged 16 commits into
mainfrom
charlie/generalize-enum-comparisons

Conversation

@charliermarsh

@charliermarsh charliermarsh commented Jun 24, 2026

Copy link
Copy Markdown
Member

Summary

Since #25955 and #26337, equality narrowing, match reachability, membership compatibility, and ordinary == and != result inference all use the same comparison evaluator, within which comparing large enums ends up performing a large member cross-product operation.

This PR adds a fast path for enums, whereby same-class operands use compact member sets and cross-class / multi-class unions project members onto cached runtime comparison keys. The cache is per enum class and comparison operator, so repeated comparisons reuse the class-wide scan instead of repeating it for every type pair or expression. Because it's in the shared evaluator, this optimization applies consistently to equality and inequality narrowing, definite comparison results and resulting reachability, value-pattern narrowing, and membership compatibility.

(Note: SameEnumComparison is a fast path within the fast path for enums of the same class. Within the same class, we can just compare member-name sets; for different classes, we need project down to a value.)

In quick local benchmarks against current main, the original astral-sh/ty#3830 reproducer drops from roughly 14.4 s to 9.3 ms, a comparison between two 256-member StrEnums drops from roughly 242 ms to 10.8 ms, and a comparison between unions spanning 16 enum classes drops from roughly 296 ms to 10.9 ms.

Closes astral-sh/ty#3830.

@charliermarsh charliermarsh added the ty Multi-file analysis & type inference label Jun 24, 2026
@astral-sh-bot

astral-sh-bot Bot commented Jun 24, 2026

Copy link
Copy Markdown

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 94.47%. The percentage of expected errors that received a diagnostic held steady at 89.19%. The number of fully passing files held steady at 95/134.

@astral-sh-bot

astral-sh-bot Bot commented Jun 24, 2026

Copy link
Copy Markdown

Memory usage report

Summary

Project Old New Diff Outcome
prefect 448.27MB 448.28MB +0.00% (11.95kB)
trio 70.29MB 70.30MB +0.00% (2.86kB)
sphinx 166.58MB 166.58MB +0.00% (160.00B)
flake8 28.95MB 28.95MB -

Significant changes

Click to expand detailed breakdown

prefect

Name Old New Diff Outcome
enum_class_key_profile 0.00B 6.55kB +6.55kB (new)
enum_metadata 1.69MB 1.69MB +0.18% (3.11kB)
infer_expression_types_impl 37.83MB 37.83MB -0.01% (2.66kB)
EnumComplementType 2.66kB 4.60kB +72.73% (1.94kB)
same_enum_comparison_profile::interned_arguments 0.00B 1.31kB +1.31kB (new)
enum_class_key_profile::interned_arguments 0.00B 1.31kB +1.31kB (new)
IntersectionType<'db>::from_two_elements_::interned_arguments 12.20kB 13.23kB +8.45% (1.03kB)
same_enum_comparison_profile 0.00B 1008.00B +1008.00B (new)
IntersectionType<'db>::from_two_elements_ 8.92kB 9.70kB +8.67% (792.00B)
is_redundant_with_impl::interned_arguments 2.35MB 2.35MB -0.03% (792.00B)
infer_definition_types 49.96MB 49.96MB -0.00% (680.00B)
all_narrowing_constraints_for_expression 5.21MB 5.21MB +0.01% (680.00B)
IntersectionType 984.91kB 984.33kB -0.06% (600.00B)
UnionType 1.07MB 1.07MB -0.04% (432.00B)
analyze_non_terminal_call 1.77MB 1.77MB -0.02% (296.00B)
... 10 more

trio

Name Old New Diff Outcome
enum_class_key_profile 0.00B 5.19kB +5.19kB (new)
infer_expression_types_impl 4.39MB 4.39MB -0.09% (3.87kB)
same_enum_comparison_profile::interned_arguments 0.00B 448.00B +448.00B (new)
enum_class_key_profile::interned_arguments 0.00B 448.00B +448.00B (new)
all_narrowing_constraints_for_expression 461.26kB 460.91kB -0.07% (352.00B)
IntersectionType<'db>::from_two_elements_::interned_arguments 4.90kB 5.24kB +7.02% (352.00B)
same_enum_comparison_profile 0.00B 336.00B +336.00B (new)
infer_definition_types 3.97MB 3.97MB -0.01% (280.00B)
IntersectionType<'db>::from_two_elements_ 4.02kB 4.25kB +5.84% (240.00B)
IntersectionType 141.02kB 141.20kB +0.13% (184.00B)
is_redundant_with_impl::interned_arguments 237.45kB 237.62kB +0.07% (176.00B)
is_redundant_with_impl 129.61kB 129.69kB +0.06% (80.00B)
infer_deferred_types 1.20MB 1.20MB -0.00% (32.00B)
loop_header_reachability 71.24kB 71.23kB -0.01% (8.00B)
Type<'db>::apply_specialization_inner_ 349.82kB 349.81kB -0.00% (8.00B)
... 1 more

sphinx

Name Old New Diff Outcome
infer_expression_types_impl 14.77MB 14.77MB -0.01% (1.91kB)
all_narrowing_constraints_for_expression 1.93MB 1.93MB -0.07% (1.39kB)
enum_class_key_profile 0.00B 608.00B +608.00B (new)
is_redundant_with_impl::interned_arguments 1.14MB 1.14MB +0.04% (528.00B)
Specialization 1.41MB 1.41MB +0.02% (336.00B)
FunctionType<'db>::signature_ 1.61MB 1.61MB +0.02% (304.00B)
is_redundant_with_impl 631.37kB 631.63kB +0.04% (272.00B)
enum_metadata 426.63kB 426.88kB +0.06% (256.00B)
EnumComplementType 248.00B 496.00B +100.00% (248.00B)
TupleType<'db>::to_class_type_ 69.62kB 69.84kB +0.30% (216.00B)
GenericAlias 620.09kB 620.30kB +0.03% (216.00B)
same_enum_comparison_profile::interned_arguments 0.00B 128.00B +128.00B (new)
enum_class_key_profile::interned_arguments 0.00B 128.00B +128.00B (new)
same_enum_comparison_profile 0.00B 96.00B +96.00B (new)
IntersectionType<'db>::from_two_elements_::interned_arguments 5.76kB 5.84kB +1.49% (88.00B)
... 6 more

@astral-sh-bot

astral-sh-bot Bot commented Jun 24, 2026

Copy link
Copy Markdown

ecosystem-analyzer results

Lint rule Added Removed Changed
invalid-argument-type 0 1 0
Total 0 1 0

Raw diff:

rotki (https://github.com/rotki/rotki)
- rotkehlchen/api/services/transactions.py:792:40 error[invalid-argument-type] Argument to bound method `list.extend` is incorrect: Expected `Iterable[Literal[SupportedBlockchain.ETHEREUM, SupportedBlockchain.OPTIMISM, SupportedBlockchain.POLYGON_POS, SupportedBlockchain.ARBITRUM_ONE, SupportedBlockchain.BASE, ... omitted 6 literals]]`, found `list[SupportedBlockchain]`

Full report with detailed diff (timing results)

@charliermarsh charliermarsh force-pushed the charlie/generalize-enum-comparisons branch from aa2f03a to 3a15517 Compare June 24, 2026 22:03
@charliermarsh charliermarsh force-pushed the charlie/generalize-enum-comparisons branch from 3a15517 to e8bf30e Compare June 25, 2026 17:51
@charliermarsh charliermarsh added the performance Potential performance improvement label Jun 25, 2026
@charliermarsh charliermarsh marked this pull request as ready for review June 25, 2026 18:24
@charliermarsh charliermarsh requested a review from a team as a code owner June 25, 2026 18:24
@astral-sh-bot astral-sh-bot Bot requested a review from carljm June 25, 2026 18:24
@charliermarsh charliermarsh changed the title [ty] Generalize enum domain comparisons [ty] Optimize enum comparisons in equality evaluation Jun 25, 2026

@carljm carljm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great results, both semantic and perf!

I admit I did not read the entire new 900-line enum-compare implementation line-by-line. I did interrogate Codex about the key abstractions in it, and they make sense to me. I did read the mdtests.

Comment on lines +202 to +206
class CoercingAlias(str, Enum):
FIRST = 1
SECOND = "1"

reveal_type(CoercingAlias.FIRST == CoercingAlias.SECOND) # revealed: bool

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex points out that this somewhat similar case regresses in precision from main to this PR:

from enum import IntEnum

class E(IntEnum):
    ZERO = 0
    FALSE = False

reveal_type(E.ZERO == E.FALSE)  # Literal[True] on main, bool here

OTOH I think maybe this is fine; OTOH modeling that False is 0 does feel a little more "core" than modeling that str("1") is 1, and might not be hard?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that will get fixed by #26349.

@charliermarsh charliermarsh merged commit 855ab30 into main Jun 25, 2026
63 checks passed
@charliermarsh charliermarsh deleted the charlie/generalize-enum-comparisons branch June 25, 2026 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Potential performance improvement ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Repeated StrEnum comparisons become very slow after truthiness narrowing

2 participants