Add more internal type hints #4105

tybug · 2024-09-13T18:42:37Z

Largely focused on non-conjecture this time. There are some source changes here as I tightened things up while typing, like no longer passing length-one strings or None to as_general_categories.

tybug · 2024-09-13T18:45:20Z

hypothesis-python/src/hypothesis/strategies/_internal/core.py

 @cacheable
 @defines_strategy(force_reusable_values=True)
 def characters(
    *,
    codec: Optional[str] = None,
    min_codepoint: Optional[int] = None,
    max_codepoint: Optional[int] = None,
-    categories: Optional[Collection[CategoryName]] = None,
-    exclude_categories: Optional[Collection[CategoryName]] = None,
+    categories: Optional[Categories] = None,


notably this is a loosening of the type, from Collection[CategoryName] to Iterable[CategoryName].

hmm, not sure about this - it works, but non-collection iterables seem like a real footgun for our users!

Something something hyrum's Law / may as well type what works? I think typing as Iterable doesn't push the footgun up to the user, so to speak – they would have to be shooting themselves already. More like Collection forces them to "fix" what in our determination is a "footgun" (but maybe they have a valid-but-cursed custom iterable type that doesn't implement e.g. __contains__).

But I am very unexperienced in maintaining forward-facing types, so I'm happy to defer this decision.

Let's stick with the status quo (Collection) here, and get the rest in 🙂

tybug · 2024-09-13T19:12:45Z

hmm, I've been testing locally with --allow-redefinition, which is the source of some of the mypy ci errors. Do we want to enable this in hypothesis? I find it ergonomic and dislike the alternative of arbitrary new names solely for uniqueness.

Zac-HD

--allow-redefinition seems fine to me

hypothesis-python/src/hypothesis/configuration.py

hypothesis-python/src/hypothesis/internal/charmap.py

Zac-HD · 2024-09-13T21:20:31Z

hypothesis-python/src/hypothesis/internal/compat.py

@@ -206,14 +218,14 @@ def get_type_hints(thing):
 # We still use the same trick on Python 3, because Numpy values and other
 # custom __floor__ or __ceil__ methods may convert via floats.
 # See issue #1667, Numpy issue 9068.
-def floor(x):
+def floor(x: float) -> int:


it's broader than this though; essentially typing.SupportsInt (& Number, except no intersection types)

can we defer this until a usage site requires it? I don't want to write a type which is subtly an overapproximation, whereas I am confident that float is an underapproximation.

I think we do use this on Numpy scalars, so maybe we already need it?

I was thinking a type-checked usage site. I did just look into typing this though and it's a rabbit hole of defining a custom protocol inheriting SupportsInt and from _typeshed import SupportsAllComparisons except then mypy errors with Non-overlapping equality check and I gave up. Let's revert and come back another day?

hypothesis-python/src/hypothesis/internal/conjecture/data.py

Zac-HD · 2024-09-13T21:24:33Z

hypothesis-python/src/hypothesis/internal/entropy.py

    # we can't use this at runtime until from_type supports
    # protocols -- breaks ghostwriter tests
    class RandomLike(Protocol):


for me, later: maybe all this takes is to mark this protocol as runtime_checkable?

hypothesis-python/src/hypothesis/internal/intervalsets.py

hypothesis-python/src/hypothesis/internal/scrutineer.py

Zac-HD · 2024-09-13T21:45:07Z

hypothesis-python/src/hypothesis/strategies/_internal/core.py

 @cacheable
 @defines_strategy(force_reusable_values=True)
 def characters(
    *,
    codec: Optional[str] = None,
    min_codepoint: Optional[int] = None,
    max_codepoint: Optional[int] = None,
-    categories: Optional[Collection[CategoryName]] = None,
-    exclude_categories: Optional[Collection[CategoryName]] = None,
+    categories: Optional[Categories] = None,


hmm, not sure about this - it works, but non-collection iterables seem like a real footgun for our users!

Zac-HD · 2024-10-05T22:12:09Z

Looking at the CI failures and typing improvements in python 3.9, I suggest we revisit this after #4125.

I've also pulled out some of the easy bits in #4126 to simplify future reviews 🙂

tybug · 2024-10-05T22:18:14Z

yeah, sorry, I haven't had time to finish this pr up. Happy to do whatever's most efficient here.

Zac-HD · 2024-10-05T22:21:28Z

oh, no worries at all about dropping stuff, just look at how old some of my PRs are getting! If working on Hypothesis would feel like an obligation instead of a fun hobby, that's a good time to go do something else instead 🙂

tybug · 2024-10-05T22:25:09Z

I can't even tell you how much I enjoy + wish I had time to work on hypothesis 😄 But, the paper takes priority for the moment.

tybug · 2024-11-09T20:44:49Z

I've brought this branch up to date, and removed some overcomplicated typing in validation.py in the interest of moving things along. I'm happy for you to make a final decision on the st.characters typing either way Zac 🙂

Zac-HD

Looks good! Comments below, then merge at will.

(unfortunately I just watched that no-longer-xfailed mypy test flake, so maybe we need to revert 😭)

hypothesis-python/src/hypothesis/errors.py

Zac-HD · 2024-11-09T20:52:57Z

hypothesis-python/src/hypothesis/internal/compat.py

@@ -206,14 +218,14 @@ def get_type_hints(thing):
 # We still use the same trick on Python 3, because Numpy values and other
 # custom __floor__ or __ceil__ methods may convert via floats.
 # See issue #1667, Numpy issue 9068.
-def floor(x):
+def floor(x: float) -> int:


I think we do use this on Numpy scalars, so maybe we already need it?

Zac-HD · 2024-11-09T21:10:36Z

hypothesis-python/src/hypothesis/strategies/_internal/core.py

 @cacheable
 @defines_strategy(force_reusable_values=True)
 def characters(
    *,
    codec: Optional[str] = None,
    min_codepoint: Optional[int] = None,
    max_codepoint: Optional[int] = None,
-    categories: Optional[Collection[CategoryName]] = None,
-    exclude_categories: Optional[Collection[CategoryName]] = None,
+    categories: Optional[Categories] = None,


Let's stick with the status quo (Collection) here, and get the rest in 🙂

tybug · 2024-11-09T22:53:34Z

Saw the same mypy flake in #4163, so I've put back the xfail here

tybug added 3 commits September 13, 2024 14:33

confidently non-behavioral type changes

e24cbb8

more complex type changes

5ce8abd

Merge remote-tracking branch 'upstream/master' into conjecture-typing

0b22e1f

tybug requested a review from Zac-HD as a code owner September 13, 2024 18:42

add release notes

0b47b9c

tybug force-pushed the conjecture-typing branch from d7139b4 to 0b47b9c Compare September 13, 2024 18:44

tybug commented Sep 13, 2024

View reviewed changes

conditionally import TypeAlias

3ecc31b

tybug force-pushed the conjecture-typing branch from d09a568 to 3ecc31b Compare September 13, 2024 19:45

Zac-HD reviewed Sep 13, 2024

View reviewed changes

tybug added 4 commits November 9, 2024 15:11

Merge branch 'master' into conjecture-typing

eb66650

fix merge, use @Final and Self in intervalset

fec4165

add allow_redefinition

a31a5e3

ignore ordering on charmap key

4c23ddb

Zac-HD approved these changes Nov 9, 2024

View reviewed changes

tybug added 2 commits November 9, 2024 17:28

Merge branch 'master' into conjecture-typing

765f027

put back xfail

af5201d

tybug enabled auto-merge November 9, 2024 22:52

tybug disabled auto-merge November 9, 2024 23:02

tybug force-pushed the conjecture-typing branch 2 times, most recently from 26e0a0b to ba09d86 Compare November 9, 2024 23:08

tybug enabled auto-merge November 9, 2024 23:16

update for review

ad455e4

tybug force-pushed the conjecture-typing branch from ba09d86 to ad455e4 Compare November 9, 2024 23:35

tybug merged commit f2b3500 into HypothesisWorks:master Nov 9, 2024
48 checks passed

tybug deleted the conjecture-typing branch November 9, 2024 23:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add more internal type hints #4105

Add more internal type hints #4105

tybug commented Sep 13, 2024 •

edited

Loading

tybug Sep 13, 2024

Zac-HD Sep 13, 2024

tybug Sep 13, 2024 •

edited

Loading

Zac-HD Nov 9, 2024

tybug commented Sep 13, 2024

Zac-HD left a comment

Zac-HD Sep 13, 2024

tybug Nov 9, 2024

Zac-HD Nov 9, 2024

tybug Nov 9, 2024

Zac-HD Sep 13, 2024

Zac-HD Sep 13, 2024

Zac-HD commented Oct 5, 2024

tybug commented Oct 5, 2024 •

edited

Loading

Zac-HD commented Oct 5, 2024

tybug commented Oct 5, 2024

tybug commented Nov 9, 2024

Zac-HD left a comment

Zac-HD Nov 9, 2024

Zac-HD Nov 9, 2024

tybug commented Nov 9, 2024

Add more internal type hints #4105

Add more internal type hints #4105

Conversation

tybug commented Sep 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tybug Sep 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tybug commented Sep 13, 2024

Zac-HD left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Zac-HD commented Oct 5, 2024

tybug commented Oct 5, 2024 • edited Loading

Zac-HD commented Oct 5, 2024

tybug commented Oct 5, 2024

tybug commented Nov 9, 2024

Zac-HD left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tybug commented Nov 9, 2024

tybug commented Sep 13, 2024 •

edited

Loading

tybug Sep 13, 2024 •

edited

Loading

tybug commented Oct 5, 2024 •

edited

Loading