Remove unused "type: ignore" comments to appease mypy #6737
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In NumPy 2 (and possibly earlier versions), lines 478-480 produced a deprecation warning: ``` DeprecationWarning: In future, it will be an error for 'np.bool' scalars to be interpreted as an index ``` This warning is somewhat misleading: it _is_ the case that Booleans are involved, but they are not being used as indices. The fields `rs`, `xs`, and `zs` of CliffordTableau as defined in file `cirq-core/cirq/qis/clifford_tableau.py` have type `Optional[np.ndarray]`, and the values in the ndarray have NumPy type `bool` in practice. The protocol buffer version of CliffordTableau defined in file `cirq-google/cirq_google/api/v2/program_pb2.pyi` defines those fields as `collections.abc.Iterable[builtins.bool]`. At first blush, you might think they're arrays of Booleans in both cases, but unfortunately, there's a wrinkle: Python defines its built-in `bool` type as being derived from `int` (see PEP 285), while NumPy explicitly does _not_ drive its `bool` from its integer class (see <https://numpy.org/doc/2.0/reference/arrays.scalars.html#numpy.bool>). The warning about converting `np.bool` to index values (i.e., integers) probably arises when the `np.bool` values in the ndarray are coerced into Python Booleans. At first, I thought the obvious solution would be to use `np.asarray` to convert the values to `builtins.bool`, but this did not work: ``` >>> import numpy as np >>> import builtins >>> arr = np.array([True, False], dtype=np.bool) >>> arr array([ True, False]) >>> type(arr[0]) <class 'numpy.bool'> >>> newarr = np.asarray(arr, dtype=builtins.bool) >>> newarr array([ True, False]) >>> type(newarr[0]) <class 'numpy.bool'> ``` They still end up being NumPy bools. Some other variations on this approach all failed to produce proper Python Booleans. In the end, what worked was to use `map()` to apply `builtins.bool` to every value in the incoming arrays. This may not be as efficient as possible; a possible optimization for the future is to look for a more efficient way to cast the types, or avoid having to do it at all.
The NumPy 2 Migration Guide [explicitly recommends changing](https://numpy.org/doc/stable/numpy_2_0_migration_guide.html#adapting-to-changes-in-the-copy-keyword) constructs of the form ```python np.array(state, copy=False) ``` to ```python np.asarray(state) ```
NumPy 2 raises deprecation warnings about converting an ndarray with dimension > 0 of values likle `[[0]]` to a scalar value like `0`. The solution is to get the value using `.item()`.
This adds a new option to make NumPy warn about data promotion behavior that has changed in NumPy 2. This new promotion can lead to lower precision results when working with floating-point scalars, and errors or overflows when working with integer scalars. Invoking pytest with `--warn-numpy-data-promotion` will cause warnings warnings to be emitted when dissimilar data types are used in an operation in such a way that NumPy ends up changing the data type of the result value. Although this new option for Cirq's pytest code is most useful during Cirq's migration to NumPy 2, the flag will likely remain for some time afterwards too, because developers will undoubtely need time to adjust to the new NumPy behavior. For more information about the NumPy warning enabled by this option, see <https://numpy.org/doc/2.0/numpy_2_0_migration_guide.html#changes-to-numpy-data-type-promotion>.
This updates the minimum NumPy version requirement to 2.0, and updates a few other packages to versions that are compatible with NumPy 2.0. Note: NumPy 2.1 was released 3 weeks ago, but at this time, Cirq can only upgrade to 2.0. This is due to the facts that (a) Google's internal codebase is moving to NumPy 2.0.2, and not 2.1 yet; and (b) conflicts arise with some other packages used by Cirq if NumPy 2.1 is required right now. These considerations will no doubt change in the near future, at which time we can update Cirq to use NumPy 2.1 or higher.
One of the changes in NumPy 2 is to the [behavior of type promotion](https://numpy.org/devdocs/numpy_2_0_migration_guide.html#changes-to-numpy-data-type-promotion). A possible negative impact of the changes is that some operations involving scalar types can lead to lower precision, or even overflow. For example, `uint8(100) + 200` previously (in Numpy < 2.0) produced a `unit16` value, but now results in a `unit8` value and an overflow _warning_ (not error). This can have an impact on Cirq. For example, in Cirq, simulator measurement result values are `uint8`'s, and in some places, arrays of values are summed; this leads to overflows if the sum > 128. It would not be appropriate to change measurement values to be larger than `uint8`, so in cases like this, the proper solution is probably to make sure that where values are summed or otherwise numerically manipulated, `uint16` or larger values are ensured. NumPy 2 offers a new option (`np._set_promotion_state("weak_and_warn")`) to produce warnings where data types are changed. Commit 6cf50eb adds a new command-line to our pytest framework, such that running ```bash check/pytest --warn-numpy-data-promotion ``` will turn on this NumPy setting. Running `check/pytest` with this option enabled revealed quite a lot of warnings. The present commit changes code in places where those warnings were raised, in an effort to eliminate as many of them as possible. It is certainly the case that not all of the type promotion warnings are meaningful. Unfortunately, I found it sometimes difficult to be sure of which ones _are_ meaningful, in part because Cirq's code has many layers and uses ndarrays a lot, and understanding the impact of a type demotion (say, from `float64` to `float32`) was difficult for me to do. In view of this, I wanted to err on the side of caution and try to avoid losses of precision. The principles followed in the changes are roughly the following: * Don't worry about warnings about changes from `complex64` to `complex128`, as this obviously does not reduce precision. * If a warning involves an operation using an ndarray, change the code to try to get the actual data type of the data elements in the array rather than use a specific data type. This is the reason some of the changes look like the following, where it reaches into an ndarray to get the dtype of an element and then later uses the `.type()` method of that dtype to cast the value of something else: ```python dtype = args.target_tensor.flat[0].dtype ..... args.target_tensor[subspace] *= dtype.type(x) ``` * In cases where the above was not possible, or where it was obvious what the type must always be, the changes add type casts with explicit types like `complex(x)` or `np.float64(x)`. It is likely that this approach resulted in some unnecessary up-promotion of values and may have impacted run-time performance. Some simple overall timing of `check/pytest` did not reveal a glaring negative impact of the changes, but that doesn't mean real applications won't be impacted. Perhaps a future review can evaluate whether speedups are possible.
This commit for one file implements a minor refactoring of 3 test functions to make them all use similar idioms (for greater ease of reading) and to address the same NumPy 2 data promotion warnings for the remaining files in commit eeeabef.
Mypy flagged a couple of the previous data type declaration changes as being incompatible with expected types. Changing them to satisfy mypy did not affect Numpy data type promotion warnings.
* Sync with new API for checking device family in qcs-sdk-python, Ref: rigetti/qcs-sdk-rust#463 in isa.pyi * Require qcs-sdk-python-0.20.1 which introduced the new family API Fixes quantumlib#6732
Pytest was happy with the previous approach to declaring the value types in a couple of expressions, but mypy was not. This new version satisfies both.
As a consequence of [NEP 51](https://numpy.org/neps/nep-0051-scalar-representation.html#nep51), the string representation of scalar numbers changed in NumPy 2 to include type information. This affected printing Cirq circuit diagrams: instead seeing numbers like 1.5, you would see `np.float64(1.5)` and similar. The solution is to avoid getting the repr output of NumPy scalars directly, and instead doing `.item()` on them before passing them to `format()` or other string-producing functions.
The recent changes support NumPy 2 (as long as cirq-rigetti is removed manually), but they don't require NumPy 2. We can maintain compatibility with Numpy 1.x.
Bumps [serve-static](https://github.com/expressjs/serve-static) and [express](https://github.com/expressjs/express). These dependencies needed to be updated together. Updates `serve-static` from 1.15.0 to 1.16.2 - [Release notes](https://github.com/expressjs/serve-static/releases) - [Changelog](https://github.com/expressjs/serve-static/blob/v1.16.2/HISTORY.md) - [Commits](expressjs/serve-static@v1.15.0...v1.16.2) Updates `express` from 4.19.2 to 4.21.0 - [Release notes](https://github.com/expressjs/express/releases) - [Changelog](https://github.com/expressjs/express/blob/4.21.0/History.md) - [Commits](expressjs/express@4.19.2...4.21.0) --- updated-dependencies: - dependency-name: serve-static dependency-type: indirect - dependency-name: express dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michael Hucka <[email protected]>
In the current version of pytest (8.3.3) with the pytest-asyncio module version 0.24.0, we see the following warnings at the beginning of a pytest run: ``` warnings.warn(PytestDeprecationWarning(_DEFAULT_FIXTURE_LOOP_SCOPE_UNSET)) ..../lib/python3.10/site-packages/pytest_asyncio/plugin.py:208: PytestDeprecationWarning: The configuration option "asyncio_default_fixture_loop_scope" is unset. The event loop scope for asynchronous fixtures will default to the fixture caching scope. Future versions of pytest-asyncio will default the loop scope for asynchronous fixtures to function scope. Set the default fixture loop scope explicitly in order to avoid unexpected behavior in the future. Valid fixture loop scopes are: "function", "class", "module", "package", "session" ``` A [currently-open issue and discussion over in the pytest-asyncio repo](pytest-dev/pytest-asyncio#924) suggests that this is an undesired side-effect of a recent change in pytest-asyncio and is not actually a significant warning. Moreover, the discussion suggests the warning will be removed or changed in the future. In the meantime, the warning is confusing because it makes it sound like something is wrong. This simple PR silences the warning by adding a suitable pytest init flag to `pyproject.toml'.
Flagged by pylint.
mhucka
requested review from
mrwojtek,
vtomole,
a team,
wcourtney and
verult
as code owners
|
I see the CI check failures. This is baffling, because mypy does not produce errors for me locally after I remove the comments – in fact it produced errors when I left them in. Need to investigate what the difference is between what's happening in the GH workflow and local execution … |
|
You might have a different mypy version locally then on CI. Please try after For a truly reproducible local run you may want to use a fresh Python 3.10 environment and redo the CI Type check steps - Lines 62 to 75 in 484df6f |
|
Well, I use pyenv to create a virtual environment, and then always use to install the Cirq requirements, which in turn causes But okay, there is clearly a difference somewhere, so I have to chase it down. |
|
CI uses mypy-1.11.1 as specified in dev_tools/requirements/deps/mypy.txt. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #6737 +/- ##
==========================================
- Coverage 97.83% 97.83% -0.01%
==========================================
Files 1077 1077
Lines 92524 92554 +30
==========================================
+ Hits 90523 90552 +29
- Misses 2001 2002 +1 ☔ View full report in Codecov by Sentry. |
mhucka
force-pushed
the
mhucka-fix-unused-ignores
branch
from
d217f44 to
41f7883
Compare
|
Good ideas, and I'll do those tests – thank you. In the interest of time, though, I will close this PR because it's clearly not a problem for the CI workflows, and is most likely some difference in my environment. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Mypy produced errors due to a number of places containing the comment
Removing those comments made mypy happy.