Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump NumPy to v2.0.2 for Python 3.13 compatibility #3444

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

claui
Copy link
Contributor

@claui claui commented Mar 5, 2025

Commit 66c24fa pinned NumPy to v1.x due to an upstream incompatibility in sentence-transformers.

In September 2024, Aider picked up sentence-transformers v3.1.1, which contains a fix for the NumPy 2 incompatibility. So it’s now safe to use NumPy 2.

To help unblock Python 3.13 compatibility (#3037), bump NumPy to v2.0.2, the most recent version still compatible to Python 3.9.

@claui claui marked this pull request as draft March 5, 2025 12:59
@claui
Copy link
Contributor Author

claui commented Mar 5, 2025

I'm trying to figure out whether these failures could be related to the recent downgrade to torch v2.2.2, which occurred in f266a9d.

@paul-gauthier Was the torch downgrade deliberate? Would you mind if I bumped it again?

@paul-gauthier
Copy link
Collaborator

There wasn't a "recent" downgrade to torch==2.2.2. Aider has been using that version since at least November (95b3913).

Torch got bumped up by you during your last PR. I didn't do anything intentionally to put it back to 2.2.2. I ran pip-compile --upgrade which seems to have put torch back to 2.2.2.

@claui
Copy link
Contributor Author

claui commented Mar 5, 2025

Alright, thanks for clarifying! Just trying to be extra cautious to make sure I don't tear down Chesterton's fence or stomp on anything important.

@claui
Copy link
Contributor Author

claui commented Mar 5, 2025

Ok, so the goal for this PR is to have torch==2.6.0 and numpy==2.0.2.
Apparently there are multiple forces at play which seem to be downgrading torch and numpy:

  1. pip-compile.sh seems to downgrade torch to 2.2.2. (This causes breakage, because its wheel is built against numpy 1.x, which clashes with installed numpy 2.0.2.) I'll try and figure out why pip-compile.sh would do that (but maybe outside this PR).

  2. Aider invokes pip install aider-chat[help] at runtime. This can be triggered by /help but also by the CI invoking pytest. Pip goes on to download the stable wheel from PyPI, looks at the wheel's metadata (ignoring the checked-out worktree altogether), installs Old Torch, downgrades numpy to Old NumPy, and downgrades other packages, too.
    That's a problem for CI tests, which is where unstable Aider versions run whose metadata may differ from PyPI. That can be worked around by having CI do pip install .[help] before running pytest.

@paul-gauthier
Copy link
Collaborator

I just pushed a change that will conflict with your work, sorry. I switched pip-compile.sh to use uv, which makes it much faster. Doesn't change anything else. This may make it easier for you to iterate.

@paul-gauthier
Copy link
Collaborator

It also runs with some verbosity, which may let you see why things are being pinned.

@paul-gauthier
Copy link
Collaborator

torch 2.2.2 seems to be the latest version available for x86 Macs.

https://discuss.pytorch.org/t/why-no-macosx-x86-64-build-after-torch-2-2-2-cp39-none-macosx-10-9-x86-64-whl/204546/2

So it seems wise to pin torch there, otherwise x86 Mac users can't use aider. Or we have to go down a complex platform dependent versioning path.

@claui
Copy link
Contributor Author

claui commented Mar 5, 2025

I switched pip-compile.sh to use uv, which makes it much faster. Doesn't change anything else. This may make it easier for you to iterate.

That’s good news! I’ve been meaning to get into uv, so that’s an excellent opportunity. I’ve heard only good things so far about uv.

torch 2.2.2 seems to be the latest version available for x86 Macs.

https://discuss.pytorch.org/t/why-no-macosx-x86-64-build-after-torch-2-2-2-cp39-none-macosx-10-9-x86-64-whl/204546/2

So it seems wise to pin torch there, otherwise x86 Mac users can't use aider. Or we have to go down a complex platform dependent versioning path.

Thanks for the heads up, good to know!

This is going to be really difficult. As far as I know, torch v2.2.2 is hard-wired to NumPy 1.x, which means no Python 3.13. I’m going to have to think about this for a while.

@claui
Copy link
Contributor Author

claui commented Mar 5, 2025

On the other hand: if we go ahead with torch v2.6.0, the only thing that x86 Mac users would be losing is the /help feature. All other Aider features should be working fine for them. While that’s certainly not perfect, maybe it’s an acceptable trade-off?

Commit 66c24fa pinned NumPy to v1.x due to an upstream incompatibility
in sentence-transformers.

In September 2024 [1], Aider picked up sentence-transformers v3.1.1,
which contains a fix for the NumPy 2 incompatibility [2].

To help unblock PR Aider-AI#3037 [3], bump NumPy to v2.0.2, the most recent
version still compatible to Python 3.9.

[1]: Aider-AI@2ca093f
[2]: UKPLab/sentence-transformers#2937
[3]: Aider-AI#3037
@claui
Copy link
Contributor Author

claui commented Mar 9, 2025

So it seems wise to pin torch there, otherwise x86 Mac users can't use aider.

They absolutely can use aider.
They’re just going to have to read the manual by themselves instead of using /help. Wouldn’t you consider that an acceptable trade-off so everyone else can get Python 3.13 support?

Or we have to go down a complex platform dependent versioning path.

I could give this a try if you prefer, similar to your tree-sitter.in approach in 25da067 but a bit more complex.

claui added a commit to claui/aider that referenced this pull request Mar 9, 2025
Commit 25da067 introduced a manually curated fragment for `tree-sitter`
to work around shortcomings [1] of pip-compile.

Generalize this technique by introducing a declarative blocklist and
associated script.

This feature is going to be needed by three unrelated features: fixing
`pydub`, running `pip-compile.sh` on non-Apple platforms, and exploring
the "complex platform dependent versioning path" proposed in PR Aider-AI#3444
[2].

As an added benefit, the blocklist script:

- allows comments in the blocklist; and

- removes extraneous comments from `requirements.txt`, avoiding
  misattribution and confusion if a human looks at them.

[1]: jazzband/pip-tools#639 (comment)

[2]: Aider-AI#3444 (comment)
@claui
Copy link
Contributor Author

claui commented Mar 9, 2025

@paul-gauthier Let me know if you feel that losing /help is an acceptable trade-off for Intel Mac owners. Then I’ll make this PR ready for review.

Alternatively, if you’d prefer me to explore the "complex platform dependent versioning path", then #3469 needs to be merged first.

claui added a commit to claui/aider that referenced this pull request Mar 9, 2025
Commit 25da067 introduced a manually curated fragment for `tree-sitter`
to work around shortcomings [1] of pip-compile.

Generalize this technique by introducing a declarative blocklist and
associated script.

This feature is going to be needed by three unrelated features: fixing
`pydub`, running `pip-compile.sh` on non-Apple platforms, and exploring
the "complex platform dependent versioning path" proposed in PR Aider-AI#3444
[2].

As an added benefit, the blocklist script:

- allows comments in the blocklist; and

- removes extraneous comments from `requirements.txt`, avoiding
  misattribution and confusion if a human looks at them.

[1]: jazzband/pip-tools#639 (comment)

[2]: Aider-AI#3444 (comment)
claui added a commit to claui/aider that referenced this pull request Mar 9, 2025
Commit 25da067 introduced a manually curated fragment for `tree-sitter`
to work around shortcomings [1] of pip-compile.

Generalize this technique by introducing a declarative blocklist and
associated script.

This feature is going to be needed by three unrelated features: fixing
`pydub`, running `pip-compile.sh` on non-Apple platforms, and exploring
the "complex platform dependent versioning path" proposed in PR Aider-AI#3444
[2].

As an added benefit, the blocklist script:

- allows comments in the blocklist; and

- removes extraneous comments from `requirements.txt`, avoiding
  misattribution and confusion if a human looks at them.

[1]: jazzband/pip-tools#639 (comment)

[2]: Aider-AI#3444 (comment)
claui added a commit to claui/aider that referenced this pull request Mar 9, 2025
Commit 25da067 introduced a manually curated fragment for `tree-sitter`
to work around shortcomings [1] of pip-compile.

Generalize this technique by introducing a declarative blocklist and
associated script.

This feature is going to be needed by three unrelated features: fixing
`pydub`, running `pip-compile.sh` on non-Apple platforms, and exploring
the "complex platform dependent versioning path" proposed in PR Aider-AI#3444
[2].

As an added benefit, the blocklist script:

- allows comments in the blocklist; and

- removes extraneous comments from `requirements.txt`, avoiding
  misattribution and confusion if a human looks at them.

[1]: jazzband/pip-tools#639 (comment)

[2]: Aider-AI#3444 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants