Skip to content

fix: resolve normalized package names to their module#10165

Merged
mscolnick merged 1 commit into
mainfrom
ms/columbia
Jul 15, 2026
Merged

fix: resolve normalized package names to their module#10165
mscolnick merged 1 commit into
mainfrom
ms/columbia

Conversation

@mscolnick

Copy link
Copy Markdown
Contributor

Closes #9801

Problem

PackageManager.package_to_module was case-sensitive. PyPI package names,
however, are case-insensitive and treat runs of -, _, and . as
equivalent (PEP 503). uv normalizes names when it writes them into a
notebook's PEP 723 script metadata — e.g. Pillow becomes pillow — so the
reverse lookup silently failed:

mgr.package_to_module("pillow")        # -> "pillow"        (should be "PIL")
mgr.package_to_module("Scikit-Learn")  # -> "Scikit_Learn"  (should be "sklearn")

In a sandbox this is the root cause of the confusing PIL/Pillow behavior in
the issue: after marimo installs pillow, it maps the installed package name
back to a module to decide which cells to re-run. Because pillow resolved to
the module pillow (which nothing imports) instead of PIL, the cell that
imports PIL was never re-run, forcing the user to manually toggle the import
every time they reopened the notebook.

Fix

package_to_module now falls back to a PEP 503-normalized lookup when there's
no exact match, before guessing. The known-name mapping is still consulted
exactly first, so existing casing is preserved.

mgr.package_to_module("pillow")        # -> "PIL"
mgr.package_to_module("PILLOW")        # -> "PIL"
mgr.package_to_module("Scikit-Learn")  # -> "sklearn"
mgr.package_to_module("scikit_learn")  # -> "sklearn"

Unknown packages still fall back to the previous -_ behavior.

Closes #9801

## Problem

`PackageManager.package_to_module` was case-sensitive. PyPI package names,
however, are case-insensitive and treat runs of `-`, `_`, and `.` as
equivalent (PEP 503). uv normalizes names when it writes them into a
notebook's PEP 723 script metadata — e.g. `Pillow` becomes `pillow` — so the
reverse lookup silently failed:

```python
mgr.package_to_module("pillow")        # -> "pillow"        (should be "PIL")
mgr.package_to_module("Scikit-Learn")  # -> "Scikit_Learn"  (should be "sklearn")
```

In a sandbox this is the root cause of the confusing PIL/Pillow behavior in
the issue: after marimo installs `pillow`, it maps the installed package name
back to a module to decide which cells to re-run. Because `pillow` resolved to
the module `pillow` (which nothing imports) instead of `PIL`, the cell that
imports `PIL` was never re-run, forcing the user to manually toggle the import
every time they reopened the notebook.

## Fix

`package_to_module` now falls back to a PEP 503-normalized lookup when there's
no exact match, before guessing. The known-name mapping is still consulted
exactly first, so existing casing is preserved.

```python
mgr.package_to_module("pillow")        # -> "PIL"
mgr.package_to_module("PILLOW")        # -> "PIL"
mgr.package_to_module("Scikit-Learn")  # -> "sklearn"
mgr.package_to_module("scikit_learn")  # -> "sklearn"
```

Unknown packages still fall back to the previous `-` → `_` behavior.
Copilot AI review requested due to automatic review settings July 13, 2026 18:44
@vercel

vercel Bot commented Jul 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
marimo-docs Ready Ready Preview, Comment Jul 13, 2026 6:45pm

Request Review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes PackageManager.package_to_module reverse lookups when package names are PEP 503-normalized (case-insensitive and treating runs of -, _, . as equivalent), which is important for environments where uv writes normalized dependency names into PEP 723 metadata (e.g. Pillowpillow). This addresses the “PIL/Pillow” sandbox rerun behavior described in #9801 by correctly mapping normalized package names back to their canonical import module (e.g. pillowPIL).

Changes:

  • Add a PEP 503 normalization helper and a lazily-built normalized reverse mapping for package-name-to-module lookups.
  • Update package_to_module to try: exact match → normalized match → legacy -_ heuristic.
  • Add a regression test covering normalized/case-variant inputs for Pillow/PIL and scikit-learn/sklearn.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
marimo/_runtime/packages/package_manager.py Adds PEP 503 normalization and a normalized fallback lookup in package_to_module while preserving exact-match behavior first.
tests/_runtime/packages/test_pypi_package_manager.py Adds regression test ensuring normalized/case-variant package names resolve to the correct module.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Architecture diagram
sequenceDiagram
    participant Caller as Cell / Kernel
    participant PM as PackageManager
    participant Init as _initializeMappings()
    participant ExactMap as _repo_name_to_module_name
    participant NormMap as _normalized_repo_name_to_module_name
    participant Normalize as _normalize_package_name()

    Note over Caller,Normalize: After uv installs a package, marimo resolves its name<br/>to module to decide which cells to re-run (PR issue #9801)

    Caller->>PM: package_to_module("pillow")
    activate PM

    PM->>Init: _initialize_mappings() if None
    activate Init
    Init->>ExactMap: reverse of module_name_to_repo_name
    Init->>NormMap: normalize keys of ExactMap (NEW)
    deactivate Init

    PM->>ExactMap: "pillow" exact match?
    ExactMap-->>PM: miss (only has "Pillow")

    alt Exact match miss
        PM->>Normalize: _normalize_package_name("pillow")
        activate Normalize
        Normalize-->>PM: "pillow"
        deactivate Normalize

        PM->>NormMap: "pillow" normalized key exists?
        NormMap-->>PM: yes → "PIL"
        Note over PM: NEW: PEP 503 normalized lookup<br/>preserves correct module name

        PM-->>Caller: "PIL"
    else Exact match found (e.g., "Pillow")
        ExactMap-->>PM: "PIL"
        PM-->>Caller: "PIL"
    end

    deactivate PM

    Note over Caller,Normalize: Unknown packages still fall back to original `-` → `_` guess
Loading

Re-trigger cubic

@mscolnick
mscolnick requested a review from kirangadhave July 13, 2026 19:19
@mscolnick mscolnick added the bug Something isn't working label Jul 13, 2026
@mscolnick
mscolnick merged commit e5bef92 into main Jul 15, 2026
38 of 46 checks passed
@mscolnick
mscolnick deleted the ms/columbia branch July 15, 2026 18:20
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Development release published. You may be able to view the changes at https://marimo.app?v=0.23.15-dev19

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unable to import pillow/PIL

3 participants