Skip to content

[Download] Resolve the revision once at the beginning of from_pretrained - #14340

Draft
Wauplin wants to merge 2 commits into
huggingface:mainfrom
Wauplin:use-resolve-revision
Draft

[Download] Resolve the revision once at the beginning of from_pretrained#14340
Wauplin wants to merge 2 commits into
huggingface:mainfrom
Wauplin:use-resolve-revision

Conversation

@Wauplin

@Wauplin Wauplin commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Follow-up on huggingface_hub v1.26.0, which shipped HfApi.resolve_revision / ResolvedRevision.

What

Loading a model or a pipeline fetches several files from the same repo one by one (model_index.json, config.json, the weight index, each checkpoint shard, custom code, ...). Each of those calls resolves revision="main" into a commit hash on its own — one HTTP request per file, and no guarantee that two calls land on the same commit if the repo is updated in between.

This PR resolves the revision once, at the beginning of the loading methods, and passes the resulting ResolvedRevision down. Since it is a str subclass whose string value stays the user-facing revision ("main"), nothing else had to change: error messages keep saying main, and the download helpers (hf_hub_download, snapshot_download, get_cached_repo_tree) pick up .resolved transparently.

Where

A single _resolve_revision helper in utils/hub_utils.py, called from:

  • DiffusionPipeline.from_pretrained and DiffusionPipeline.download
  • ModelMixin.from_pretrained
  • AutoPipelineForText2Image / Image2Image / Inpainting / Text2Audio.from_pretrained
  • ModularPipeline.from_pretrained and ModularPipelineBlocks.from_pretrained

Deliberately left out, since they fetch a single file from the repo and resolving would only add a request: ConfigMixin.load_config / SchedulerMixin.from_pretrained, _fetch_state_dict (LoRA), from_single_file, textual inversion, and the DDUF path. load_ip_adapter is also left out because a single revision there can span several repos.

The helper is best-effort by design: local folders are returned untouched, and if the Hub can't answer (repo/revision not found, offline with an empty cache, invalid repo id) the original revision is returned so the download that follows raises its usual, diffusers-flavored error instead of a RevisionResolutionError leaking from the top of from_pretrained.

Measurements

HTTP calls (telemetry excluded), counted by wrapping httpx.Client.send:

cold cache warm cache
DiffusionPipeline.from_pretrained (tiny-stable-diffusion-torch) 33 → 33 3 → 2
ModelMixin.from_pretrained (single-file unet) 6 → 7 4 → 3
ModelMixin.from_pretrained (2-shard transformer) 12 → 12 6 → 4

The win is on reuse; a cold single-file load pays one extra repo_info. The commit-consistency guarantee applies in all cases.

Also in this PR

  • Bumped the minimum huggingface_hub to 1.26.0 (from 1.23.0) in setup.py and dependency_versions_table.py.
  • Two test fixes made necessary by the new code path:
    • test_kwargs_local_files_only wrote <sha>hug into refs/main; that value is now read back by resolve_revision and passed around as a commit hash, so it has to stay a syntactically valid one.
    • test_local_files_only_with_sharded_checkpoint raised HfHubHTTPError(response=mock.Mock()); resolve_revision reads error.response.status_code to tell a Hub outage from a definitive answer, so the mock now carries the real response.

Possible follow-up

Now that revision is always resolved before load_config, the commit_hash plumbing in ModelMixin.from_pretrained (load_config(return_commit_hash=True)_get_model_file(revision=revision or commit_hash)) is redundant. Left alone here since return_commit_hash is public API.

Testing

Ran locally against huggingface_hub==1.26.0:

  • tests/pipelines/test_pipelines.py — 53 passed, 33 skipped, 10 xpassed
  • tests/pipelines/test_pipelines_auto.py — 19 passed, 3 skipped
  • tests/pipelines/test_pipeline_utils.py, tests/others/, tests/lora/test_lora_loader_utils.py — pass (tests/others/test_cli_commands.py::test_toplevel_help_lists_all_commands fails on main too)
  • tests/models/test_modeling_common.py — 12 passed, 5 skipped
  • tests/modular_pipelines/test_modular_pipelines_custom_blocks.py, test_components_manager.py, test_conditional_pipeline_blocks.py — 47 passed, 6 skipped

🤖 Generated with Claude Code

Use `huggingface_hub.resolve_revision` (new in huggingface_hub 1.26.0) at the
top of the loading entrypoints so that every file fetched afterwards is pinned
to the same commit and can be served from the cache without re-resolving the
revision on each call.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@Wauplin

Wauplin commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator Author

CI status: everything green except Hub tests for models, schedulers, and pipelines, which fails on TestModelPushToHub::test_push_to_hub / test_push_to_hub_in_organization with 504 Gateway Timeout from hub-ci on /api/repos/create and /api/repos/delete. Unrelated to this PR — push_to_hub never goes through resolve_revision, and the same two tests time out the same way on other branches (e.g. run 30535166896, branch lora-tests-migration-pipelines, which predates this PR). Re-running the job kept the same signature.

Green: check_code_quality, check_repository_consistency, check_auto_docs, check_dependencies, check_torch_dependencies, Fast PyTorch Pipeline / Models & Schedulers / Modular Pipeline CPU tests, PyTorch Example CPU tests, Torch CUDA Tests (models, others, schedulers, lora), Torch Pipelines CUDA Tests, doc build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants