Describe the feature request
Describe the feature request*
Foundry Local currently provides separate hardware-specific model variants for CPU, GPU, and NPU execution.
For the same logical model, users may need to download and cache multiple variants if they want to switch between CPU, GPU, and NPU. For large models, this can result in significant duplicated storage usage.
I would like Foundry Local to support a shared model storage architecture where common model data, especially large model weights and tokenizer files, can be reused across CPU/GPU/NPU variants whenever technically possible.
For example:
Model
├── shared/
│ ├── weights
│ └── tokenizer
├── cpu/
│ └── device-specific artifacts
├── gpu/
│ └── device-specific artifacts
└── npu/
└── device-specific artifacts
The request is not to remove hardware-specific variants completely. I understand that different devices may require different quantization formats, graph optimizations, execution-provider-specific layouts, or compiled artifacts.
Instead, Foundry Local could deduplicate or share identical portions of model variants, while storing only genuinely hardware-specific data separately.
Possible implementations could include shared base weights, content-addressable storage, block-level deduplication, or runtime-generated per-device optimization caches.
Describe scenario use case
Describe scenario use case*
A typical AI PC may contain all three compute devices:
As a developer, I may want to allow users to select the inference device dynamically:
Inference Device
○ Auto
○ CPU
○ GPU
○ NPU
This is useful for several scenarios:
- Benchmarking CPU, GPU, and NPU performance on the same machine.
- Comparing power consumption and performance between devices.
- Allowing users to choose between performance and power efficiency.
- Falling back to CPU when GPU or NPU resources are unavailable.
- Testing compatibility across different execution providers.
- Developing applications that can dynamically select the most appropriate compute device.
With the current hardware-variant model, supporting this scenario may require downloading three separate variants of the same model.
For small models this may be acceptable, but for larger models the storage overhead becomes significant.
For example, if each hardware variant is approximately 10 GB:
CPU variant ~10 GB
GPU variant ~10 GB
NPU variant ~10 GB
----------------------
Total ~30 GB
Ideally, the large common model data would only need to be stored once, with smaller hardware-specific artifacts cached separately.
Customer requirement and impact
Customer requirement and impact*
The main requirement is to reduce duplicated disk usage when the same logical model is installed for multiple hardware targets.
This becomes increasingly important as model sizes grow.
Users with CPU, GPU, and NPU capable systems should ideally be able to take advantage of all available compute devices without having to maintain multiple full copies of the same model whenever a significant portion of the underlying data is identical.
The expected benefits would include:
- Reduced disk usage for multi-device model installations.
- Reduced download size when adding another hardware variant.
- More practical CPU/GPU/NPU switching.
- Better support for large local models.
- Easier benchmarking and development across different accelerators.
- Better utilization of modern AI PCs containing multiple compute devices.
- More efficient model cache management.
For example, a large model may require tens of gigabytes per variant. Keeping CPU, GPU, and NPU versions could therefore consume tens or even hundreds of gigabytes of additional storage.
Even partial deduplication of shared model weights would significantly improve this situation.
A preferred design would be:
Shared model weights
│
├── CPU-specific cache/artifacts
├── GPU-specific cache/artifacts
└── NPU-specific cache/artifacts
rather than requiring three fully independent model copies whenever the underlying weights can be shared.
This would preserve Foundry Local's hardware-optimized variant architecture while making multi-device usage much more storage-efficient.
Describe the feature request
Describe the feature request*
Foundry Local currently provides separate hardware-specific model variants for CPU, GPU, and NPU execution.
For the same logical model, users may need to download and cache multiple variants if they want to switch between CPU, GPU, and NPU. For large models, this can result in significant duplicated storage usage.
I would like Foundry Local to support a shared model storage architecture where common model data, especially large model weights and tokenizer files, can be reused across CPU/GPU/NPU variants whenever technically possible.
For example:
The request is not to remove hardware-specific variants completely. I understand that different devices may require different quantization formats, graph optimizations, execution-provider-specific layouts, or compiled artifacts.
Instead, Foundry Local could deduplicate or share identical portions of model variants, while storing only genuinely hardware-specific data separately.
Possible implementations could include shared base weights, content-addressable storage, block-level deduplication, or runtime-generated per-device optimization caches.
Describe scenario use case
Describe scenario use case*
A typical AI PC may contain all three compute devices:
As a developer, I may want to allow users to select the inference device dynamically:
This is useful for several scenarios:
With the current hardware-variant model, supporting this scenario may require downloading three separate variants of the same model.
For small models this may be acceptable, but for larger models the storage overhead becomes significant.
For example, if each hardware variant is approximately 10 GB:
Ideally, the large common model data would only need to be stored once, with smaller hardware-specific artifacts cached separately.
Customer requirement and impact
Customer requirement and impact*
The main requirement is to reduce duplicated disk usage when the same logical model is installed for multiple hardware targets.
This becomes increasingly important as model sizes grow.
Users with CPU, GPU, and NPU capable systems should ideally be able to take advantage of all available compute devices without having to maintain multiple full copies of the same model whenever a significant portion of the underlying data is identical.
The expected benefits would include:
For example, a large model may require tens of gigabytes per variant. Keeping CPU, GPU, and NPU versions could therefore consume tens or even hundreds of gigabytes of additional storage.
Even partial deduplication of shared model weights would significantly improve this situation.
A preferred design would be:
rather than requiring three fully independent model copies whenever the underlying weights can be shared.
This would preserve Foundry Local's hardware-optimized variant architecture while making multi-device usage much more storage-efficient.