Cannot load module 'Llama' as 'llama' when running the test suite

[spicymatt]

Hi
First of all, thanks for putting together such an elegant package.
I wanted to test GGUF local models and failed to compile my test app after enabling the trait 'Llama'...so I figured I would test this using the package's test suite. But unfortunately I get the same error. I must be doing something wrong.

1. I enabled the 'Llama' trait inside the Package definition:
   .default(enabledTraits: ["Llama"]),
2. I checked that the test LlamaLanguageModelTests was enabled and that #if Llama was true

I still get the Cannot load module ... error

Any help with this would be greatly appreciated
Regards

Matthieu

[mattt]

Hi @spicymatt. I'm sorry this isn't working as expected. I just tried this myself and it was a partial success — I could generate basic responses just fine, but running the full test suite consistently crashed Terminal (!)

Part of this has to do with the unnecessarily lax requirements currently in place for the llama.swift / llama.cpp XCFramework dependency, which is basically "give me the latest version". Pinning to the specific release I developed against ("1.6818.0") improved things somewhat, but there are clearly some problems with the current implementation.

But taking a step back, let's dig into the "Cannot load module" error:

Here are the command I ran locally to get this working:

# Install Hugging Face CLI
$ curl -LsSf https://hf.co/cli/install.sh | bash

# Authenticate
$ hf auth login

# Download a small GGUF model
$ hf download TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF

# Get a path to the blob corresponding to a GGUF file
$ readlink -f $(ls -S ~/.cache/huggingface/hub/models--TheBloke--TinyLlama-1.1B-Chat-v1.0-GGUF/snapshots/**Q8_0.gguf)
# /Users/mattt/.cache/huggingface/hub/models--TheBloke--TinyLlama-1.1B-Chat-v1.0-GGUF/blobs/a4c9bb1dbaa372f6381a035fa5c02ef087aaa1ff1f843a56a22328114f03fc59

# Run tests with model path configured
$ LLAMA_MODEL_PATH=$(readlink -f $(ls -S ~/.cache/huggingface/hub/models--TheBloke--TinyLlama-1.1B-Chat-v1.0-GGUF/snapshots/**Q8_0.gguf | head -1)) swift test --traits Llama --filter basicResponse
􁁛  Suite "LlamaLanguageModel" passed after 1.242 seconds.

Can you please try running these steps and let me know if you're able to get this far?

For reference, I'm running macOS 26 on an M4 Mac Studio

$ sw_vers
ProductName:		macOS
ProductVersion:		26.0
BuildVersion:		25A353

$ system_profiler SPHardwareDataType
Hardware:

    Hardware Overview:

      Model Name: Mac Studio
      Model Identifier: Mac16,9
      Model Number: Z1CD0013MLL/A
      Chip: Apple M4 Max
      Total Number of Cores: 14 (10 performance and 4 efficiency)
      Memory: 36 GB
      System Firmware Version: 13822.1.2
      OS Loader Version: 13822.1.2

[mattt]

Update: I determined the cause of the crash to be a problem with how we were initializing greedy sampling. This is fixed by #55. I also opened a few more PRs with additional improvements in #53 and #54.

When you have a chance, please try the reproduction steps I listed in my last post and let me know if that works for you.

[mattt]

@spicymatt Hi again. I merged a bunch of PRs yesterday to get llama working as expected. Can you please update to 0.5.0 and let me know if that works for you now?

[BarredEwe]

@spicymatt Hi again. I merged a bunch of PRs yesterday to get llama working as expected. Can you please update to 0.5.0 and let me know if that works for you now?

The same problem on 0.5.0.

[mattt]

@BarredEwe Can you please share more information about what software you're using, what you tried, and what error you're seeing?

[BarredEwe]

1. Create an empty Swift package
2. Add AnyLanguageModel with traits Llama to the dependencies
3. We are waiting for a successful package resolution
4. Connect this package to the Xcode of the project
5. When building the Xcode project, we will get the error:

/.../SourcePackages/checkouts/AnyLanguageModel/Sources/AnyLanguageModel/Availability.swift Cannot load module 'Llama' as 'llama'
/.../SourcePackages/checkouts/AnyLanguageModel/Sources/AnyLanguageModel/ConvertibleFromGeneratedContent.swift Cannot load module 'Llama' as 'llama'
/...SourcePackages/checkouts/AnyLanguageModel/Sources/AnyLanguageModel/GenerationGuide.swift Cannot load module 'Llama' as 'llama'
/.../SourcePackages/checkouts/AnyLanguageModel/Sources/AnyLanguageModel/GenerationID.swift Cannot load module 'Llama' as 'llama'
/.../SourcePackages/checkouts/AnyLanguageModel/Sources/AnyLanguageModel/Prompt.swift Cannot load module 'Llama' as 'llama'
/.../SourcePackages/checkouts/AnyLanguageModel/Sources/AnyLanguageModel/SendableMetatype.swift Cannot load module 'Llama' as 'llama'

The package resolution logs are successful.
Cleaning DeriveData and other caches did not help.

$ sw_vers
ProductName:		macOS
ProductVersion:		26.1
BuildVersion:			25B78

$ system_profiler SPHardwareDataType
Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: Mac15,11
      Model Number: Z1AH0003LCH/A
      Chip: Apple M3 Max
      Total Number of Cores: 14 (10 performance and 4 efficiency)
      Memory: 36 GB
      System Firmware Version: 13822.41.1
      OS Loader Version: 13822.41.1

$ xcodebuild -version
Xcode 26.1.1
Build version 17B100

[mattt]

Connect this package to the Xcode of the project

Can you tell me more about how you're including this in your project? Xcode doesn't yet support package traits, so you have to use the workaround documented in the README. Are you following those instructions?

(FWIW, when I set up AnyLanguageModel in my Xcode app project, I don't recall it displaying with that 🏛️ framework icon in the explorer / issue navigator, so that makes me think that it's not being built through the blessed Swift Package Manager path)

[BarredEwe]

Are you following those instructions?

I did it according to the instructions in the Readme. Even the MyAppKit has a error:

MyAppKit.zip

[BarredEwe]

The same problem occurs when I just connect the llama.swift project separately.

MyAppKit_Llama.zip

mattt/llama.swift#3