Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ There is an example of using:
- **Local Execution:** Run Gemma and other LLMs (Qwen, DeepSeek, Phi, FastVLM, SmolLM, …) directly on user devices for enhanced privacy and offline functionality.
- **Platform Support:** Compatible with iOS, Android, Web, macOS, Windows, and Linux platforms.
- **🖥️ Desktop Support:** Native desktop apps (macOS, Windows, Linux) with GPU acceleration via LiteRT-LM, called directly from Dart through `dart:ffi` — no JVM/JRE bundling. See [DESKTOP_SUPPORT.md](DESKTOP_SUPPORT.md) for details.
- **🧩 Desktop Runtime Extensions:** Register custom desktop runtimes and fall back to the built-in LiteRT path when an extension declines a model.
- **🍎 Built-in macOS MLX bridge:** When the host app links `flm_dispatch_json` / `flm_bridge_free_string`, flutter_gemma now auto-routes local model directories to MLX without changing the chat/session API.
- **🖼️ Multimodal Support:** Text + Image input with Gemma 4, Gemma3n, and FastVLM vision models
- **🎙️ Audio Input:** Record and send audio messages with Gemma3n E2B/E4B models (Android, iOS device, Desktop)
- **🛠️ Function Calling:** Enable your models to call external functions and integrate with other services (supported by select models)
Expand Down Expand Up @@ -153,6 +155,38 @@ await FlutterGemma.installModel(modelType: ModelType.general)

2. Run `flutter pub get` to install.

## Custom desktop runtimes

On macOS, flutter_gemma now includes a built-in MLX desktop extension. If the
active model resolves to a local directory and the host process already links an
MLX bridge exposing `flm_dispatch_json` / `flm_bridge_free_string`, the desktop
plugin auto-selects MLX before falling back to LiteRT-LM.

If you already have a different native desktop runtime in another package, you
can still replace or extend that behavior without forking the main chat/session
APIs:

```dart
FlutterGemmaDesktop.registerRuntimeExtension(
DesktopRuntimeExtension(
name: 'mlx',
createInferenceModel: (request) async {
if (!request.modelPath.endsWith('.mlx') &&
!request.modelPath.contains('mlx')) {
return null;
}

// Return your own InferenceModel implementation here.
return MyMlxInferenceModel.fromRequest(request);
},
),
);
```

The first extension that returns a non-null model handles the request. If every
extension returns `null`, flutter_gemma continues with its built-in LiteRT
desktop runtime exactly as before.

## Platform & Architecture Support

The plugin ships native prebuilts only for the architectures below. Other ABIs fail at native load with a typed error.
Expand Down
21 changes: 19 additions & 2 deletions lib/core/infrastructure/platform_file_system_service.dart
Original file line number Diff line number Diff line change
Expand Up @@ -51,15 +51,22 @@ class PlatformFileSystemService implements FileSystemService {
@override
Future<bool> fileExists(String filePath) async {
final file = File(filePath);
return await file.exists();
if (await file.exists()) {
return true;
}
return Directory(filePath).exists();
}

@override
Future<int> getFileSize(String filePath) async {
final file = File(filePath);

if (!await file.exists()) {
return 0;
final directory = Directory(filePath);
if (!await directory.exists()) {
return 0;
}
return _directorySize(directory);
}

final stat = await file.stat();
Expand Down Expand Up @@ -175,6 +182,16 @@ class PlatformFileSystemService implements FileSystemService {
// The actual tracking is done in ProtectedFilesRegistry.registerExternalPath()
}

Future<int> _directorySize(Directory directory) async {
int total = 0;
await for (final entity in directory.list(recursive: true)) {
if (entity is File) {
total += await entity.length();
}
}
return total;
}

/// Gets the model storage directory with caching.
///
/// Mobile (Android, iOS): app's Documents — sandboxed, never cloud-synced.
Expand Down
31 changes: 31 additions & 0 deletions lib/core/model_management/utils/file_system_manager.dart
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,15 @@ class ModelFileSystemManager {
int minSizeBytes = _defaultMinSizeBytes,
}) async {
try {
final directory = Directory(filePath);
if (await directory.exists()) {
final isValidDirectory = await _isModelDirectoryValid(directory);
if (!isValidDirectory) {
debugPrint('Model directory validation failed: $filePath');
}
return isValidDirectory;
}

final file = File(filePath);

if (!await file.exists()) {
Expand All @@ -83,6 +92,28 @@ class ModelFileSystemManager {
}
}

static Future<bool> _isModelDirectoryValid(Directory directory) async {
final entries = await directory.list(followLinks: false).toList();
if (entries.isEmpty) {
return false;
}

for (final entry in entries) {
final name = path.basename(entry.path).toLowerCase();
if (entry is File &&
(name == 'config.json' ||
name == 'tokenizer.json' ||
name == 'tokenizer_config.json' ||
name.endsWith('.safetensors') ||
name.endsWith('.gguf') ||
name.endsWith('.bin'))) {
return true;
}
}

return entries.any((entry) => entry is File);
}

/// Gets the full file path for a model file.
///
/// Delegates to [FileSystemService.getTargetPath] so that the correct
Expand Down
Loading