Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
e2f7a04
test(qwen3): committed non-circular golden generator + fixtures
DenisovAV Aug 3, 2026
2dcda55
feat(qwen3): npy/npz reader with fp16 row-lookup
DenisovAV Aug 3, 2026
4937707
feat(qwen3): Qwen2 byte-level BPE encoder
DenisovAV Aug 3, 2026
6cb427c
feat(qwen3): host embedding tables + text projection MLP
DenisovAV Aug 3, 2026
ebb2f1d
fix(qwen3): await Qwen3Tables.load in the two tests that used .then()
DenisovAV Aug 3, 2026
6ae13e3
feat(qwen3): prompt-embedding assembly (x-vector, language)
DenisovAV Aug 3, 2026
5bbf92a
feat(qwen3): sampler + cb0 scoring (suppress, rep-penalty)
DenisovAV Aug 3, 2026
9966b8b
refactor(tts): extract shared LiteRT graph helpers (matcha byte-exact)
DenisovAV Aug 3, 2026
0daf5ac
feat(qwen3): frozen talker layout + load-time shape assert
DenisovAV Aug 3, 2026
af65539
feat(qwen3): Qwen3TtsCore load + talker prefill
DenisovAV Aug 3, 2026
63fc386
feat(qwen3): talker decode step + first-frame cb0 golden
DenisovAV Aug 3, 2026
4e29054
feat(qwen3): MTP 15-codebook inner loop + residual golden
DenisovAV Aug 3, 2026
b8eb051
feat(qwen3): codec chunked decode → 24 kHz PCM
DenisovAV Aug 3, 2026
326ef53
feat(qwen3): end-to-end AR synthesize (fp32-greedy corr≈1.0 gate)
DenisovAV Aug 3, 2026
6e25f2d
feat(qwen3): core TtsModelType.qwen3 + bundle manifest
DenisovAV Aug 3, 2026
8a2891b
test(qwen3): builder-level regression test for the joinUrl/urlSuffixF…
DenisovAV Aug 3, 2026
5c884b3
feat(qwen3): TtsModelProfile.qwen3 + pipeline kind
DenisovAV Aug 3, 2026
317b237
feat(qwen3): worker branch for the AR pipeline
DenisovAV Aug 3, 2026
37de0f7
feat(qwen3): language selection + catalog + picker (one voice, v1)
DenisovAV Aug 3, 2026
8d23332
fix(qwen3): language-blind TTS singleton reuse + case-sensitive 'auto'
DenisovAV Aug 3, 2026
999485d
feat(qwen3): device gate + CHANGELOG
DenisovAV Aug 3, 2026
7d344c4
fix(qwen3): drop unsafe install-time legacy-file adoption + core 1.6.0
DenisovAV Aug 4, 2026
a6ee64a
chore(flutter_gemma): bump 1.5.1 -> 1.5.2
DenisovAV Aug 4, 2026
96990f7
chore(flutter_gemma_speech): bump 0.4.0 -> 0.4.1
DenisovAV Aug 4, 2026
790220a
fix(speech): review fixes A — dep pin, maxFrames warning, load-time g…
DenisovAV Aug 4, 2026
c9c3254
test(qwen3): close codec multi-window coverage gap
DenisovAV Aug 4, 2026
c4a2e19
docs(qwen3): dartdoc publish-hygiene sweep + vendor reference recipe
DenisovAV Aug 4, 2026
81a8f5f
docs(qwen3): sweep 2 residual comment refs (wrapped citation + Tasks …
DenisovAV Aug 4, 2026
8c03f7c
fix(speech): review fixes D — maxFrames fail-loud throw, tensor-buffe…
DenisovAV Aug 4, 2026
7dd17d2
docs: bump site to flutter_gemma 1.5.2 / speech 0.4.1 + document Qwen…
DenisovAV Aug 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ Core has NO pigeon (dropped at the 1.0 cut; its value types are hand-written in
- **MediaPipe Web**: v0.10.27, Android/iOS: v0.10.33
- **LiteRT-LM**: native libs from `native-v0.14.0` GitHub Release. Android tarball bundles the Qualcomm QNN dispatch stack and Windows tarball bundles Intel NPU dispatch (`LiteRtDispatch.dll` + OpenVino runtime + TBB) for `PreferredBackend.npu` (Qualcomm Snapdragon / Intel LunarLake/PantherLake). v0.14.0: native per-session sampler (opaque session-config) + #214 GPU output-garbage fix; Dawn split static→dynamic (Linux/Windows bundle `libwebgpu_dawn`). Windows discrete GPU (WebGPU/Dawn) regressed upstream — use CPU/NPU on Windows (LiteRT-LM #2957).
- **large_file_handler**: `^0.5.0` (core dep; 0.5.0 declares all 6 platforms — needed for pana platform support + the dart2wasm-clean web graph)
- **Current Version**: core `flutter_gemma` `1.5.1`, `flutter_gemma_rag_sqlite` `1.1.0`, `flutter_gemma_rag_qdrant` `1.1.0`; `flutter_gemma_litertlm` `1.3.1`, `flutter_gemma_mediapipe` `1.0.4`, `flutter_gemma_embeddings` `1.0.4`, `flutter_gemma_speech` `0.4.0`; `flutter_gemma_agent` `0.1.0`, `flutter_gemma_builtin_ai` `0.1.0`
- **Current Version**: core `flutter_gemma` `1.5.2`, `flutter_gemma_rag_sqlite` `1.1.0`, `flutter_gemma_rag_qdrant` `1.1.0`; `flutter_gemma_litertlm` `1.3.1`, `flutter_gemma_mediapipe` `1.0.4`, `flutter_gemma_embeddings` `1.0.4`, `flutter_gemma_speech` `0.4.1`; `flutter_gemma_agent` `0.1.0`, `flutter_gemma_builtin_ai` `0.1.0`
- **0.15.2**: embedding unified on LiteRT C API via Dart FFI on all native platforms (Android + iOS + Desktop). Drops `localagents-rag` JVM dep on Android and the separate TFLite C 0.12.7 tarball on Desktop; `TensorFlowLiteC` pod no longer needed on iOS. Single source of truth for `TaskType.prefix` in Dart, fixes cross-platform embedding drift (#264).

## Platform-Specific Setup
Expand Down
3 changes: 3 additions & 0 deletions packages/flutter_gemma/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
## 1.5.2
- Add TtsModelType.qwen3.

## 1.5.1
- fix: namespace companion install files (tokenizers, TTS bundle aux) per model, fixing STT/embedding tokenizer collisions.
- fix: migrate legacy (pre-namespacing) tokenizer installs on restore, so existing STT/embedding models keep working after upgrade.
Expand Down
157 changes: 157 additions & 0 deletions packages/flutter_gemma/example/integration_test/qwen3_tts_test.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
// On-device Qwen3-TTS e2e — installs Qwen3-TTS from HuggingFace through the
// REAL public flow (initialize -> installTts -> getActiveTts -> synthesize),
// exercising the plain-basename manifest -> getModelFilePaths ->
// Qwen3TtsCore.load consistency ON DEVICE (the artifact-gated unit tests
// build their own artifactPaths by directory scan, so they never touch the
// install path itself).
//
// Unlike `tts_matcha_test.dart` (byte-exact against a committed golden,
// fixed CFM seed), this test asserts PLAUSIBILITY only: the runtime uses the
// default int4 talker with `doSample: true`, so exact bytes are not
// reproducible across runs. The byte-exact / corr~=1.0 oracle for the shared
// AR pipeline lives in the fp32-greedy unit test (`qwen3_synthesize_test.dart`,
// artifact-gated, not a device test).
//
// Also asserts language selection actually changes the output end to end:
// synthesizing the same text with `language: 'german'` after closing the
// `'english'` synthesizer (required by the fail-loud language-cache guard —
// see Task 5.4) must NOT produce byte-identical audio to the english pass.
//
// The full 9-file bundle is ~1.9 GB; this test can take tens of minutes on a
// slow connection. Run: cd packages/flutter_gemma/example && \
// flutter test integration_test/qwen3_tts_test.dart -d macos
import 'dart:math' as math;
import 'dart:typed_data';

import 'package:flutter/foundation.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:integration_test/integration_test.dart';
import 'package:flutter_gemma/flutter_gemma.dart'
show FlutterGemma, TtsModelType;
import 'package:flutter_gemma_speech/flutter_gemma_speech.dart'
show LiteRtTtsBackend;

const _modelUrl =
'https://huggingface.co/litert-community/Qwen3-TTS-12Hz-0.6B-Base/resolve/main/';
const _text = 'Hello from on device text to speech.';

double _rms(Uint8List pcm) {
final samples = Int16List.sublistView(pcm);
if (samples.isEmpty) return 0;
var sumSquares = 0.0;
for (final s in samples) {
final n = s / 32768.0;
sumSquares += n * n;
}
return math.sqrt(sumSquares / samples.length);
}

bool _bytesEqual(Uint8List a, Uint8List b) {
if (a.length != b.length) return false;
for (var i = 0; i < a.length; i++) {
if (a[i] != b[i]) return false;
}
return true;
}

void main() {
IntegrationTestWidgetsFlutterBinding.ensureInitialized();

testWidgets(
'Qwen3-TTS installs from HF, synthesizes plausible english audio, and '
'german differs from english',
(_) async {
await FlutterGemma.initialize(ttsBackends: const [LiteRtTtsBackend()]);

await FlutterGemma.installTts()
.fromNetwork(_modelUrl)
.ofType(TtsModelType.qwen3)
.install();

final englishSynth = await FlutterGemma.getActiveTts(language: 'english');
Uint8List englishPcm;
try {
englishPcm = await englishSynth.synthesize(_text);

expect(englishPcm, isNotEmpty, reason: 'english PCM was empty');
expect(englishSynth.sampleRate, 24000);

final durationSeconds = englishPcm.length / 2 / englishSynth.sampleRate;
final rms = _rms(englishPcm);

debugPrint(
'QWEN3-TTS-DEVICE-GATE<<<lang=english pcmBytes=${englishPcm.length} '
'durationSeconds=${durationSeconds.toStringAsFixed(4)} '
'rms=${rms.toStringAsFixed(4)}>>>',
);

// Plausibility oracle (int4 + doSample:true is not byte-reproducible
// across runs/backends): non-silent, non-clipped-garbage, and a
// ~7-word sentence should take at least ~1s of audio.
expect(
durationSeconds,
greaterThanOrEqualTo(1.0),
reason: 'Synthesized audio implausibly short: ${durationSeconds}s',
);
expect(
rms,
inInclusiveRange(0.005, 0.5),
reason: 'RMS out of the plausible speech band: $rms',
);
} finally {
// REQUIRED before requesting a different language on the same
// active model — the language-cache guard (Task 5.4) throws
// StateError otherwise instead of silently reusing the english
// synthesizer for a german request.
await englishSynth.close();
}

final germanSynth = await FlutterGemma.getActiveTts(language: 'german');
try {
final germanPcm = await germanSynth.synthesize(_text);

expect(germanPcm, isNotEmpty, reason: 'german PCM was empty');
expect(germanSynth.sampleRate, 24000);

final durationSeconds = germanPcm.length / 2 / germanSynth.sampleRate;
final rms = _rms(germanPcm);

debugPrint(
'QWEN3-TTS-DEVICE-GATE<<<lang=german pcmBytes=${germanPcm.length} '
'durationSeconds=${durationSeconds.toStringAsFixed(4)} '
'rms=${rms.toStringAsFixed(4)}>>>',
);

expect(
durationSeconds,
greaterThanOrEqualTo(1.0),
reason: 'Synthesized audio implausibly short: ${durationSeconds}s',
);
expect(
rms,
inInclusiveRange(0.005, 0.5),
reason: 'RMS out of the plausible speech band: $rms',
);

// Language selection actually changes the model's output, end to
// end through the public API: german must not equal english either
// in length or in bytes (a byte match at equal length would mean
// the language control token silently had no effect).
final identical =
germanPcm.length == englishPcm.length &&
_bytesEqual(germanPcm, englishPcm);
expect(
identical,
isFalse,
reason:
'german synthesis produced byte-identical audio to english '
'(pcmBytes=${germanPcm.length} vs ${englishPcm.length}) — '
'language selection had no effect',
);
} finally {
await germanSynth.close();
}
},
timeout: const Timeout(Duration(minutes: 90)),
);
}
23 changes: 22 additions & 1 deletion packages/flutter_gemma/example/lib/models/tts_model.dart
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,11 @@ import 'package:flutter_gemma/flutter_gemma.dart' show TtsModelType;
/// Catalog of on-device TTS models. SELECTABLE like STT — each entry carries
/// the [TtsModelType] that tells the generic `LiteRtTtsBackend` which
/// `TtsModelProfile` to run. Install uses one base URL + `.ofType`.
/// Only [matcha] is wired; kokoro/supertonic are follow-ons (isSupported false).
/// [matcha] and [qwen3] are wired; kokoro/supertonic are follow-ons
/// (isSupported false). [qwen3] additionally exposes 11 selectable
/// languages (`tts_screen.dart`'s language dropdown, populated from
/// `flutter_gemma_speech`'s `qwen3SupportedLanguages`) — [matcha] is
/// English-only (its locale comes from its bundle, not a runtime param).
enum TtsModel {
matcha(
baseUrl: 'https://huggingface.co/litert-community/Matcha-TTS/resolve/main/',
Expand All @@ -12,6 +16,17 @@ enum TtsModel {
ttsModelType: TtsModelType.matcha,
isSupported: true,
),
qwen3(
baseUrl:
'https://huggingface.co/litert-community/Qwen3-TTS-12Hz-0.6B-Base/resolve/main/',
displayName: 'Qwen3-TTS 0.6B (11 languages)',
size: '~1.9GB',
ttsModelType: TtsModelType.qwen3,
isSupported: true,
notes:
'CPU only, slow (RTF≈3 — ~3s of compute per 1s of audio), needs a '
'6 GB-RAM-class device',
),
kokoro(
baseUrl: 'https://huggingface.co/litert-community/Kokoro-82M/resolve/main/',
displayName: 'Kokoro 82M',
Expand All @@ -38,6 +53,7 @@ enum TtsModel {
required this.ttsModelType,
this.isSupported = true,
this.unsupportedReason,
this.notes,
});

/// HuggingFace repo base URL each manifest filename is resolved against.
Expand All @@ -59,4 +75,9 @@ enum TtsModel {

/// Why [isSupported] is false; null when supported.
final String? unsupportedReason;

/// UI-only performance/hardware caveat shown under the model info card
/// (e.g. CPU-only, expected RTF, RAM class) — null when there's nothing
/// notable to call out.
final String? notes;
}
Loading
Loading