fix(litertlm): detect stream-callback ABI at runtime (LiteRT-LM v0.15.0) - #410
Open
DenisovAV wants to merge 3 commits into
Open
fix(litertlm): detect stream-callback ABI at runtime (LiteRT-LM v0.15.0)#410DenisovAV wants to merge 3 commits into
DenisovAV wants to merge 3 commits into
Conversation
LiteRT-LM v0.15.0 changed LiteRtLmStreamCallback from void (*)(void*, const char* chunk, bool is_final, const char* error_msg) to void (*)(void*, const LiteRtLmStreamChunk* chunk) with no compatibility overload and no version symbol in the C API. A shim built for the old shape keeps linking and reads arguments 3 and 4 out of whatever the registers hold, so strdup() runs on a garbage pointer: an access violation on Windows, malformed UTF-8 on macOS. Probe for litert_lm_stream_chunk_get_text in the loaded libLiteRtLm and register the matching callback. One binary now works against both ABIs, so the native bump stays revertible. Also decode native strings leniently. Utf8Pointer.toDartString throws on malformed bytes, which on the error path replaces the real native message with a complaint about character encoding — the exact misdirection that made this break hard to trace. Verified on macOS against both native-v0.14.0 (shipped) and v0.15.0.
The ABI probe uses dlsym(RTLD_DEFAULT, ...). Apple's libc declares RTLD_DEFAULT unconditionally, but on glibc it lives behind __USE_GNU, so <dlfcn.h> hides it unless _GNU_SOURCE is defined first. The macOS build compiled clean and Linux failed with "RTLD_DEFAULT undeclared". Define it ahead of every include, Linux-only, so the Apple and Windows paths are untouched.
The stream-callback ABI break, the NPU stacks the build does not produce, the LiteRT pin as a second ABI surface, and the deployment target that silently tracks the build host — each cost real time this migration, and none of them were written down anywhere the next build would look.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LiteRT-LM v0.15.0 changed
LiteRtLmStreamCallbackfromto
with no compatibility overload, and the C API exposes no version symbol to detect it. A shim built for the old shape keeps linking and starts reading arguments 3 and 4 out of whatever the registers happen to hold —
strdup()then runs on a garbage pointer. That surfaced as an ACCESS_VIOLATION inStreamProxy.dllon Windows and aFormatException: Missing extension byteon macOS, neither of which points anywhere near the real cause.What this does
stream_proxy.cnow probes the loadedlibLiteRtLmforlitert_lm_stream_chunk_get_text— a symbol that simply does not exist before v0.15.0 — and registers the matching callback shape. One binary works against both ABIs, which keeps the upcoming native bump revertible instead of one-way.Two details worth flagging for review:
NULL. Dart treats any non-NULL error pointer as a failed stream, so the adapter forwardsNULLunlesserror_msg[0]is set — without that, every ordinary token would abort the stream.dlsym/GetProcAddressrather than linked, so the shim still builds and runs against v0.14.0.litert_lm_client.dartalso decodes native strings leniently now.Utf8Pointer.toDartStringthrows on malformed bytes, so on the error path a native failure was being replaced by a complaint about character encoding, losing the only message that explained it.Verification
Same test, same machine, both native versions:
native-v0.14.0(shipped)All tests passed2117fc43)All tests passedThe v0.14.0 run is the one that matters: it is the currently published bundle, so this proves no regression for existing users.
flutter analyzeclean, 32 package unit tests pass.Not included
This is the adapter only — no native bump, no version changes. The
native-v0.15.0migration follows separately so that a bisect can distinguish "the adapter broke it" from "the bump broke it".