Common issues and fixes. For desktop-specific problems (Linux native logs, glibc, Windows DXC, stale GPU shader cache) see Desktop Support → Troubleshooting.
Downloads#
-
Resume isn't supported by the HuggingFace CDN. flutter_edge_ai uses smart retry with exponential backoff and
automatic restart of interrupted downloads instead. Tune the attempt count via
maxDownloadRetriesinFlutterEdgeAi.initialize(...)(default: 10). -
Large downloads on Android need
foreground: trueto get a foreground service (which shows a notification) and bypass Android's 9-minute background execution limit. The default (null) does not configure a notification, and without one the platform never starts the service — so the size threshold alone does nothing. iOS uses native URLSession and needs no special handling. See Models → downloads. -
Using
background_downloaderin your own app too? flutter_edge_ai no longer listens toFileDownloader().updates(fixed influtter_gemma1.6.3). That stream takes a single subscription, so claiming it made every laterFileDownloader().updates.listen(...)in the host app throw "Stream has already been listened to". Updates are now scoped to flutter_edge_ai's own task group and the stream stays yours. - Custom servers on Web must enable CORS headers. HuggingFace is already configured correctly; for Firebase Storage see the CORS configuration docs.
Gated models / download errors (401, 403)#
installModel(...).install() throws a public DownloadException carrying a sealed
DownloadError, so you can react to gated HuggingFace models (HTTP 401/403) by type
instead of substring-matching error strings:
try {
await FlutterEdgeAi.installModel(modelType: ModelType.gemmaIt)
.fromNetwork(url, token: hfToken)
.install();
} on DownloadException catch (e) {
switch (e.error) {
case UnauthorizedError(): // 401 — missing/invalid HuggingFace token
case ForbiddenError(): // 403 — token lacks access to this gated model
showGatedModelDialog();
case NotFoundError(): // 404 — bad URL (use /resolve/main/, not /blob/main/)
showNotFoundDialog();
case RateLimitedError(): // 429
case ServerError(): // 5xx
case NetworkError(): // connectivity
case CanceledError(): // user canceled
case UnknownError():
showRetryDialog(e.error.toUserMessage());
}
}
For a gated model (401/403): pass a valid huggingFaceToken to FlutterEdgeAi.initialize(...)
(or token: on fromNetwork(...)), open the model page on HuggingFace, accept its license, and request access. Each
DownloadError also exposes toUserMessage(), toTitle(), isRetryable, and
requiresUserAction for building UI.
Auth errors (401/403/404) fail fast after one attempt — they are not retried. Only NetworkError, ServerError, and RateLimitedError are retryable (see isRetryable).
Memory#
-
iOS: ensure
Runner.entitlementscontains the memory entitlements and the deployment target is at least 15.0 (16.0 withflutter_edge_ai_mediapipe, 16.4 withflutter_edge_ai_llama) — in Xcode on the Runner target under SPM, or in thePodfilewhen one exists. See Installation → iOS. -
iOS, MediaPipe
.taskon GPU: the app is terminated while the model loads, with the entitlements present and no crash report. Run.taskmodels onPreferredBackend.cpuon iOS, or use the.litertlmbuild. See MediaPipe → Backends. -
Reduce
maxTokensif you hit memory pressure — but keep it at 1024 or higher for.litertlmmodels (see "maxTokens vs maxOutputTokens" below). To shorten replies, usemaxOutputTokens, not a smallermaxTokens. - Use smaller models (1B-2B parameters) for devices with <6GB RAM. Gemma 4 E2B needs 6 GB or more (a 4 GB phone is killed while it loads); the larger Gemma 4 E4B and Gemma 3n files have no measured floor.
- Close sessions and models when not needed; monitor usage with
sizeInTokens(). -
To see what a model actually costs, measure the memory the OS cannot reclaim before and after loading it with
flutter_edge_ai_diagnostics(Android + iOS + macOS + Linux + Windows). The RSS a profiler shows also counts mmapped weights the OS can drop, so it does not show that cost.
maxTokens vs maxOutputTokens#
maxTokens (on getActiveModel/createModel) is the context window
— the total budget shared by the input (system prompt + history + your message) and
the generated output (the KV-cache size). It is not the reply length.
.litertlm models require a context window of at least 1024. Passing a smaller
maxTokens (e.g. 100) used to crash with DYNAMIC_UPDATE_SLICE failed to prepare
/ Failed to allocate tensors (even on CPU — the failure is at graph compile, before the backend runs). As of
flutter_gemma_litertlm 1.0.2 a too-small maxTokens is clamped up to 1024 automatically with a log warning — except on
PreferredBackend.npu, where the context is baked into the compiled bundle (see NPU).
To limit how many tokens the model generates, use maxOutputTokens on openSession
or openChat instead:
final model = await FlutterEdgeAi.getActiveModel(maxTokens: 1024); // context window
final chat = await model.openChat(maxOutputTokens: 100); // reply cap
(maxOutputTokens is honored on .litertlm, ONNX, .gguf and built-in AI on Android, iOS and macOS; the MediaPipe
.task path has no session-level output cap and ignores it, and so does built-in AI on Windows and the Web.)
iOS#
-
Build issues: ensure the minimum iOS version is at least 15.0 (16.0 with
flutter_edge_ai_mediapipe, 16.4 withflutter_edge_ai_llama). If the app has aPodfile(any app usingflutter_edge_ai_mediapipedoes — it ships noPackage.swift), also use static linking (use_frameworks! :linkage => :static) and reinstall withcd ios && pod install --repo-update. An SPM-only app has noPodfile; there the deployment target lives on the Runner target in Xcode. - Simulator GPU disabled: iOS Simulator's Metal has a 256 MB single-allocation cap that LLM weight tensors exceed (e.g. Gemma 3 1B's KV cache alone is 288 MB). Use CPU on the simulator, or test GPU on a physical iPhone. This is a simulator limit, not a plugin bug.
macOS#
-
Library not loaded: @rpath/GemmaModelConstraintProvider.framework/…(or@rpath/libGemmaModelConstraintProvider.dylibbeforeflutter_edge_ai_litertlm1.19.0; inside an error that saysLiteRT-LM did not load on macOSand names this cause): LiteRT-LM's companion libraries are not in the app. Every LiteRT feature fails to load (LLM, embeddings, speech), on every backend, and the build itself succeeds. With Swift Package Manager on,flutter_edge_ai_litertlm1.19.0 and later bring them through its Swift package: upgrade, then runflutter cleanand build again. With it off, replace thepost_installblock ofmacos/Podfilewith ours, see Desktop → macOS; if the block is already there, runflutter cleanand build again. -
The package product 'flutter-edge-ai-litertlm-product' requires minimum platform version 14.0 for the macOS platform, or CocoaPods'required a higher minimum deployment targetnamingflutter_edge_ai_litertlm: raise the Runner target's macOS deployment target to 14.0 in Xcode, andplatform :osx, '14.0'inmacos/Podfileif the app has one. LiteRT-LM's companion libraries are built for macOS 14. -
Flutter 3.44 with Xcode 27:
The macOS deployment target 'MACOSX_DEPLOYMENT_TARGET' is set to 10.15, but the range of supported deployment target versions is 12.0 to …. Flutter 3.44's template targets macOS 10.15, which Xcode 27 refuses, for any app. Raise the Runner target to 12.0 in Xcode andplatform :osx, '12.0'inmacos/Podfile; with a Podfile, also addtarget.build_configurations.each { |c| c.build_settings['MACOSX_DEPLOYMENT_TARGET'] = '12.0' }inside theinstaller.pods_project.targets.eachloop ofpost_install, because 3.44 raises pods only to 10.15. Flutter 3.47's template needs neither (measured on Xcode 27.1, 2026-10-21). Withflutter_edge_ai_litertlmuse 14.0 in all three places instead, because its companion libraries are built for macOS 14 (Desktop → macOS). -
incompatible architecture (have 'arm64', need 'x86_64'): LiteRT-LM ships for Apple silicon only. It does not run on an Intel Mac or in an x86_64 (Rosetta) process. -
library load disallowed by system policy: macOS refused a quarantined or un-notarized library. Notarize the app; for a local test build, remove the quarantine attribute withxattr -dr com.apple.quarantineon the app. -
have different Team IDs: macOS library validation refused a library. Sign the app and its frameworks with the same team, or add thecom.apple.security.cs.disable-library-validationentitlement.
Android#
-
.litertlmmodels require minSdk 30.libLiteRtLm.sodepends on API 30+ Bionic syscalls (pthread_cond_clockwait,sem_clockwait) that can't be shimmed on older devices. MediaPipe.taskmodels work on lower API levels. -
.litertlm/ embeddings / vision arearm64-v8aonly. MediaPipe text inference (.task/.bin) also runs onx86_64andarmeabi-v7a. If you only use arm64-only features, addndk { abiFilters 'arm64-v8a' }(inbuild.gradle.kts:ndk { abiFilters += listOf("arm64-v8a") }) so the Play Store doesn't offer broken APKs. See Installation → Android architecture. -
GPU: nothing to add — the OpenCL
<uses-native-library>entries come from the core plugin's own manifest (flutter_gemma1.2.0+) through the manifest merger. If the GPU backend still falls back, check that the merged manifest containslibvndksupport.soandlibOpenCL.so. See Installation → Android. -
Which GPU flush ran (
kernel_batch_size). On an Adreno GPU (/dev/kgsl-3d0exists) the GPU load asks LiteRT-LM to flush every 2 kernels, so the UI keeps drawing during a long prefill (#364); on every other Android GPU it sets nothing and the engine's default applies, which is faster on Mali (#763). Debug builds log the choice ([LiteRtLmFfi] Android GPU (Adreno): hint_kernel_batch_size=2). In a release build, create the engine withLiteRtLmEngine(nativeLogLevel: LiteRtLmLogLevel.verbose)and look in logcat for the engine'skernel_batch_sizeline at load:-1is the engine default (no flush), as reported in #763. -
Gemma 4 on an Adreno GPU fails at the first token:
Stream error: INTERNAL … Failed to invoke the compiled model, withInvalid work group sizein logcat. The model loads, then the first decode fails. LiteRT's GPU kernels ask for more work items than some Adreno GPUs allow with the current native build: an Adreno 610 at the default half precision (a Galaxy A05s atmaxTokens4096; LiteRT #12075), and an Adreno 830 atActivationDataType.float32(a Galaxy S25 Ultra). Load the model on the CPU (preferredBackend: PreferredBackend.cpu); a future native build is expected to fix it. Sinceflutter_edge_ai_litertlm1.21.0 the error message says so. -
Google Play rejects the release: "Your app does not support 16 KB memory page sizes".
Fixed in
flutter_gemma_litertlm1.8.0; everyflutter_edge_ai_litertlmrelease includes the fix. Nothing fails at build or run time — the rejection happens at submission. The Qualcomm Hexagon DSP blobs of the NPU path (libQnnHtpV{73,75,79,81}Skel.so) come from Qualcomm with a 4 KBp_align, and Play scanslib/**/*.sowithout caring that a Hexagon image is loaded by the DSP rather than mapped by the kernel. Since 1.10.0 they ship only in apps that setqualcomm_npu: true, and the build hook raises them to 16 KB. Check your own build with Google’scheck_elf_alignment.shagainst the APK. See #529. -
GPU backend crashes at
engine_createon Mali GPUs (SIGSEGV,pc 0inlibLiteRtOpenClAccelerator.so). Fixed influtter_gemma_litertlm1.8.2; everyflutter_edge_ai_litertlmrelease includes the fix. In 1.7.0–1.8.1 the OpenCL and GPU accelerators calledAHardwareBuffer_allocatewithout declaringlibandroid.so, so Android bound the call to address 0; only Mali GPUs (Samsung A-series, MediaTek, Google Tensor) take that path, so CPU and Adreno were unaffected. See #545. -
Zero chunks and
Stream error: <U+FFFD>, thenSIGABRT. Fixed influtter_gemma_litertlm1.5.2; everyflutter_edge_ai_litertlmrelease includes the fix. On Android the firstdlopenoflibLiteRtLmdecides for the whole process whether its symbols are reachable from the default search scope, and bionic never promotes it afterwards — so an app that embedded or transcribed anything before its first generation left the stream-callback ABI probe blind and the wrong callback shape was registered. If your own or third-party code loadslibLiteRtLmfirst, load it withRTLD_GLOBAL— the package cannot repair that case, but it raises aStateErrornaming it rather than generating corrupt text. See #447.
Web#
-
MediaPipe is GPU-only on web. The web engine ignores
preferredBackendand always runs on the browser's GPU (WebGPU). ONNX on web is the exception:PreferredBackend.cpupins WASM there. -
Mobile
.taskmodels often don't work on web — use the-web.task(MediaPipe) or-web.litertlm(LiteRT-LM) export; only Gemma 4 and Gemma 3n publish the latter today. A native.litertlminstalls on web and then fails at the first generation. - Memory / cache limits:
| Browser | Max Model Size | Notes |
|---|---|---|
| Chrome/Firefox | ~2 GB | ArrayBuffer limit |
| Safari | — | ⚠️ Not suitable |
-
Large models (>2GB): use
WebStorageMode.streaming(OPFS) to bypass the ~2 GB blob limit. Check support withawait FlutterEdgeAi.isStreamingSupported(). See Installation → web storage. -
Storage modes:
cacheApi(default, persists across restarts, <2GB),streaming(OPFS, large models, requires Chrome 86+/Edge 86+/Safari 15.2+),none(ephemeral, testing only).
LiteRT-LM on web (early preview)#
Web .litertlm inference runs .litertlm models in the browser via the
upstream @litert-lm/core
package (WebGPU + WASM). It is an early preview and a subset of the native
path. MediaPipe .task on web is not subject to these limits, though MediaPipe
LLM Inference is maintenance-only — see MediaPipe.
Works on web .litertlm: text generation (sync + streaming), multi-turn chat
with history, system instruction, function calling / tool calls (Gemma 4),
thinking mode (Gemma 4 measured in Chrome; core parses <think>
tags from
any model that emits them), concurrent sessions (serialized), large models via
OPFS streaming.
Not supported on web .litertlm yet:
-
❌ Vision / image input (native only) — an image message throws
UnsupportedError; the JS API creates the LLM engine with no vision executor. -
❌ Audio input (native only) — an audio message throws
UnsupportedError; no audio executor either. -
❌ LoRA weights —
loraPaththrowsUnsupportedError, as it does on native.litertlm; LoRA works only with MediaPipe.taskmodels.
For vision on Web today, use a compatible MediaPipe .task build. Neither Web
engine supports audio. These Web .litertlm limits track the upstream
@litert-lm/core early-preview API and will lift as Google extends the JS
executor surface.
Windows desktop GPU crashes#
Historical. Windows discrete GPUs crashed on PreferredBackend.gpu in
flutter_gemma_litertlm 1.2.0–1.3.1, fixed in 1.4.0 — before the rename, so
every flutter_edge_ai_litertlm release has the fix. macOS/Linux GPU and
Windows CPU/NPU were never affected. See Desktop → Known
limitations.
Wrong numbers on GPU#
Gemma 4 copies digits wrongly out of a long prompt on some GPUs, the same
way on every run: asked when a delivery arrived (2026/06/23), it answers
20226/12/17. It shows from about 2,000 prompt tokens, on Metal and on Adreno.
The published Gemma 4 .litertlm files ask for half-precision activations,
and at half precision the rotary positions past about 2,048 tokens stop being
distinct (LiteRT-LM #4001).
Since flutter_edge_ai_litertlm 1.20.1 a model installed as ModelType.gemma4
and loaded with maxTokens above 2,048 and no activationDataType
gets full
precision on its GPU attempt
by itself, except on Android (litertlm 1.21.0 keeps half precision there and
prints a WARNING). With an older version, or a smaller maxTokens, ask for
it, but not on an Adreno 8xx GPU (a Galaxy S25, for one): with the current
native build float32 fails there at the first token, at any maxTokens:
final model = await FlutterEdgeAi.getActiveModel(
preferredBackend: PreferredBackend.gpu,
activationDataType: ActivationDataType.float32,
);
Prefill gets slower (about 3× on a Snapdragon 8 Elite and an iPhone 11, under 1.5× on an Apple M3 Max); decode speed barely changes. Upstream: LiteRT-LM#3012 (Adreno), LiteRT-LM#2814 (Metal).
Not on web. The web engine ignores activationDataType — it has no such
setting — and so do MediaPipe, ONNX and built-in AI. On the engines that read
it, it reaches the text decoder only: the vision and audio encoders keep the
type the model file asks for.
Every flutter_edge_ai_litertlm release applies it (it arrived in
flutter_gemma_litertlm 1.8.3; older versions accepted the argument and
ignored it).
On Android the GPU shares system memory, so on a 4–6 GB phone running out of it
at float32 can end the app rather than fall back to CPU. Both precisions share
one compiled GPU program cache per model, so switching recompiles the GPU
programs (about 600 MB for Gemma 4 E2B): pick one precision per install rather
than per request.
float32 activations also need more GPU memory than float16. If the GPU
engine cannot be created the model falls back to CPU without an exception,
only a WARNING line in the log — the digits are then right and the model is far
slower, which is easy to mistake for the fix working. Register
LiteRtLmEngine(backendFallback: false) to get a BackendInitException
with
LiteRT-LM's reason instead, or read the backend the model actually got:
final model = await FlutterEdgeAi.getActiveModel(
preferredBackend: PreferredBackend.gpu,
activationDataType: ActivationDataType.float32,
);
if (model.activeBackend != PreferredBackend.gpu) {
// CPU fallback: right digits, but not the run you asked for.
}
Windows embeddings and speech fail with status 3#
Fixed in litertlm 1.7.0. On Windows, embeddings and on-device speech
(STT/TTS) fail with LiteRT call failed: CreateTensorBufferFromHostMemory(...) (status=3) in litertlm 1.4.0–1.6.4. Fixed in flutter_gemma_litertlm 1.7.0 and
flutter_gemma_speech 0.5.1, so every flutter_edge_ai_* release has it. Text generation and the other platforms
were never affected.
Qwen3 0.6B fails with "Failed to create engine"#
Qwen3-0.6B.litertlm from
litert-community/Qwen3-0.6B
fails at engine creation (Failed to create engine on <backend>: <LiteRT-LM's reason>. Model: …; before
flutter_edge_ai_litertlm 1.14.0,
Failed to create engine. Model may be invalid) whatever the maxTokens
and
backend. Since LiteRT-LM v0.17.0
(flutter_gemma_litertlm 1.7.0) the runtime rejects a piece in that bundle's
SentencePiece tokenizer (piece must not include null character — on Android
only in logcat); upstream
LiteRT-LM#3865. On
2026-10-13 at 15:53 UTC the repo replaced the file's tokenizer
(e6687177), and the file downloaded since loads (measured on macOS with
flutter_edge_ai_litertlm 1.15.0). A copy downloaded before that keeps the
old tokenizer and the same name, and the installer does not fetch a file it
already has: uninstall the model and install it again. Or use
Qwen3-0.6B_dynamic_wi4b32_afp32.litertlm from the same repo, installed the
same way (ModelType.qwen3, ModelFileType.litertlm). See
#659.
NPU#
The model answers, fluently, about the beginning of a long prompt and ignores
the rest. That is most Gemma 3 bundles on an NPU. Nothing raises, nothing is
logged: every prefill chunk after the first is dropped, so the model genuinely
never saw the end of your prompt. On Qualcomm the published 270M bundles for
SM8750 and SM8850 and the 1B …_sm8750 and …_sm8550 do it; the 1B
…_sm8850 does not. In a bundle you build with npu_export
it is a size
threshold in the prefill mask (~1 MiB; at prefill 128 the largest working
cache_length for Gemma 3 — 4 attention heads per KV head — is 896), and a
build calibrated only on short prompts loses a longer prompt past ~450 tokens
(Gemma 3 1B,
litert-torch#1344).
On Intel it happens regardless of size, through a different defect. Run a
Gemma 4 bundle on the NPU (none is published for MediaTek: whether its Gemma 3 bundles keep
their chunks is not measured), or the same model's generic bundle on
PreferredBackend.cpu / .gpu — a bundle compiled for an NPU is refused there
with a LiteRtLmBundleException. Upstream:
LiteRT-LM#3508.
Note that maxTokens is not clamped up to 1024 on the NPU attempt the way
it is on CPU and GPU — the context is baked into the compiled bundle (4096 for
the Gemma 4 E2B NPU builds, 1280 for Gemma 3 1B and FastVLM, 4096 for Gemma 3
270M; the catalog's requiredMaxTokens), so keep maxTokens
at or below it. A generic bundle is never tried on
the NPU (it has no NPU graph: LiteRT-LM would run it on the CPU and report
npu), and a bundle compiled for an NPU never runs on the GPU or CPU, so the
floor applies only to the GPU and CPU attempts of a generic bundle.
PreferredBackend.npu on Android falls back to GPU, or an NPU bundle fails
to load, after an upgrade. From flutter_edge_ai_litertlm 1.10.0 the
Qualcomm QNN runtime is not part of the package; the app opts in with
qualcomm_npu: true under hooks: user_defines: litertlm_dart:
in
its pubspec (flutter_edge_ai_litertlm: before flutter_edge_ai_litertlm
1.21.0) — see LiteRT-LM. Without it a bundle compiled for
an NPU is refused with a BackendInitException whose
NpuDispatchUnavailableException names it. A generic bundle never runs on the
NPU, with or without the flag: it has no NPU graph, so npu falls back to GPU,
then CPU, for it, and the log says so. If it says the stack is not enabled although the flag is
set, the flag is in the wrong pubspec: in a pub workspace only the workspace
root's user_defines count. If the build fails fetching qnn-runtime, the
machine cannot reach Maven Central; set qualcomm_npu_maven_url to a mirror or
qualcomm_npu_aar to a local copy of the same AAR.
PreferredBackend.npu on a Qualcomm Linux board falls back to GPU, or an NPU
bundle fails to load. The same qualcomm_npu: true covers Linux arm64
builds. The BackendInitException for a bundle compiled for an NPU names
which of these is missing:
- The build did not bundle the stack — the flag is not set, or not in the workspace root's pubspec.
/dev/fastrpc-cdspdoes not exist — the board exposes no compute DSP.-
It, or
/dev/dma_heap/system, is not accessible — add the user to groupfastrpc(sudo usermod -aG fastrpc $USER) and log in again. -
libcdsprpc.so.1does not open — install Qualcomm's FastRPC (qcom-fastrpc1on Ubuntu).
If the build fails reading the QAIRT SDK, the machine cannot reach
softwarecenter.qualcomm.com; download v2.50.0.260828.zip
elsewhere and set
qualcomm_npu_qairt_zip to its path. Do not set DSP_LIBRARY_PATH: FastRPC
prefers it over ADSP_LIBRARY_PATH, which is where the bundled Skels are
announced.
A Google Tensor bundle fails to load. The BackendInitException names the
cause: the app does not set google_tensor_npu: true (in a pub workspace, at the
root); the device runs Android below 16 (API 36) or a BP2A build; it is not a
Tensor SoC; or the Pixel's libedgetpu_litert.so did not open. A G5 bundle on a
G6 Pixel, or the reverse, is refused by LiteRT when the engine is created. It
never falls back to the GPU or the CPU.
A MediaTek bundle fails to load. The BackendInitException names the cause:
the app does not set mediatek_npu: true (in a pub workspace, at the root);
the device runs Android below 12 (API 31) or its SoC manufacturer is not
MediaTek; or none of the phone's NeuroPilot adapters (libneuron_adapter_mgvi.so,
libneuronusdk_adapter*.mtk.so, …) opened in the app, which means the phone's
vendor does not expose NeuroPilot to apps. A bundle compiled for another SoC is
refused when the engine is created (measured on an MT6989 with MT6991 and MT6993
bundles: NeuroPilot reports an unsupported target or a Neuron SDK version
mismatch). It never falls back to the GPU or the CPU.
A build without network access fails while fetching the dispatch AAR: download
it and set mediatek_npu_aar: <path>.
PreferredBackend.npu on Windows fails at load or the process ends with
0xC0000409. Every release from flutter_gemma_litertlm
1.7.0
(native-v0.17.0) through flutter_edge_ai_litertlm 1.14.0 has this bug in its
Intel NPU load (LiteRT-LM#4023); upgrade to 1.15.0.
PreferredBackend.npu is unavailable on a recent Snapdragon. SoC coverage is
the runtime's, not ours: Snapdragon 8 Gen 5 (SM8845 — OnePlus 15R, iQOO 15R
and the like) is not covered upstream, and a context compiled for SM8850 is
rejected by an SM8845 device even though both are Hexagon v81. Tracked at
LiteRT#7516; note the
easily-confused naming — 8s Gen 4 is SM8735, not SM8845.
Desktop storage locations#
Desktop builds store downloaded models outside the user's Documents/ folder
to avoid OneDrive / iCloud / Domain-Roaming sync corrupting FFI mmap of large
.litertlm files:
- Windows:
%LOCALAPPDATA%\flutter_gemma\(never OneDrive-synced) - macOS:
~/Library/Application Support/<bundle>/flutter_gemma/ - Linux:
~/.local/share/<app>/flutter_gemma/
The directory keeps its flutter_gemma name across the rename, so installed models stay found.
Models installed by older 0.14.x / 0.15.0 builds that still live under
Documents/ keep working via a fallback read.
On Windows, flutter_gemma before 1.4.0 could write a fresh download to a
$CWD-relative path (<cwd>\Users\…\AppData\Local\flutter_gemma\) instead of the
absolute %LOCALAPPDATA%\flutter_gemma\, because %LOCALAPPDATA%
is not one of
background_downloader's base directories. The model then reported "installed"
but failed to load with "model file paths not found". Fixed in 1.4.0
— it
affected every fresh inference / embedding / STT download on Windows, so upgrade
if you hit it.
Multimodal#
- Ensure the model has a vision (or audio) encoder — see Multimodal and Models.
-
Set
supportImage: true(andsupportAudio: truefor audio) when creating the model. On MediaPipe (Android and iOS), passsupportImage: truetoopenChatas well, or the image is dropped. - Check device memory — multimodal models require more RAM.
-
Image input crashes at model load on a GPU text backend (older releases).
On Metal (iOS/macOS) and WebGPU (Windows/Linux) the vision encoder's ops can't
be prepared by the GPU delegate. Fixed in
flutter_gemma_litertlm1.4.2 / core 1.5.9 — a vision encoder its bundle says nothing about runs on CPU while the text decoder keeps the GPU. Upgrade if you hit it. -
Use the GPU backend for faster text decoding. An unset encoder backend
follows the bundle: the section's
backend_constraint, else the NPU it is compiled for, else CPU. SetpreferredVisionBackend/preferredAudioBackendonly where the section allows it ((await readLiteRtLmBundleInfo(path)).audioEncoder?.backendConstraint, orvisionEncoder); the published Gemma 4 and Gemma 3n audio encoders declare cpu, and a backend the section excludes is refused before the model loads. See Multimodal.
Native libraries fetched at build time#
Some packages download their native library from a GitHub Release when you first
build for a platform, then cache it. LiteRT-LM (litertlm_dart, under
flutter_edge_ai_litertlm) caches each release in its own directory under
~/.cache/litertlm_dart/native/ (~/Library/Caches/… on macOS,
%LOCALAPPDATA%\… on Windows); flutter_edge_ai_onnx and
flutter_edge_ai_sqlite cache under ~/.cache/flutter_gemma/native/. This
applies to flutter_edge_ai_litertlm (always has), flutter_edge_ai_onnx, and
flutter_edge_ai_sqlite (every release; as flutter_gemma_rag_sqlite
from
1.3.0 — before that it shipped the loadables inside the package).
-
The build fails with a download error or an HTTP status. The first build of
each platform needs
github.comreachable; an Android build withflutter_edge_ai_onnxalso needsrepo1.maven.org, and thequalcomm_npuopt-in needs Maven Central (Android) orsoftwarecenter.qualcomm.com(Linux arm64). In an air-gapped or proxied CI, copy the cache directories (litertlm_dart/nativeandflutter_gemma/native) from a machine that built the same package versions; forflutter_gemma/native, include its hidden version-marker files — a folder copied without them is discarded and fetched again. Only the Qualcomm NPU stack has settings that point the build at a mirror or a copy you vendor yourself (qualcomm_npu_maven_url,qualcomm_npu_aar,qualcomm_npu_qairt_zip; see NPU). -
The build fails with
CHECKSUM MISMATCH. The bytes served do not match what the package version was pinned to. Re-run once to rule out a corrupt transfer. If it persists, the release asset was replaced after publication — do not work around it by clearing the checksum; report it. -
A build that used to succeed now fails instead of quietly skipping. That is
deliberate. These hooks used to report success while bundling nothing, which
surfaced later as an opaque
dlopencrash on a user's device. A platform the package claims to support now fails the build when its library cannot be produced. -
Maintainers only: a local
native/<name>/prebuilt/<target>/overrides the pinned release.flutter_edge_ai_sqlitesays so on stderr when it takes that path;litertlm_dart(underflutter_edge_ai_litertlm) andflutter_edge_ai_onnxtake it silently. If a new release "did not take", look for that directory first.
Another package bundles its own LiteRT-LM#
The LiteRT-LM library at … is not the one litertlm_dart ships (it lacks litert_lm_engine_settings_set_kernel_batch_size)
— a StateError (an
EmbeddingLoadException for an embedder) when a .litertlm
model, an embedder
or speech loads. Another package in the app ships its own LiteRT-LM under the
same library name — llamadart before 0.12.0 does by default (from 0.12.0 its
build hook leaves it out next to flutter_edge_ai_litertlm), and
llamadart_litert_lm_flutter links one through Swift Package Manager. On iOS
and macOS that app still builds: depending on the order of the app's
dependencies, Flutter renames our framework to LiteRtLm1.framework, and
LiteRtLm.framework is the other package's build, which
flutter_edge_ai_litertlm refuses rather than binds. On Android, Linux and
Windows the build fails instead.
For llamadart, upgrade it to 0.12.0 or later, or leave its LiteRT-LM runtime
out in the app's pubspec.yaml (the workspace root's, if the app is a workspace
member) and remove llamadart_litert_lm_flutter if the app depends on it; then
run flutter clean once:
hooks:
user_defines:
llamadart:
llamadart_native_runtimes: [llama_cpp]
Embeddings#
-
EmbeddingLoadExceptionnamingweb/litert_embeddings.json web, at the firstgetActiveEmbedder()after upgrading toflutter_edge_ai_litertlm1.13.0. The LiteRT.js bundle changed its API, and your app'sweb/still holds the copy of an earlier version. Copy all four files again from the package'sweb/(litert_embeddings.js,sentencepiece.js,litert.js,tensorflow.js). See Installation → Web. -
EmbeddingLoadException: window.EmbeddingEngine is not seton web, when an EmbeddingGemma 2 (.litertlm) embedder loads. Sinceflutter_edge_ai_litertlm1.13.1 it runs on@litert-lm/core'sEmbeddingEngine, which yourweb/index.htmldoes not expose yet. Addwindow.EmbeddingEngine = m.EmbeddingEngine;next towindow.Engine = m.Engine;in the@litert-lm/coremodule script. See Installation → Web. -
EmbeddingLoadException: EmbeddingGemma 2 on the web needs WebGPUon web, when an EmbeddingGemma 2 (.litertlm) embedder loads in a browser without WebGPU — nonavigator.gpu, or no GPU adapter (WebGPU switched off, a blocklisted driver, a headless browser without a GPU). On the web EmbeddingGemma 2 runs on WebGPU only: the CPU (WebAssembly) path of@litert-lm/core0.18.0 fails on these bundles with "Failed to get external weight", so there is nothing to fall back to. Use a browser with WebGPU enabled; checknavigator.gpuandawait navigator.gpu.requestAdapter()in its console. See Embeddings & RAG. -
EmbeddingInputTooLongExceptionfromembed,embedAllorRagIndex.add/search. The input is longer than the model'smaxInputTokens, task prefix, BOS and EOS included; since 2.3.0 it is refused instead of silently truncated. Split the text into smaller chunks — at most 512 tokens for EmbeddingGemma 2, whose limit is 1024. See Embeddings & RAG. -
EmbeddingLoadException…No embedding tokenizer is configuredfromgetActiveEmbedder(), for a.tfliteor ONNX embedder (an EmbeddingGemma 2.litertlmcarries its own and needs none). Sinceflutter_gemma1.9.0 an embedding backend no longer carries a tokenizer: which family a model needs (Gemma SentencePiece, BERT WordPiece) is a property of the model, not of the engine that runs it, so the app registers it once. Addflutter_edge_ai_embeddingstopubspec.yaml, import it, and passembeddingTokenizers: [GemmaEmbeddingTokenizers()]toFlutterEdgeAi.initialize()besideembeddingBackends:. The error text names the package and the parameter. See Embeddings & RAG. -
On the old
flutter_gemma_*packages:Target of URI doesn't exist: package:flutter_gemma_embeddings/web_embedding_model.dartatflutter build web.flutter_gemma_embeddings2.2.0 moved that file intoflutter_gemma_litertlm1.8.0, alongside the rest of the LiteRT.js bundle it belongs to. A lockfile holdingflutter_gemma_litertlmat 1.7.x whileflutter_gemma_embeddingsmoves to 2.2.0 resolves cleanly and only then fails to compile. Upgradeflutter_gemma_litertlmto 1.8.0. Native builds are unaffected — that import sits behind a web-only conditional export.
Function calling#
-
Which models call functions, and how: see Function Calling and Models. If you pass
toolswithsupportsFunctionCalls: false, the chat logs a warning (debug builds only), does not write them into the prompt and does not parse calls out of the reply — the model still works for text generation. Gemma 4 and FunctionGemma on a.litertlmstill get the declarations through LiteRT-LM, and a.ggufmodel drops them with no warning. PasssupportsFunctionCalls: truefor models that support it.
