LogoFlutter Edge AI

Troubleshooting

Common issues — downloads, memory, iOS simulator GPU, Android minSdk, web caching, and desktop storage.

Common issues and fixes. For desktop-specific problems (Linux native logs, glibc, Windows DXC, stale GPU shader cache) see Desktop Support → Troubleshooting.

Downloads#

  • Resume isn't supported by the HuggingFace CDN. flutter_edge_ai uses smart retry with exponential backoff and automatic restart of interrupted downloads instead. Tune the attempt count via maxDownloadRetries in FlutterEdgeAi.initialize(...) (default: 10).
  • Large downloads on Android need foreground: true to get a foreground service (which shows a notification) and bypass Android's 9-minute background execution limit. The default (null) does not configure a notification, and without one the platform never starts the service — so the size threshold alone does nothing. iOS uses native URLSession and needs no special handling. See Models → downloads.
  • Using background_downloader in your own app too? flutter_edge_ai no longer listens to FileDownloader().updates (fixed in flutter_gemma 1.6.3). That stream takes a single subscription, so claiming it made every later FileDownloader().updates.listen(...) in the host app throw "Stream has already been listened to". Updates are now scoped to flutter_edge_ai's own task group and the stream stays yours.
  • Custom servers on Web must enable CORS headers. HuggingFace is already configured correctly; for Firebase Storage see the CORS configuration docs.

Gated models / download errors (401, 403)#

installModel(...).install() throws a public DownloadException carrying a sealed DownloadError, so you can react to gated HuggingFace models (HTTP 401/403) by type instead of substring-matching error strings:

try {
  await FlutterEdgeAi.installModel(modelType: ModelType.gemmaIt)
      .fromNetwork(url, token: hfToken)
      .install();
} on DownloadException catch (e) {
  switch (e.error) {
    case UnauthorizedError():   // 401 — missing/invalid HuggingFace token
    case ForbiddenError():      // 403 — token lacks access to this gated model
      showGatedModelDialog();
    case NotFoundError():       // 404 — bad URL (use /resolve/main/, not /blob/main/)
      showNotFoundDialog();
    case RateLimitedError():    // 429
    case ServerError():         // 5xx
    case NetworkError():        // connectivity
    case CanceledError():       // user canceled
    case UnknownError():
      showRetryDialog(e.error.toUserMessage());
  }
}

For a gated model (401/403): pass a valid huggingFaceToken to FlutterEdgeAi.initialize(...) (or token: on fromNetwork(...)), open the model page on HuggingFace, accept its license, and request access. Each DownloadError also exposes toUserMessage(), toTitle(), isRetryable, and requiresUserAction for building UI.

Auth errors (401/403/404) fail fast after one attempt — they are not retried. Only NetworkError, ServerError, and RateLimitedError are retryable (see isRetryable).

Memory#

  • iOS: ensure Runner.entitlements contains the memory entitlements and the deployment target is at least 15.0 (16.0 with flutter_edge_ai_mediapipe, 16.4 with flutter_edge_ai_llama) — in Xcode on the Runner target under SPM, or in the Podfile when one exists. See Installation → iOS.
  • iOS, MediaPipe .task on GPU: the app is terminated while the model loads, with the entitlements present and no crash report. Run .task models on PreferredBackend.cpu on iOS, or use the .litertlm build. See MediaPipe → Backends.
  • Reduce maxTokens if you hit memory pressure — but keep it at 1024 or higher for .litertlm models (see "maxTokens vs maxOutputTokens" below). To shorten replies, use maxOutputTokens, not a smaller maxTokens.
  • Use smaller models (1B-2B parameters) for devices with <6GB RAM. Gemma 4 E2B needs 6 GB or more (a 4 GB phone is killed while it loads); the larger Gemma 4 E4B and Gemma 3n files have no measured floor.
  • Close sessions and models when not needed; monitor usage with sizeInTokens().
  • To see what a model actually costs, measure the memory the OS cannot reclaim before and after loading it with flutter_edge_ai_diagnostics (Android + iOS + macOS + Linux + Windows). The RSS a profiler shows also counts mmapped weights the OS can drop, so it does not show that cost.

maxTokens vs maxOutputTokens#

maxTokens (on getActiveModel/createModel) is the context window — the total budget shared by the input (system prompt + history + your message) and the generated output (the KV-cache size). It is not the reply length.

.litertlm models require a context window of at least 1024. Passing a smaller maxTokens (e.g. 100) used to crash with DYNAMIC_UPDATE_SLICE failed to prepare / Failed to allocate tensors (even on CPU — the failure is at graph compile, before the backend runs). As of flutter_gemma_litertlm 1.0.2 a too-small maxTokens is clamped up to 1024 automatically with a log warning — except on PreferredBackend.npu, where the context is baked into the compiled bundle (see NPU).

To limit how many tokens the model generates, use maxOutputTokens on openSession or openChat instead:

final model = await FlutterEdgeAi.getActiveModel(maxTokens: 1024); // context window
final chat = await model.openChat(maxOutputTokens: 100);        // reply cap

(maxOutputTokens is honored on .litertlm, ONNX, .gguf and built-in AI on Android, iOS and macOS; the MediaPipe .task path has no session-level output cap and ignores it, and so does built-in AI on Windows and the Web.)

iOS#

  • Build issues: ensure the minimum iOS version is at least 15.0 (16.0 with flutter_edge_ai_mediapipe, 16.4 with flutter_edge_ai_llama). If the app has a Podfile (any app using flutter_edge_ai_mediapipe does — it ships no Package.swift), also use static linking (use_frameworks! :linkage => :static) and reinstall with cd ios && pod install --repo-update. An SPM-only app has no Podfile; there the deployment target lives on the Runner target in Xcode.
  • Simulator GPU disabled: iOS Simulator's Metal has a 256 MB single-allocation cap that LLM weight tensors exceed (e.g. Gemma 3 1B's KV cache alone is 288 MB). Use CPU on the simulator, or test GPU on a physical iPhone. This is a simulator limit, not a plugin bug.

macOS#

  • Library not loaded: @rpath/GemmaModelConstraintProvider.framework/… (or @rpath/libGemmaModelConstraintProvider.dylib before flutter_edge_ai_litertlm 1.19.0; inside an error that says LiteRT-LM did not load on macOS and names this cause): LiteRT-LM's companion libraries are not in the app. Every LiteRT feature fails to load (LLM, embeddings, speech), on every backend, and the build itself succeeds. With Swift Package Manager on, flutter_edge_ai_litertlm 1.19.0 and later bring them through its Swift package: upgrade, then run flutter clean and build again. With it off, replace the post_install block of macos/Podfile with ours, see Desktop → macOS; if the block is already there, run flutter clean and build again.
  • The package product 'flutter-edge-ai-litertlm-product' requires minimum platform version 14.0 for the macOS platform, or CocoaPods' required a higher minimum deployment target naming flutter_edge_ai_litertlm: raise the Runner target's macOS deployment target to 14.0 in Xcode, and platform :osx, '14.0' in macos/Podfile if the app has one. LiteRT-LM's companion libraries are built for macOS 14.
  • Flutter 3.44 with Xcode 27: The macOS deployment target 'MACOSX_DEPLOYMENT_TARGET' is set to 10.15, but the range of supported deployment target versions is 12.0 to …. Flutter 3.44's template targets macOS 10.15, which Xcode 27 refuses, for any app. Raise the Runner target to 12.0 in Xcode and platform :osx, '12.0' in macos/Podfile; with a Podfile, also add target.build_configurations.each { |c| c.build_settings['MACOSX_DEPLOYMENT_TARGET'] = '12.0' } inside the installer.pods_project.targets.each loop of post_install, because 3.44 raises pods only to 10.15. Flutter 3.47's template needs neither (measured on Xcode 27.1, 2026-10-21). With flutter_edge_ai_litertlm use 14.0 in all three places instead, because its companion libraries are built for macOS 14 (Desktop → macOS).
  • incompatible architecture (have 'arm64', need 'x86_64'): LiteRT-LM ships for Apple silicon only. It does not run on an Intel Mac or in an x86_64 (Rosetta) process.
  • library load disallowed by system policy: macOS refused a quarantined or un-notarized library. Notarize the app; for a local test build, remove the quarantine attribute with xattr -dr com.apple.quarantine on the app.
  • have different Team IDs: macOS library validation refused a library. Sign the app and its frameworks with the same team, or add the com.apple.security.cs.disable-library-validation entitlement.

Android#

  • .litertlm models require minSdk 30. libLiteRtLm.so depends on API 30+ Bionic syscalls (pthread_cond_clockwait, sem_clockwait) that can't be shimmed on older devices. MediaPipe .task models work on lower API levels.
  • .litertlm / embeddings / vision are arm64-v8a only. MediaPipe text inference (.task / .bin) also runs on x86_64 and armeabi-v7a. If you only use arm64-only features, add ndk { abiFilters 'arm64-v8a' } (in build.gradle.kts: ndk { abiFilters += listOf("arm64-v8a") }) so the Play Store doesn't offer broken APKs. See Installation → Android architecture.
  • GPU: nothing to add — the OpenCL <uses-native-library> entries come from the core plugin's own manifest (flutter_gemma 1.2.0+) through the manifest merger. If the GPU backend still falls back, check that the merged manifest contains libvndksupport.so and libOpenCL.so. See Installation → Android.
  • Which GPU flush ran (kernel_batch_size). On an Adreno GPU (/dev/kgsl-3d0 exists) the GPU load asks LiteRT-LM to flush every 2 kernels, so the UI keeps drawing during a long prefill (#364); on every other Android GPU it sets nothing and the engine's default applies, which is faster on Mali (#763). Debug builds log the choice ([LiteRtLmFfi] Android GPU (Adreno): hint_kernel_batch_size=2). In a release build, create the engine with LiteRtLmEngine(nativeLogLevel: LiteRtLmLogLevel.verbose) and look in logcat for the engine's kernel_batch_size line at load: -1 is the engine default (no flush), as reported in #763.
  • Gemma 4 on an Adreno GPU fails at the first token: Stream error: INTERNAL … Failed to invoke the compiled model, with Invalid work group size in logcat. The model loads, then the first decode fails. LiteRT's GPU kernels ask for more work items than some Adreno GPUs allow with the current native build: an Adreno 610 at the default half precision (a Galaxy A05s at maxTokens 4096; LiteRT #12075), and an Adreno 830 at ActivationDataType.float32 (a Galaxy S25 Ultra). Load the model on the CPU (preferredBackend: PreferredBackend.cpu); a future native build is expected to fix it. Since flutter_edge_ai_litertlm 1.21.0 the error message says so.
  • Google Play rejects the release: "Your app does not support 16 KB memory page sizes". Fixed in flutter_gemma_litertlm 1.8.0; every flutter_edge_ai_litertlm release includes the fix. Nothing fails at build or run time — the rejection happens at submission. The Qualcomm Hexagon DSP blobs of the NPU path (libQnnHtpV{73,75,79,81}Skel.so) come from Qualcomm with a 4 KB p_align, and Play scans lib/**/*.so without caring that a Hexagon image is loaded by the DSP rather than mapped by the kernel. Since 1.10.0 they ship only in apps that set qualcomm_npu: true, and the build hook raises them to 16 KB. Check your own build with Google’s check_elf_alignment.sh against the APK. See #529.
  • GPU backend crashes at engine_create on Mali GPUs (SIGSEGV, pc 0 in libLiteRtOpenClAccelerator.so). Fixed in flutter_gemma_litertlm 1.8.2; every flutter_edge_ai_litertlm release includes the fix. In 1.7.0–1.8.1 the OpenCL and GPU accelerators called AHardwareBuffer_allocate without declaring libandroid.so, so Android bound the call to address 0; only Mali GPUs (Samsung A-series, MediaTek, Google Tensor) take that path, so CPU and Adreno were unaffected. See #545.
  • Zero chunks and Stream error: <U+FFFD>, then SIGABRT. Fixed in flutter_gemma_litertlm 1.5.2; every flutter_edge_ai_litertlm release includes the fix. On Android the first dlopen of libLiteRtLm decides for the whole process whether its symbols are reachable from the default search scope, and bionic never promotes it afterwards — so an app that embedded or transcribed anything before its first generation left the stream-callback ABI probe blind and the wrong callback shape was registered. If your own or third-party code loads libLiteRtLm first, load it with RTLD_GLOBAL — the package cannot repair that case, but it raises a StateError naming it rather than generating corrupt text. See #447.

Web#

  • MediaPipe is GPU-only on web. The web engine ignores preferredBackend and always runs on the browser's GPU (WebGPU). ONNX on web is the exception: PreferredBackend.cpu pins WASM there.
  • Mobile .task models often don't work on web — use the -web.task (MediaPipe) or -web.litertlm (LiteRT-LM) export; only Gemma 4 and Gemma 3n publish the latter today. A native .litertlm installs on web and then fails at the first generation.
  • Memory / cache limits:
BrowserMax Model SizeNotes
Chrome/Firefox~2 GBArrayBuffer limit
Safari—⚠️ Not suitable
  • Large models (>2GB): use WebStorageMode.streaming (OPFS) to bypass the ~2 GB blob limit. Check support with await FlutterEdgeAi.isStreamingSupported(). See Installation → web storage.
  • Storage modes: cacheApi (default, persists across restarts, <2GB), streaming (OPFS, large models, requires Chrome 86+/Edge 86+/Safari 15.2+), none (ephemeral, testing only).

LiteRT-LM on web (early preview)#

Web .litertlm inference runs .litertlm models in the browser via the upstream @litert-lm/core package (WebGPU + WASM). It is an early preview and a subset of the native path. MediaPipe .task on web is not subject to these limits, though MediaPipe LLM Inference is maintenance-only — see MediaPipe.

Works on web .litertlm: text generation (sync + streaming), multi-turn chat with history, system instruction, function calling / tool calls (Gemma 4), thinking mode (Gemma 4 measured in Chrome; core parses <think> tags from any model that emits them), concurrent sessions (serialized), large models via OPFS streaming.

Not supported on web .litertlm yet:

  • ❌ Vision / image input (native only) — an image message throws UnsupportedError; the JS API creates the LLM engine with no vision executor.
  • ❌ Audio input (native only) — an audio message throws UnsupportedError; no audio executor either.
  • ❌ LoRA weights — loraPath throws UnsupportedError, as it does on native .litertlm; LoRA works only with MediaPipe .task models.

For vision on Web today, use a compatible MediaPipe .task build. Neither Web engine supports audio. These Web .litertlm limits track the upstream @litert-lm/core early-preview API and will lift as Google extends the JS executor surface.

Windows desktop GPU crashes#

Historical. Windows discrete GPUs crashed on PreferredBackend.gpu in flutter_gemma_litertlm 1.2.0–1.3.1, fixed in 1.4.0 — before the rename, so every flutter_edge_ai_litertlm release has the fix. macOS/Linux GPU and Windows CPU/NPU were never affected. See Desktop → Known limitations.

Wrong numbers on GPU#

Gemma 4 copies digits wrongly out of a long prompt on some GPUs, the same way on every run: asked when a delivery arrived (2026/06/23), it answers 20226/12/17. It shows from about 2,000 prompt tokens, on Metal and on Adreno. The published Gemma 4 .litertlm files ask for half-precision activations, and at half precision the rotary positions past about 2,048 tokens stop being distinct (LiteRT-LM #4001). Since flutter_edge_ai_litertlm 1.20.1 a model installed as ModelType.gemma4 and loaded with maxTokens above 2,048 and no activationDataType gets full precision on its GPU attempt by itself, except on Android (litertlm 1.21.0 keeps half precision there and prints a WARNING). With an older version, or a smaller maxTokens, ask for it, but not on an Adreno 8xx GPU (a Galaxy S25, for one): with the current native build float32 fails there at the first token, at any maxTokens:

final model = await FlutterEdgeAi.getActiveModel(
  preferredBackend: PreferredBackend.gpu,
  activationDataType: ActivationDataType.float32,
);

Prefill gets slower (about 3× on a Snapdragon 8 Elite and an iPhone 11, under 1.5× on an Apple M3 Max); decode speed barely changes. Upstream: LiteRT-LM#3012 (Adreno), LiteRT-LM#2814 (Metal).

Not on web. The web engine ignores activationDataType — it has no such setting — and so do MediaPipe, ONNX and built-in AI. On the engines that read it, it reaches the text decoder only: the vision and audio encoders keep the type the model file asks for.

Every flutter_edge_ai_litertlm release applies it (it arrived in flutter_gemma_litertlm 1.8.3; older versions accepted the argument and ignored it).

On Android the GPU shares system memory, so on a 4–6 GB phone running out of it at float32 can end the app rather than fall back to CPU. Both precisions share one compiled GPU program cache per model, so switching recompiles the GPU programs (about 600 MB for Gemma 4 E2B): pick one precision per install rather than per request.

float32 activations also need more GPU memory than float16. If the GPU engine cannot be created the model falls back to CPU without an exception, only a WARNING line in the log — the digits are then right and the model is far slower, which is easy to mistake for the fix working. Register LiteRtLmEngine(backendFallback: false) to get a BackendInitException with LiteRT-LM's reason instead, or read the backend the model actually got:

final model = await FlutterEdgeAi.getActiveModel(
  preferredBackend: PreferredBackend.gpu,
  activationDataType: ActivationDataType.float32,
);
if (model.activeBackend != PreferredBackend.gpu) {
  // CPU fallback: right digits, but not the run you asked for.
}

Windows embeddings and speech fail with status 3#

Fixed in litertlm 1.7.0. On Windows, embeddings and on-device speech (STT/TTS) fail with LiteRT call failed: CreateTensorBufferFromHostMemory(...) (status=3) in litertlm 1.4.0–1.6.4. Fixed in flutter_gemma_litertlm 1.7.0 and flutter_gemma_speech 0.5.1, so every flutter_edge_ai_* release has it. Text generation and the other platforms were never affected.

Qwen3 0.6B fails with "Failed to create engine"#

Qwen3-0.6B.litertlm from litert-community/Qwen3-0.6B fails at engine creation (Failed to create engine on <backend>: <LiteRT-LM's reason>. Model: …; before flutter_edge_ai_litertlm 1.14.0, Failed to create engine. Model may be invalid) whatever the maxTokens and backend. Since LiteRT-LM v0.17.0 (flutter_gemma_litertlm 1.7.0) the runtime rejects a piece in that bundle's SentencePiece tokenizer (piece must not include null character — on Android only in logcat); upstream LiteRT-LM#3865. On 2026-10-13 at 15:53 UTC the repo replaced the file's tokenizer (e6687177), and the file downloaded since loads (measured on macOS with flutter_edge_ai_litertlm 1.15.0). A copy downloaded before that keeps the old tokenizer and the same name, and the installer does not fetch a file it already has: uninstall the model and install it again. Or use Qwen3-0.6B_dynamic_wi4b32_afp32.litertlm from the same repo, installed the same way (ModelType.qwen3, ModelFileType.litertlm). See #659.

NPU#

The model answers, fluently, about the beginning of a long prompt and ignores the rest. That is most Gemma 3 bundles on an NPU. Nothing raises, nothing is logged: every prefill chunk after the first is dropped, so the model genuinely never saw the end of your prompt. On Qualcomm the published 270M bundles for SM8750 and SM8850 and the 1B …_sm8750 and …_sm8550 do it; the 1B …_sm8850 does not. In a bundle you build with npu_export it is a size threshold in the prefill mask (~1 MiB; at prefill 128 the largest working cache_length for Gemma 3 — 4 attention heads per KV head — is 896), and a build calibrated only on short prompts loses a longer prompt past ~450 tokens (Gemma 3 1B, litert-torch#1344). On Intel it happens regardless of size, through a different defect. Run a Gemma 4 bundle on the NPU (none is published for MediaTek: whether its Gemma 3 bundles keep their chunks is not measured), or the same model's generic bundle on PreferredBackend.cpu / .gpu — a bundle compiled for an NPU is refused there with a LiteRtLmBundleException. Upstream: LiteRT-LM#3508.

Note that maxTokens is not clamped up to 1024 on the NPU attempt the way it is on CPU and GPU — the context is baked into the compiled bundle (4096 for the Gemma 4 E2B NPU builds, 1280 for Gemma 3 1B and FastVLM, 4096 for Gemma 3 270M; the catalog's requiredMaxTokens), so keep maxTokens at or below it. A generic bundle is never tried on the NPU (it has no NPU graph: LiteRT-LM would run it on the CPU and report npu), and a bundle compiled for an NPU never runs on the GPU or CPU, so the floor applies only to the GPU and CPU attempts of a generic bundle.

PreferredBackend.npu on Android falls back to GPU, or an NPU bundle fails to load, after an upgrade. From flutter_edge_ai_litertlm 1.10.0 the Qualcomm QNN runtime is not part of the package; the app opts in with qualcomm_npu: true under hooks: user_defines: litertlm_dart: in its pubspec (flutter_edge_ai_litertlm: before flutter_edge_ai_litertlm 1.21.0) — see LiteRT-LM. Without it a bundle compiled for an NPU is refused with a BackendInitException whose NpuDispatchUnavailableException names it. A generic bundle never runs on the NPU, with or without the flag: it has no NPU graph, so npu falls back to GPU, then CPU, for it, and the log says so. If it says the stack is not enabled although the flag is set, the flag is in the wrong pubspec: in a pub workspace only the workspace root's user_defines count. If the build fails fetching qnn-runtime, the machine cannot reach Maven Central; set qualcomm_npu_maven_url to a mirror or qualcomm_npu_aar to a local copy of the same AAR.

PreferredBackend.npu on a Qualcomm Linux board falls back to GPU, or an NPU bundle fails to load. The same qualcomm_npu: true covers Linux arm64 builds. The BackendInitException for a bundle compiled for an NPU names which of these is missing:

  • The build did not bundle the stack — the flag is not set, or not in the workspace root's pubspec.
  • /dev/fastrpc-cdsp does not exist — the board exposes no compute DSP.
  • It, or /dev/dma_heap/system, is not accessible — add the user to group fastrpc (sudo usermod -aG fastrpc $USER) and log in again.
  • libcdsprpc.so.1 does not open — install Qualcomm's FastRPC (qcom-fastrpc1 on Ubuntu).

If the build fails reading the QAIRT SDK, the machine cannot reach softwarecenter.qualcomm.com; download v2.50.0.260828.zip elsewhere and set qualcomm_npu_qairt_zip to its path. Do not set DSP_LIBRARY_PATH: FastRPC prefers it over ADSP_LIBRARY_PATH, which is where the bundled Skels are announced.

A Google Tensor bundle fails to load. The BackendInitException names the cause: the app does not set google_tensor_npu: true (in a pub workspace, at the root); the device runs Android below 16 (API 36) or a BP2A build; it is not a Tensor SoC; or the Pixel's libedgetpu_litert.so did not open. A G5 bundle on a G6 Pixel, or the reverse, is refused by LiteRT when the engine is created. It never falls back to the GPU or the CPU.

A MediaTek bundle fails to load. The BackendInitException names the cause: the app does not set mediatek_npu: true (in a pub workspace, at the root); the device runs Android below 12 (API 31) or its SoC manufacturer is not MediaTek; or none of the phone's NeuroPilot adapters (libneuron_adapter_mgvi.so, libneuronusdk_adapter*.mtk.so, …) opened in the app, which means the phone's vendor does not expose NeuroPilot to apps. A bundle compiled for another SoC is refused when the engine is created (measured on an MT6989 with MT6991 and MT6993 bundles: NeuroPilot reports an unsupported target or a Neuron SDK version mismatch). It never falls back to the GPU or the CPU. A build without network access fails while fetching the dispatch AAR: download it and set mediatek_npu_aar: <path>.

PreferredBackend.npu on Windows fails at load or the process ends with 0xC0000409. Every release from flutter_gemma_litertlm 1.7.0 (native-v0.17.0) through flutter_edge_ai_litertlm 1.14.0 has this bug in its Intel NPU load (LiteRT-LM#4023); upgrade to 1.15.0.

PreferredBackend.npu is unavailable on a recent Snapdragon. SoC coverage is the runtime's, not ours: Snapdragon 8 Gen 5 (SM8845 — OnePlus 15R, iQOO 15R and the like) is not covered upstream, and a context compiled for SM8850 is rejected by an SM8845 device even though both are Hexagon v81. Tracked at LiteRT#7516; note the easily-confused naming — 8s Gen 4 is SM8735, not SM8845.

Desktop storage locations#

Desktop builds store downloaded models outside the user's Documents/ folder to avoid OneDrive / iCloud / Domain-Roaming sync corrupting FFI mmap of large .litertlm files:

  • Windows: %LOCALAPPDATA%\flutter_gemma\ (never OneDrive-synced)
  • macOS: ~/Library/Application Support/<bundle>/flutter_gemma/
  • Linux: ~/.local/share/<app>/flutter_gemma/

The directory keeps its flutter_gemma name across the rename, so installed models stay found.

Models installed by older 0.14.x / 0.15.0 builds that still live under Documents/ keep working via a fallback read.

On Windows, flutter_gemma before 1.4.0 could write a fresh download to a $CWD-relative path (<cwd>\Users\…\AppData\Local\flutter_gemma\) instead of the absolute %LOCALAPPDATA%\flutter_gemma\, because %LOCALAPPDATA% is not one of background_downloader's base directories. The model then reported "installed" but failed to load with "model file paths not found". Fixed in 1.4.0 — it affected every fresh inference / embedding / STT download on Windows, so upgrade if you hit it.

Multimodal#

  • Ensure the model has a vision (or audio) encoder — see Multimodal and Models.
  • Set supportImage: true (and supportAudio: true for audio) when creating the model. On MediaPipe (Android and iOS), pass supportImage: true to openChat as well, or the image is dropped.
  • Check device memory — multimodal models require more RAM.
  • Image input crashes at model load on a GPU text backend (older releases). On Metal (iOS/macOS) and WebGPU (Windows/Linux) the vision encoder's ops can't be prepared by the GPU delegate. Fixed in flutter_gemma_litertlm 1.4.2 / core 1.5.9 — a vision encoder its bundle says nothing about runs on CPU while the text decoder keeps the GPU. Upgrade if you hit it.
  • Use the GPU backend for faster text decoding. An unset encoder backend follows the bundle: the section's backend_constraint, else the NPU it is compiled for, else CPU. Set preferredVisionBackend/preferredAudioBackend only where the section allows it ((await readLiteRtLmBundleInfo(path)).audioEncoder?.backendConstraint, or visionEncoder); the published Gemma 4 and Gemma 3n audio encoders declare cpu, and a backend the section excludes is refused before the model loads. See Multimodal.

Native libraries fetched at build time#

Some packages download their native library from a GitHub Release when you first build for a platform, then cache it. LiteRT-LM (litertlm_dart, under flutter_edge_ai_litertlm) caches each release in its own directory under ~/.cache/litertlm_dart/native/ (~/Library/Caches/… on macOS, %LOCALAPPDATA%\… on Windows); flutter_edge_ai_onnx and flutter_edge_ai_sqlite cache under ~/.cache/flutter_gemma/native/. This applies to flutter_edge_ai_litertlm (always has), flutter_edge_ai_onnx, and flutter_edge_ai_sqlite (every release; as flutter_gemma_rag_sqlite from 1.3.0 — before that it shipped the loadables inside the package).

  • The build fails with a download error or an HTTP status. The first build of each platform needs github.com reachable; an Android build with flutter_edge_ai_onnx also needs repo1.maven.org, and the qualcomm_npu opt-in needs Maven Central (Android) or softwarecenter.qualcomm.com (Linux arm64). In an air-gapped or proxied CI, copy the cache directories (litertlm_dart/native and flutter_gemma/native) from a machine that built the same package versions; for flutter_gemma/native, include its hidden version-marker files — a folder copied without them is discarded and fetched again. Only the Qualcomm NPU stack has settings that point the build at a mirror or a copy you vendor yourself (qualcomm_npu_maven_url, qualcomm_npu_aar, qualcomm_npu_qairt_zip; see NPU).
  • The build fails with CHECKSUM MISMATCH. The bytes served do not match what the package version was pinned to. Re-run once to rule out a corrupt transfer. If it persists, the release asset was replaced after publication — do not work around it by clearing the checksum; report it.
  • A build that used to succeed now fails instead of quietly skipping. That is deliberate. These hooks used to report success while bundling nothing, which surfaced later as an opaque dlopen crash on a user's device. A platform the package claims to support now fails the build when its library cannot be produced.
  • Maintainers only: a local native/<name>/prebuilt/<target>/ overrides the pinned release. flutter_edge_ai_sqlite says so on stderr when it takes that path; litertlm_dart (under flutter_edge_ai_litertlm) and flutter_edge_ai_onnx take it silently. If a new release "did not take", look for that directory first.

Another package bundles its own LiteRT-LM#

The LiteRT-LM library at … is not the one litertlm_dart ships (it lacks litert_lm_engine_settings_set_kernel_batch_size) — a StateError (an EmbeddingLoadException for an embedder) when a .litertlm model, an embedder or speech loads. Another package in the app ships its own LiteRT-LM under the same library name — llamadart before 0.12.0 does by default (from 0.12.0 its build hook leaves it out next to flutter_edge_ai_litertlm), and llamadart_litert_lm_flutter links one through Swift Package Manager. On iOS and macOS that app still builds: depending on the order of the app's dependencies, Flutter renames our framework to LiteRtLm1.framework, and LiteRtLm.framework is the other package's build, which flutter_edge_ai_litertlm refuses rather than binds. On Android, Linux and Windows the build fails instead.

For llamadart, upgrade it to 0.12.0 or later, or leave its LiteRT-LM runtime out in the app's pubspec.yaml (the workspace root's, if the app is a workspace member) and remove llamadart_litert_lm_flutter if the app depends on it; then run flutter clean once:

hooks:
  user_defines:
    llamadart:
      llamadart_native_runtimes: [llama_cpp]

Embeddings#

  • EmbeddingLoadException naming web/litert_embeddings.js on web, at the first getActiveEmbedder() after upgrading to flutter_edge_ai_litertlm 1.13.0. The LiteRT.js bundle changed its API, and your app's web/ still holds the copy of an earlier version. Copy all four files again from the package's web/ (litert_embeddings.js, sentencepiece.js, litert.js, tensorflow.js). See Installation → Web.
  • EmbeddingLoadException: window.EmbeddingEngine is not set on web, when an EmbeddingGemma 2 (.litertlm) embedder loads. Since flutter_edge_ai_litertlm 1.13.1 it runs on @litert-lm/core's EmbeddingEngine, which your web/index.html does not expose yet. Add window.EmbeddingEngine = m.EmbeddingEngine; next to window.Engine = m.Engine; in the @litert-lm/core module script. See Installation → Web.
  • EmbeddingLoadException: EmbeddingGemma 2 on the web needs WebGPU on web, when an EmbeddingGemma 2 (.litertlm) embedder loads in a browser without WebGPU — no navigator.gpu, or no GPU adapter (WebGPU switched off, a blocklisted driver, a headless browser without a GPU). On the web EmbeddingGemma 2 runs on WebGPU only: the CPU (WebAssembly) path of @litert-lm/core 0.18.0 fails on these bundles with "Failed to get external weight", so there is nothing to fall back to. Use a browser with WebGPU enabled; check navigator.gpu and await navigator.gpu.requestAdapter() in its console. See Embeddings & RAG.
  • EmbeddingInputTooLongException from embed, embedAll or RagIndex.add/search. The input is longer than the model's maxInputTokens, task prefix, BOS and EOS included; since 2.3.0 it is refused instead of silently truncated. Split the text into smaller chunks — at most 512 tokens for EmbeddingGemma 2, whose limit is 1024. See Embeddings & RAG.
  • EmbeddingLoadException … No embedding tokenizer is configured from getActiveEmbedder(), for a .tflite or ONNX embedder (an EmbeddingGemma 2 .litertlm carries its own and needs none). Since flutter_gemma 1.9.0 an embedding backend no longer carries a tokenizer: which family a model needs (Gemma SentencePiece, BERT WordPiece) is a property of the model, not of the engine that runs it, so the app registers it once. Add flutter_edge_ai_embeddings to pubspec.yaml, import it, and pass embeddingTokenizers: [GemmaEmbeddingTokenizers()] to FlutterEdgeAi.initialize() beside embeddingBackends:. The error text names the package and the parameter. See Embeddings & RAG.
  • On the old flutter_gemma_* packages: Target of URI doesn't exist: package:flutter_gemma_embeddings/web_embedding_model.dart at flutter build web. flutter_gemma_embeddings 2.2.0 moved that file into flutter_gemma_litertlm 1.8.0, alongside the rest of the LiteRT.js bundle it belongs to. A lockfile holding flutter_gemma_litertlm at 1.7.x while flutter_gemma_embeddings moves to 2.2.0 resolves cleanly and only then fails to compile. Upgrade flutter_gemma_litertlm to 1.8.0. Native builds are unaffected — that import sits behind a web-only conditional export.

Function calling#

  • Which models call functions, and how: see Function Calling and Models. If you pass tools with supportsFunctionCalls: false, the chat logs a warning (debug builds only), does not write them into the prompt and does not parse calls out of the reply — the model still works for text generation. Gemma 4 and FunctionGemma on a .litertlm still get the declarations through LiteRT-LM, and a .gguf model drops them with no warning. Pass supportsFunctionCalls: true for models that support it.