LogoFlutter Edge AI

Installation

Add the packages, register engines, and complete per-platform setup for iOS, Android, Web, and Desktop.

Flutter Edge AI is a small core package plus opt-in packages for each engine / backend (split since 1.0, then flutter_gemma), so your app only pulls the native weight it actually uses. Add the core package, then the packages for the model formats and features you need.

1. Add packages to pubspec.yaml#

dependencies:
  flutter_edge_ai: ^2.7.0                      # Core — always required (no engine on its own)

  # Inference engines — add at least one:
  flutter_edge_ai_litertlm: ^1.21.0            # .litertlm models (FFI; mobile + desktop + web) + LiteRtEmbeddingBackend
  flutter_edge_ai_mediapipe: ^1.3.1            # .task / .bin models (MediaPipe; mobile + web)
  flutter_edge_ai_builtin_ai: ^0.4.3           # OS system models — Gemini Nano (Android) / Apple FM (iOS 26+/macOS 26+) / Windows AI Foundry / Chrome Prompt API (Web)
  flutter_edge_ai_onnx: ^0.10.0                 # ONNX models — ORT-GenAI (FFI, native) / Transformers.js (web) + OnnxEmbeddingBackend
  flutter_edge_ai_llama: ^0.2.0                # .gguf models — llama.cpp through llamadart, + LlamaEmbeddingBackend (EmbeddingGemma 2 text, native) (see /docs/llama)

  # Optional — text-embedding tokenizer implementations (.tflite / .onnx models; an EmbeddingGemma 2 .litertlm needs none):
  flutter_edge_ai_embeddings: ^2.3.3           # embedding tokenizer implementations (Gemma SentencePiece, BERT WordPiece), registered via embeddingTokenizers: (needs a backend, e.g. LiteRtEmbeddingBackend above)

  # Optional — independent RAG orchestration plus one storage provider:
  flutter_edge_ai_rag: ^1.2.0               # RagIndex, profiles, filters
  flutter_edge_ai_qdrant: ^2.1.1            # qdrant-edge; fastest on native
  flutter_edge_ai_sqlite: ^2.2.0            # sqlite-vec / vec0; all platforms incl. web — needs Flutter 3.47

  # Optional — on-device speech (STT + TTS):
  flutter_edge_ai_speech: ^0.6.2               # transcribe audio + synthesize speech (on-device STT + TTS; native only) + voice loop

  # Optional — on-device agent skills:
  flutter_edge_ai_agent: ^0.4.3                # agent skills the model runs itself (text / JS / native-intent / MCP)

  # Optional — memory diagnostics (Android + iOS + macOS + Linux + Windows):
  flutter_edge_ai_diagnostics: ^0.6.1          # what a model costs in memory the OS cannot reclaim, read from the OS

Pick by need:

You want to…Add
Run .litertlm models on any platform, desktop included (see Models ) flutter_edge_ai_litertlm
Run .task / .bin models on mobile and web (see Models ); MediaPipe is maintenance-only, so prefer .litertlm in new apps flutter_edge_ai_mediapipe
Run the OS system model with no download (Gemini Nano / Apple FM / Windows AI Foundry / the browser Prompt API) flutter_edge_ai_builtin_ai
Run ONNX models — ORT-GenAI (native) or Transformers.js (Web) flutter_edge_ai_onnx
Run .gguf models (llama.cpp) flutter_edge_ai_llama (see setup)
Generate text embeddings with EmbeddingGemma 2 (.litertlm) flutter_edge_ai_litertlm (LiteRtEmbeddingBackend) alone; 1.13.1 on Web
Embed images with EmbeddingGemma 2 (the 440m or 740m .litertlm) flutter_edge_ai_litertlm 1.13.1 ( LiteRtEmbeddingBackend ) alone; 1.13.1 on Web
Embed audio, or several parts in one vector, with EmbeddingGemma 2 (the 740m .litertlm) flutter_edge_ai_litertlm 1.13.1 ( LiteRtEmbeddingBackend ) with flutter_edge_ai 2.3.1; 1.13.2 on Web
Generate text embeddings with EmbeddingGemma 2 (.gguf, native) flutter_edge_ai_llama 0.0.3 ( LlamaEmbeddingBackend ), no tokenizer package; on iOS/macOS also llamadart_llama_cpp_flutter ; see llama.cpp
Generate text embeddings with EmbeddingGemma 1 / Gecko (.tflite) flutter_edge_ai_embeddings + flutter_edge_ai_litertlm ( LiteRtEmbeddingBackend )
Generate text embeddings from ONNX/ORT models flutter_edge_ai_embeddings + flutter_edge_ai_onnx ( OnnxEmbeddingBackend )
On-device RAG on native, fastest (Android/iOS/desktop) flutter_edge_ai_rag + flutter_edge_ai_qdrant
On-device RAG on web, or a portable/exact store on any platform flutter_edge_ai_rag + flutter_edge_ai_sqlite (Flutter 3.47+)
Transcribe audio, synthesize speech, or run a voice loop on-device (STT + TTS + voice) flutter_edge_ai_speech
Run on-device agent skills the model executes itself (text / JS / native-intent / MCP) flutter_edge_ai_agent
Measure what a model costs in memory the OS cannot reclaim (Android + iOS + macOS + Linux + Windows) flutter_edge_ai_diagnostics

Core registers no AI runtime by itself — wire inference, embedding, and speech packages in FlutterEdgeAi.initialize(...) below. RAG is independent: construct FlutterEdgeAiRag with a storage provider. Run flutter pub get to install.

Migrating from flutter_gemma or Flutter Edge AI 1.x? See the Migration guide. Version 2.0 moves RAG out of core and adds durable embedding profiles.

2. Initialize Flutter Edge AI#

Call await FlutterEdgeAi.initialize(...) once in main() and register the opt-in packages you added to pubspec.yaml. Core registers no engine on its own, so without this step getActiveModel() / getActiveEmbedder() throw a clear "add the engine package" error.

import 'package:flutter/widgets.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:flutter_edge_ai_embeddings/flutter_edge_ai_embeddings.dart';
import 'package:flutter_edge_ai_litertlm/flutter_edge_ai_litertlm.dart';
import 'package:flutter_edge_ai_mediapipe/flutter_edge_ai_mediapipe.dart';
import 'package:flutter_edge_ai_builtin_ai/flutter_edge_ai_builtin_ai.dart';
import 'package:flutter_edge_ai_speech/flutter_edge_ai_speech.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();

  await FlutterEdgeAi.initialize(
    // Inference engines — add the ones whose packages you depend on:
    inferenceEngines: const [
      LiteRtLmEngine(),     // flutter_edge_ai_litertlm  — .litertlm models
      MediaPipeEngine(),    // flutter_edge_ai_mediapipe — .task / .bin models
      BuiltInAiEngine(),    // flutter_edge_ai_builtin_ai — Gemini Nano / Apple FM
    ],
    // Optional — embeddings (also usable by an independent RAG index):
    embeddingBackends: const [
      LiteRtEmbeddingBackend(), // flutter_edge_ai_litertlm
    ],
    // The tokenizer is registered separately — which family a model needs is a
    // property of the model, not of the engine that runs it.
    embeddingTokenizers: const [
      GemmaEmbeddingTokenizers(), // flutter_edge_ai_embeddings
    ],
    // Optional — on-device speech-to-text:
    sttBackends: const [
      LiteRtSttBackend(), // flutter_edge_ai_speech
    ],
    // Optional — on-device text-to-speech:
    ttsBackends: const [
      LiteRtTtsBackend(), // flutter_edge_ai_speech
    ],
    // Common settings:
    // String.fromEnvironment yields '' when the define is absent, and an
    // empty token still sends a bare `Authorization: Bearer` header. Pass
    // null instead so the request goes out unauthenticated.
    huggingFaceToken: const String.fromEnvironment('HUGGINGFACE_TOKEN').isNotEmpty
        ? const String.fromEnvironment('HUGGINGFACE_TOKEN')
        : null,
    maxDownloadRetries: 10,
  );

  runApp(MyApp());
}

Which parameter ← which package:

ParameterProvided byNotes
inferenceEngines: [LiteRtLmEngine()] flutter_edge_ai_litertlm .litertlm (mobile + desktop + web)
inferenceEngines: [MediaPipeEngine()] flutter_edge_ai_mediapipe .task / .bin (mobile + web)
inferenceEngines: [OnnxEngine()] flutter_edge_ai_onnx ONNX models — ORT-GenAI (FFI: macOS arm64, Linux x64, Windows x64, Windows ARM64, Android arm64, iOS arm64) or Transformers.js (Web)
inferenceEngines: [BuiltInAiEngine()] flutter_edge_ai_builtin_ai the OS model, ModelFileType.builtIn (Android, iOS, macOS, Windows, Web)
inferenceEngines: [LlamaEngineProvider()] flutter_edge_ai_llama .gguf models via llama.cpp, ModelFileType.gguf (mobile + desktop + web) — llama.cpp
embeddingBackends: [LiteRtEmbeddingBackend()] flutter_edge_ai_litertlm embeddings: EmbeddingGemma 2 .litertlm (text, images, audio; native and WebGPU) and .tflite models
embeddingBackends: [OnnxEmbeddingBackend()] flutter_edge_ai_onnx text embeddings from ONNX/ORT models
embeddingBackends: [LlamaEmbeddingBackend()] flutter_edge_ai_llama EmbeddingGemma 2 .gguf text embeddings, native only (automatic placement: GPU on Apple silicon and iOS devices; CPU on Android, iOS simulators and macOS x64; Linux and Windows the SDK's own choice, Vulkan where available); the GGUF carries its own tokenizer — see llama.cpp
embeddingTokenizers: [GemmaEmbeddingTokenizers()] flutter_edge_ai_embeddings required by LiteRtEmbeddingBackend and OnnxEmbeddingBackend for .tflite and .onnx models — on the web for .onnx only, since LiteRT.js tokenizes .tflite in the browser; an EmbeddingGemma 2 .litertlm carries its own tokenizer
sttBackends: [LiteRtSttBackend()] flutter_edge_ai_speech speech-to-text (native only)
ttsBackends: [LiteRtTtsBackend()] flutter_edge_ai_speech text-to-speech (native only)

Add only the runtimes you ship. Passing both LiteRtLmEngine() and MediaPipeEngine() lets one app run both formats — the registry routes each model to the engine that handles its file type. For RAG, register SqliteVectorStoreProvider() or QdrantVectorStoreProvider() on a separate FlutterEdgeAiRag instance; see Embeddings & RAG.

Common settings:

  • huggingFaceToken: authentication token for gated models (Gemma3n, Gemma 3 1B / 270M, EmbeddingGemma 1).
  • maxDownloadRetries: number of retry attempts for failed downloads (default: 10).
  • webStorageMode (Web only): storage strategy for model files (default: cacheApi).
    • WebStorageMode.cacheApi: Cache API with Blob URLs (for models <2GB). Since core 2.7.0 a download streams into the Cache API without a Dart copy, so a --wasm build installs files over 1 GiB too.
    • WebStorageMode.streaming: OPFS streaming (for large models >2GB like E4B, 7B).
    • WebStorageMode.none: no caching (ephemeral mode for testing). Restore notices about model files that are gone stay in the debug log in this mode.

Use WebStorageMode.streaming when shipping .litertlm web models — the @litert-lm/core engine consumes an OPFS ReadableStream and avoids Chrome's ~2 GB blob-fetch limit on Gemma 4 E2B/E4B web builds.

3. Platform-specific setup#

Complete platform-specific setup before running a model.

iOS#

Required by any engine package: flutter_edge_ai_litertlm, flutter_edge_ai_mediapipe, flutter_edge_ai_onnx, flutter_edge_ai_llama, flutter_edge_ai_builtin_ai and/or flutter_edge_ai_speech.

Set the minimum iOS version to 15.0 — or 16.0 if your app depends on flutter_edge_ai_mediapipe, which needs MediaPipe GenAI, 16.4 with flutter_edge_ai_llama, whose llama.cpp companion needs it, and 15.1 with flutter_edge_ai_onnx, whose iOS ONNX Runtime is built for it. Core, flutter_edge_ai_litertlm and embeddings build from 15.0. (flutter_gemma before 1.6.4 declared 16.0.)

flutter_edge_ai_builtin_ai no longer sets an Apple floor of its own — since 0.3.0 its native layer is flutter_local_ai, which builds from iOS 13.0 / macOS 12.0. On iOS that changes nothing (core's 15.0 still wins); on macOS it raises the floor from 10.15 to 12.0, and a lower deployment target fails resolution with a message naming the flutter_local_ai pod rather than the package you added. flutter_edge_ai_litertlm (and so speech) needs macOS 14.0, from 1.19.0 by declaration; its companion libraries are built for macOS 14.

Where you set it depends on the dependency manager. Swift Package Manager is the default since Flutter 3.44 (opt-in before that), and an SPM-only app has no Podfile at all. Flutter 3.47+ creates (and migrates) apps at iOS 15.0; on 3.44.x the template is 13.0 — set iOS Deployment Target on the Runner target in Xcode, or the build fails with requires minimum platform version 15.0 … but this target supports 13.0. Neither iOS nor macOS needs a Podfile step with Swift Package Manager on; macOS with it off does — see Desktop → macOS. flutter_edge_ai_mediapipe ships no Package.swift, so an app using it also gets a Podfile; set the platform there as well:

platform :ios, '16.0'   # 15.0 if the app does not use flutter_edge_ai_mediapipe

Declare platform only once — CocoaPods rejects a second one with Invalid Podfile file: The target 'Pods' already has a platform set.

Change the linking type of pods to static in Podfile:

use_frameworks! :linkage => :static

Enable file sharing in Info.plist:

<key>UIFileSharingEnabled</key>
<true/>

Add a network access description in Info.plist (for development):

<key>NSLocalNetworkUsageDescription</key>
<string>This app requires local network access for model inference services.</string>

Enable performance optimization in Info.plist (optional):

<key>CADisableMinimumFrameDurationOnPhone</key>
<true/>

Add memory entitlements in Runner.entitlements (for large models):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>com.apple.developer.kernel.extended-virtual-addressing</key>
	<true/>
	<key>com.apple.developer.kernel.increased-memory-limit</key>
	<true/>
	<key>com.apple.developer.kernel.increased-debugging-memory-limit</key>
	<true/>
</dict>
</plist>

No host-side Podfile post_install is required on iOS — flutter_edge_ai patches the upstream LiteRT-LM dlopen path to use @executable_path/Frameworks/<X>.framework/<X> so dyld resolves Metal accelerators directly through the Native-Assets-bundled framework. This also keeps Runner.app/Frameworks/ App-Store-clean (fixes ITMS-90432).

Android#

Release builds need <uses-permission android:name="android.permission.INTERNET"/> in android/app/src/main/AndroidManifest.xml to download models (Flutter's template adds it only to the debug and profile manifests).

Add-to-app hosts must declare the Kotlin Gradle Plugin themselves. Flutter auto-applies KGP to plugin modules only when the host provides it, so a Java-only native host fails with Could not find method kotlin(). Add KGP to the host's root buildscript/plugins {}. A normal flutter build app needs nothing — Flutter's own Gradle plugin carries KGP.

GPU and NPU (any engine): nothing to add to the manifest. flutter_edge_ai's own manifest declares the OpenCL, FastRPC, Google Tensor and (from core 2.5.1) MediaTek NeuroPilot runtime namespace entries and the manifest merger folds them into your app. These are what your merged manifest must contain if you pin or audit it — note libvndksupport.so: without it the OpenCL ICD load is denied on Android 12+, and OpenCL is the only GPU path on Android (no WebGPU libraries ship there since native-v0.18.0).

<uses-native-library android:name="libvndksupport.so" android:required="false"/>
<uses-native-library android:name="libOpenCL.so" android:required="false"/>
<uses-native-library android:name="libOpenCL-car.so" android:required="false"/>
<uses-native-library android:name="libOpenCL-pixel.so" android:required="false"/>
<uses-native-library android:name="libcdsprpc.so" android:required="false"/>
<uses-native-library android:name="libedgetpu_litert.so" android:required="false"/>
<uses-native-library android:name="libedgetpu_util.so" android:required="false"/>

From core 2.5.1 it also declares 25 MediaTek NeuroPilot libraries (libneuron_adapter_mgvi.so, libneuronusdk_adapter*.mtk.so, libapuware*.mtk.so, libmvpu*.mtk.so, …), all required="false"; the full list is in flutter_edge_ai's android/src/main/AndroidManifest.xml.

Qualcomm NPU (opt-in). PreferredBackend.npu needs Qualcomm's QNN runtime, which Qualcomm licenses for redistribution only inside an application, so the app asks for it in its pubspec.yaml (the workspace root's, if the app is a pub workspace member):

hooks:
  user_defines:
    litertlm_dart:
      qualcomm_npu: true

The litertlm_dart: key is read from flutter_edge_ai_litertlm 1.21.0; earlier versions read the same flags under flutter_edge_ai_litertlm:. The build hook then fetches the runtime — from Maven Central for Android, from Qualcomm's QAIRT SDK for Linux arm64 boards. Without the flag a bundle compiled for an NPU is refused with a BackendInitException that says why; a generic bundle never runs on the NPU, flag or not. See LiteRT-LM.

Google Tensor NPU (opt-in). A bundle compiled for a Tensor NPU (…_Google_Tensor_G5.litertlm, …_Google_Tensor_G6.litertlm) needs google_tensor_npu: true in the same user_defines block (flutter_edge_ai_litertlm 1.15.0). The build then bundles the Tensor dispatch, built from LiteRT and covered by Google's Tensor SDK terms; the runtime it drives is part of the Pixel system image. It runs on a Pixel with that Tensor generation and Android 16 or later, except on a build whose ID starts with BP2A, which LiteRT does not run the NPU on. Run so far on Tensor G6 only, with debug builds. See LiteRT-LM.

MediaTek NPU (opt-in). A bundle compiled for a MediaTek SoC (…_MediaTek_MT6991.litertlm, …_mt6991.litertlm) needs mediatek_npu: true in the same user_defines block (flutter_edge_ai_litertlm 1.17.0). The build then fetches Google's litert-npu-runtime-mediatek AAR from Google Maven and bundles its dispatch behind this package's adapter (needed by LiteRT-LM 0.19); the NeuroPilot runtime it drives is part of the phone. It runs on a phone with that SoC and Android 12 or later whose vendor exposes NeuroPilot to apps. Offline, download the AAR and set mediatek_npu_aar: <path>. See LiteRT-LM.

ProGuard/R8 (only if you use flutter_edge_ai_mediapipe): the package ships its own consumer ProGuard rules, so a release build needs no rules in your app (built on AGP 9.1). The old flutter_gemma_mediapipe 1.0.5 and earlier failed R8 with Missing class (seen on AGP 9); if you are still on it, move to flutter_edge_ai_mediapipe, or add to your proguard-rules.pro:

-dontwarn com.google.auto.value.**
-dontwarn com.google.mediapipe.proto.CalculatorProfileProto$CalculatorProfile
-dontwarn com.google.mediapipe.proto.GraphTemplateProto$CalculatorGraphTemplate

If a release build then fails at run time with UnsatisfiedLinkError or a missing MediaPipe class, also add:

# MediaPipe
-keep class com.google.mediapipe.** { *; }

# Protocol Buffers
-keep class com.google.protobuf.** { *; }
-dontwarn com.google.protobuf.**

flutter_edge_ai_litertlm is delivered as a Native-Assets dylib (no MediaPipe Java classes), so it needs no ProGuard rules.

flutter_edge_ai_builtin_ai brings in flutter_local_ai, which declares minSdk 26 (the ML Kit GenAI / AICore floor) — raise your app's minSdk to 26 or the manifest merger fails. The ML Kit Prompt API beta4 artifact carries Kotlin 2.3 metadata, so the app also needs Kotlin 2.3.21. Since 0.2.0 flutter_local_ai no longer applies KGP itself (Flutter 3.44+ does); it only puts KGP 2.3.21 on its buildscript classpath, so it does not stop an app on AGP 9 from using built-in Kotlin — other plugins in the app still may.

Android architecture support

MediaPipe text inference (.task / .bin) works on arm64-v8a, x86_64, and armeabi-v7a. Everything backed by libLiteRtLm (.litertlm inference, including its vision and audio input, embedding via LiteRT FFI, speech) is arm64-v8a only:

Android feature arm64-v8a x86_64 armeabi-v7a
Text inference (.task / .bin) ✅ ✅ ✅
.litertlm (FFI) ✅ ❌ ❌
Embedding (LiteRT FFI) ✅ ❌ ❌
Speech STT + TTS (LiteRT FFI) ✅ ❌ ❌

If your app uses only the arm64-only features, restrict the build to arm64 so the Play Store does not offer broken APKs to incompatible devices:

android {
    defaultConfig {
        ndk { abiFilters 'arm64-v8a' }
    }
}

In a Kotlin build script (build.gradle.kts):

android {
    defaultConfig {
        ndk { abiFilters += listOf("arm64-v8a") }
    }
}

Anything backed by libLiteRtLm.so on Android — .litertlm inference, embeddings, and speech (STT + TTS) — requires minSdk 30: the library depends on API 30+ Bionic syscalls (pthread_cond_clockwait, sem_clockwait) that cannot be shimmed on older devices. MediaPipe .task models work on lower API levels.

Web#

On web, MediaPipe ignores preferredBackend and always runs on the GPU (WebGPU); ONNX honours PreferredBackend.cpu by pinning WASM.

Every web app needs flutter_edge_ai's model storage helpers. Copy cache_api.js and opfs_helper.js from the flutter_edge_ai package's web/ directory into your app's web/ (find the package directory with grep -A1 '"name": "flutter_edge_ai"' .dart_tool/package_config.json), then load them in web/index.html by adding the following within the <head> section:

<script src="cache_api.js"></script>
<script src="opfs_helper.js"></script>

Then add the CDN script(s) for the engine package(s) you use.

flutter_edge_ai_mediapipe (.task / -web.task models):

<script type="module">
import { FilesetResolver, LlmInference } from 'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-genai@0.10.30';
window.FilesetResolver = FilesetResolver;
window.LlmInference = LlmInference;
</script>

flutter_edge_ai_litertlm (.litertlm web models — early preview, text only). Only dedicated web exports (-web.litertlm) load — Gemma 4 and Gemma 3n today; a native .litertlm installs on web and then fails at the first generation. The @litert-lm/core ESM doesn't assign window globals and module scripts are deferred, so Dart must await window.litertLmReady before any static interop:

<script type="module">
window.litertLmReady = (async () => {
  const m = await import('https://cdn.jsdelivr.net/npm/@litert-lm/core@0.18.0/+esm');
  window.Engine = m.Engine;
  window.EmbeddingEngine = m.EmbeddingEngine;
  return m.Engine;
})();
</script>

window.Engine runs .litertlm LLMs. window.EmbeddingEngine (litertlm 1.13.1+) runs EmbeddingGemma 2 .litertlm embedders on WebGPU — text and images, and from 1.13.2 audio and combined inputs; without that line such an embedder fails to load with an EmbeddingLoadException that names it. Both need a browser with WebGPU: the CPU (WebAssembly) path of @litert-lm/core 0.18.0 cannot read the EmbeddingGemma 2 bundles' external weights, so without WebGPU the embedder's load fails with an EmbeddingLoadException that says so — there is no CPU fallback.

flutter_edge_ai_onnx (ONNX models on Web): generation runs on Transformers.js v4, embeddings on onnxruntime-web. Both are readiness-handshake shims, same shape as the litertLmReady promise above:

<script type="module">
window.transformersReady = (async () => {
  const m = await import('https://cdn.jsdelivr.net/npm/@huggingface/transformers@4.3.1');
  window.transformers = m;
  return m;
})();
</script>

<script type="module">
window.ortReady = (async () => {
  const m = await import('https://cdn.jsdelivr.net/npm/onnxruntime-web@1.30.0/dist/ort.bundle.min.mjs');
  m.env.wasm.wasmPaths = 'https://cdn.jsdelivr.net/npm/onnxruntime-web@1.30.0/dist/';
  window.ort = m;
  return m;
})();
</script>

Only add the shim(s) for the arm(s) you use — transformersReady for OnnxEngine, ortReady for OnnxEmbeddingBackend.

LiteRtEmbeddingBackend (web embeddings, flutter_edge_ai_litertlm): runs on LiteRT.js, which needs the four files in flutter_edge_ai_litertlm's web/ copied into your app's web/: litert_embeddings.js, sentencepiece.js, litert.js and tensorflow.js. The first imports the other three by relative path — they are one bundle in four pieces — so they sit together and the entry module is loaded locally:

<script type="module" src="litert_embeddings.js"></script>

Copy the four files again after every flutter_edge_ai_litertlm upgrade: 1.13.0 changed the bundle's API, and a copy from an earlier version fails getActiveEmbedder() with an EmbeddingLoadException that names litert_embeddings.js. These four files serve .tflite embedding models; EmbeddingGemma 2 .litertlm embedders (flutter_edge_ai_litertlm 1.13.1+) need none of them, only the window.EmbeddingEngine line in the @litert-lm/core script above.

The WASM runtime underneath comes from a pinned CDN copy by default (flutter_edge_ai_litertlm) — nothing else to install. To serve it yourself, copy node_modules/@litertjs/core/wasm/ into web/wasm/ and set LiteRtWebRuntime.wasmPath = '/wasm/'; before getActiveEmbedder(), which loads the runtime. See flutter_edge_ai_litertlm's embeddings on web.

flutter_edge_ai_sqlite (web RAG): no <script>. Copy the package's web/rag/sqlite3.wasm (a sqlite3.wasm with sqlite-vec statically linked) into your app's web root as rag/sqlite3.wasm, and serve the app with the cross-origin isolation headers OPFS persistence needs:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

Model compatibility: mobile .task models often don't work on web, and native .litertlm models never do — use the -web.task (MediaPipe) or -web.litertlm (LiteRT-LM) web variant. Check the model repo for web-compatible builds.

Desktop (macOS, Windows, Linux)#

Desktop is served primarily by flutter_edge_ai_litertlm (.litertlm files), whose native library is fetched at build time by the package's Native-Assets hook (no manual download/bundling). flutter_edge_ai_onnx (ONNX Runtime) and flutter_edge_ai_llama (llama.cpp, .gguf) also run on all three desktop OSes (macOS/Windows/Linux), and the OS built-in model is available via flutter_edge_ai_builtin_ai on macOS (Apple Foundation Models) and on Windows (AI Foundry — nothing to configure to build: flutter_local_ai resolves the Windows App SDK projection itself; running needs Windows 11 25H2+ on Copilot+-class hardware and a packaged app); not on Linux. What holds across all of desktop: there is no MediaPipe engine on desktop — .task / .bin models are NOT compatible with desktop.

macOS: a deployment target of 14.0. With Swift Package Manager on (Flutter's default) that is all, from flutter_edge_ai_litertlm 1.19.0. With it off (enable-swift-package-manager: false), replace the post_install block of macos/Podfile — see Desktop → macOS. Without it the build succeeds and the first model load fails.

See Desktop Support for the full per-platform reference (macOS companion libraries, entitlements, Windows DLL loading — no VC++ redistributable needed — Linux Vulkan driver, and known limitations).

Platform & architecture support#

flutter_edge_ai_litertlm, flutter_edge_ai_speech and the LiteRT embedders ship prebuilts only for the architectures below. Other ABIs fail at native load with a typed error.

PlatformSupported architectureNot supported
Android arm64-v8a (full) armeabi-v7a, x86_64 ¹
iOS devicearm64—
iOS Simulator arm64 (Apple Silicon Mac) x86_64 (Intel Mac)
macOSarm64 (Apple Silicon)x86_64 (Intel Mac)
Linuxx86_64, arm64—
Windowsx86_64arm64

¹ MediaPipe text inference also works on Android x86_64 / armeabi-v7a (see the Android section above).

flutter_edge_ai_onnx also runs on Windows ARM64 (and has no Linux arm64 build), and flutter_edge_ai_llama follows llamadart's targets (its Android build includes x86_64) — see ONNX and llama.cpp.

For development, prefer an Apple Silicon Mac — the Android emulator runs arm64-v8a natively, and macOS / iOS Simulator builds are arm64.

HuggingFace authentication#

Many models require authentication to download from HuggingFace. Never commit tokens to version control.

Create a config template config.json.example:

{
  "HUGGINGFACE_TOKEN": ""
}

Copy it and add your token from huggingface.co/settings/tokens:

cp config.json.example config.json

Add config.json to .gitignore, then run with the config:

flutter run --dart-define-from-file=config.json

Access it in code:

void main() async {
  WidgetsFlutterBinding.ensureInitialized();

  const token = String.fromEnvironment('HUGGINGFACE_TOKEN');

  await FlutterEdgeAi.initialize(
    huggingFaceToken: token.isNotEmpty ? token : null,
    // plus inferenceEngines / embeddingBackends from step 2
  );

  runApp(MyApp());
}

Which models require authentication?#

Gated (auth required): Gemma3n (E2B, E4B), Gemma 3 1B, Gemma 3 270M, EmbeddingGemma 1 (.tflite). EmbeddingGemma 2 is public.

For every other model, see the HF token column on Models.

To use a gated repo: visit the model page → "Request Access" button.

Logging#

In release builds Flutter Edge AI's internal log prints the start-up notices about active models that could not be restored — model output, prompts, and conversation history are never written to logcat / syslog. In debug builds it's shown according to FlutterEdgeAi.logLevel:

LevelWhat it prints (debug builds)
EdgeAiLogLevel.none Nothing from this log, the restore notices included.
EdgeAiLogLevel.info (default) Lifecycle, errors, diagnostics. No model output / prompts.
EdgeAiLogLevel.verbose Everything above plus model output, prompts, and conversation history.
import 'package:flutter_edge_ai/flutter_edge_ai.dart';

// See the model's generated tokens and prompts while debugging:
FlutterEdgeAi.logLevel = EdgeAiLogLevel.verbose;

// Or silence this log, the restore notices included:
FlutterEdgeAi.logLevel = EdgeAiLogLevel.none;

Release builds keep one part of this log, the restore notices, and EdgeAiLogLevel.none silences them too; under WebStorageMode.none the notices about model files that are gone stay in the debug log, since no file outlives a page load there. Some messages are printed in every build whatever logLevel says: warnings about failures the app would not otherwise see, and a few notices (for example the sampler latch, at most once per engine, and the hidden-reasoning note, once per chat). The level is process-global and per-isolate; set it once at startup.