Models

flutter_edge_ai supports Gemma 4, Gemma3n, FastVLM, Qwen2-VL, SmolVLM2, LLaVA-OneVision, Gemma 3, FunctionGemma, Qwen3, Qwen 2.5, Phi-4 (incl. Phi-4 Mini Reasoning), DeepSeek R1, SmolLM, SmolLM3 and more. On desktop (macOS, Windows, Linux) LiteRT-LM models must be .litertlm — there is no MediaPipe engine there; ONNX models and the OS built-in model also run on desktop.

For a small custom model, LiteTune provides the easier end-to-end path from Hugging Face data/checkpoints through fine-tuning, .litertlm conversion, verification, and bundling. It is alpha software, so verify the converted model on the devices and backends you plan to ship.

Model file types#

Flutter Edge AI supports different model file formats, grouped into three types based on how chat templates are handled.

Type 1: SDK-managed templates#

• .task files: MediaPipe-optimized format for mobile (Android/iOS) and web.
• .litertlm files: LiteRT-LM format for Android, iOS, Desktop, and web (early preview). Never handled by MediaPipe — the two formats go to different engines.

Chat templates are applied by the runtime rather than by your code — MediaPipe for .task, LiteRT-LM for .litertlm — on every platform, iOS included.

Type 2: Manual template formatting#

• .bin files: standard binary format.
• .tflite files: LiteRT format (formerly TensorFlow Lite).

Both formats require manual chat template formatting in your code.

Type 3: System OS models (no file)#

Gemini Nano (Android, via AICore / ML Kit GenAI), Apple Foundation Models (iOS 26+/macOS), Windows AI Foundry / Phi Silica (Windows), and Gemini Nano via the browser Prompt API (Web — Gemini Nano in Chrome, Phi-4-mini in Edge) are built into the OS/browser — there is no model file to bundle or download; the platform owns the weights. Add flutter_edge_ai_builtin_ai, register BuiltInAiEngine(), and use ModelFileType.builtIn. Availability is device-gated — Gemini Nano needs Pixel 9+/Galaxy S25+, Apple FM needs Apple Intelligence enabled on iPhone 15 Pro+/M-series, and Windows needs Windows 11 25H2+ on Copilot+-class hardware and a packaged app (nothing to configure to build — flutter_local_ai resolves the Windows App SDK projection itself). On Android the package requires minSdk 26 and Kotlin 2.3.21.

👉 See Built-in AI for the full setup, the availability probe (BuiltInAi.availability() / BuiltInAi.ensureReady()), and fallback guidance.

Format by platform#

¹ iOS .litertlm runs on the FFI engine — vision and audio supported on physical devices. The Simulator stays CPU-only because Metal sim has a 256 MB single-allocation cap.

² Web .litertlm is an early preview via @litert-lm/core — text plus function calling. No vision, audio, the Gemma 4 thinking channel, or LoRA; Qwen3 reasoning tags are still parsed when the model emits them. See the feature matrix in Troubleshooting. For full multimodal on web, use a MediaPipe .task build.

³ On desktop the OS model exists on macOS (Apple Foundation Models) and Windows (AI Foundry; Windows 11 25H2+ on Copilot+-class hardware, packaged app) — not on Linux. Every platform still gates it at runtime: probe with BuiltInAi.availability().

Model capabilities#

¹ Gemma 4 Thinking Mode needs the native extraContext channel: Android, iOS and Desktop only. The measured Web .litertlm test receives no Gemma 4 ThinkingResponse; MediaPipe Web cannot enable it and ONNX Web ignores it. Qwen3 is different: core parses its emitted <think> tags on every platform, including Web.

² Gemma3n function calling: on the downloadable E4B .litertlm build; not on E2B or the MediaPipe .task builds.

‡ Reasons, but emits no ThinkingResponse. These models run as ModelType.general, which has no reasoning parser — their thinking blocks arrive inside the answer as ordinary text and are not stripped. See Thinking Mode.

ModelType reference#

When installing models, specify the correct ModelType:

Usage example:

// Gemma models
await FlutterEdgeAi.installModel(modelType: ModelType.gemmaIt)
  .fromNetwork(url).install();

// DeepSeek models
await FlutterEdgeAi.installModel(modelType: ModelType.deepSeek)
  .fromNetwork(url).install();

// Phi-4 (its own type — parses Phi's tool-call markers)
await FlutterEdgeAi.installModel(modelType: ModelType.phi)
  .fromNetwork(url).install();

Supported models & platforms#

Installation sources#

// Network — .litertlm is the cross-platform default (Android/iOS/Desktop).
// For mobile-only or web-only apps you can substitute a .task URL — and drop
// the fileType, which defaults to ModelFileType.task.
await FlutterEdgeAi.installModel(
  modelType: ModelType.gemmaIt,
  fileType: ModelFileType.litertlm,
)
  .fromNetwork('https://example.com/model.litertlm', token: 'optional')
  .install();

// Flutter assets
await FlutterEdgeAi.installModel(
  modelType: ModelType.gemmaIt,
  fileType: ModelFileType.litertlm,
)
  .fromAsset('assets/models/model.litertlm')
  .install();

// Native bundle
await FlutterEdgeAi.installModel(
  modelType: ModelType.gemmaIt,
  fileType: ModelFileType.litertlm,
)
  .fromBundled('model.litertlm')
  .install();

// External file (native only)
await FlutterEdgeAi.installModel(
  modelType: ModelType.gemmaIt,
  fileType: ModelFileType.litertlm,
)
  .fromFile('/path/to/model.litertlm')
  .install();

Hugging Face#

Install a model straight from a Hugging Face repo. The resolver that reads a repo's deployment manifest ships with the engine, so registering the engine is enough — there is no separate resolver list to maintain.

One call (manifest-driven) — omit file, and fromHuggingFace(repo) resolves the repo's manifest (e.g. litertlm_manifest.json) at install time, installs the right revision-pinned variant for the device, and returns the manifest's overridable runtime defaults on the result:

// The engine carries its resolver — nothing else to register.
await FlutterEdgeAi.initialize(inferenceEngines: [LiteRtLmEngine()]);

final install = await FlutterEdgeAi.installModel(
  modelType: ModelType.general,     // fallback — the manifest overrides it
  fileType: ModelFileType.litertlm, // selects the litertlm resolver
)
  .fromHuggingFace('litert-community/Qwen3-4B-Thinking-2507')
  .install();

final model = await FlutterEdgeAi.getActiveModel(defaults: install.runtime);
// `minOutputTokens` is a FLOOR, not a cap — leave `maxOutputTokens` unset (or
// keep it >= `install.runtime?.minOutputTokens`). Passing the floor as the cap
// would truncate a reasoning model mid-thought.
final session = await model.createSession(
  enableThinking: install.runtime?.isThinking ?? false,
);

.onnx repos install the whole ORT-GenAI directory — the ONNX resolver (carried by OnnxEngine) lists the repo, picks a CPU execution-provider folder (the bundled runtime is CPU-only; pin one with OnnxHuggingFaceResolver(variant: …)) and downloads every file in it; see ONNX. .builtIn repos surface that OS models have no Hugging Face file.

Explicit file — pass file, and fromHuggingFace(repo, file:) resolves …/resolve/<revision>/<file> and installs it directly, for any fileType (no manifest needed; the HF token is applied to huggingface.co automatically):

await FlutterEdgeAi.installModel(
  modelType: ModelType.gemmaIt,
  fileType: ModelFileType.litertlm,
)
  .fromHuggingFace('litert-community/Gemma3-1B-IT', file: 'model.litertlm', token: 'optional')
  .install();

Inspect first — resolveHuggingFace(repo, fileType:) returns the resolved identity plus overridable runtime defaults WITHOUT installing, so you can inspect the variant, its notes, and defaults before committing:

final r = await FlutterEdgeAi.resolveHuggingFace(
  'litert-community/Qwen3-4B-Thinking-2507',
  fileType: ModelFileType.litertlm,
);
// … inspect r.file / r.notes / r.runtime …
await FlutterEdgeAi.installModel(
  modelType: r.modelType ?? ModelType.general,
  fileType: r.fileType,
)
  .fromNetwork(r.url) // r.url pins the resolver's revision
  .install();

The resolver lives in the engine package — LitertlmManifestResolver in flutter_edge_ai_litertlm reads litertlm_manifest.json — and rides on the engine via HuggingFaceResolverSource, so initialize(inferenceEngines: […]) auto-registers it. Pass initialize(huggingFaceResolvers: [...]) only to override an engine's default (e.g. LitertlmManifestResolver(revision: 'abc123') to pin a commit).

Source capabilities#

Android foreground service (large downloads)#

Android has a 9-minute background execution limit. A foreground service shows a notification and exempts the download from battery-optimization kills, and you have to ask for it — the default configures no notification, and the platform will not start the service without one, so file size alone changes nothing:

// DEFAULT — no foreground service; pass foreground: true to get one
await FlutterEdgeAi.installModel(modelType: ModelType.gemmaIt)
  .fromNetwork(url)  // foreground: null
  .install();

// Force foreground mode
await FlutterEdgeAi.installModel(modelType: ModelType.gemmaIt)
  .fromNetwork(url, foreground: true)
  .install();

iOS uses native URLSession which handles long downloads automatically — no foreground service needed.

Cancelling downloads#

import 'package:flutter_edge_ai/flutter_edge_ai.dart'; // exports CancelToken

final cancelToken = CancelToken();

final future = FlutterEdgeAi.installModel(modelType: ModelType.gemmaIt)
  .fromNetwork(url)
  .withCancelToken(cancelToken)
  .withProgress((progress) => print('Progress: $progress%'))
  .install();

// Cancel from elsewhere (e.g. user pressed a cancel button)
cancelToken.cancel('User cancelled download');

try {
  await future;
} catch (e) {
  if (CancelToken.isCancel(e)) {
    print('Download was cancelled by user');
  }
}

CancelToken cancels all files in multi-file downloads (e.g. embedding model + tokenizer), works on mobile + web, and throws DownloadCancelledException.

Speech models#

On-device speech via the opt-in flutter_edge_ai_speech package — selectable, profile-driven pipelines (you pick the model with SttModelType / TtsModelType). moonshine-tiny, Whisper, and Parakeet (STT) and Matcha, Qwen3-TTS (multilingual), and Inflect-Nano-v2 (fast) TTS work end-to-end today; kokoro / supertonic TTS voices are follow-ons. Native only (no Web). See Speech for usage. A VoiceSession also chains STT → LLM → TTS into a push-to-talk voice loop — see Speech.

Speech-to-text

Whisper is multilingual — the shipped checkpoints are the multilingual ones (no .en suffix), so all 99 of Whisper's languages are available. Set a default for the recognizer, override it per transcription, or both:

// Default for every transcription on this recognizer.
final stt = await FlutterEdgeAi.getActiveStt(language: 'de');
final german = await stt.transcribe(germanPcm);

// One call in another language — same recognizer, nothing reloaded.
final french = await stt.transcribe(frenchPcm, language: 'fr');

Both are free: the language is one token in the decoder's seed prompt, and that prompt is rebuilt on every transcription. Switching languages never reloads the model or invalidates the recognizer you are holding.

The value is Whisper's own language code without the delimiters, and it defaults to 'en'. It decides the OUTPUT language only — the weights understand the audio either way, so asking for 'en' on German speech returns an English translation rather than an error.

Moonshine and Parakeet have no language token in their decoder prompt and reject the parameter with an ArgumentError rather than ignoring it; both are English-only.

Text-to-speech

Text embedding models#

The LiteRT embedding models below generate 768-dimensional vectors. The numbers in names (64/256/512/1024/2048) indicate maximum input sequence length in tokens, not embedding dimension. (With the ONNX backend the dimension is model-dependent — e.g. all-MiniLM-L6-v2 is 384-dim.) See Embeddings & RAG for usage.

Performance (Android Pixel 8):

• Gecko 64: ~109 ms/doc embedding, 130 ms search (fastest — 2.6× faster than EmbeddingGemma).
• EmbeddingGemma 256: ~286 ms/doc embedding, 342 ms search (more accurate — 300M vs 110M params).

SigLIP 2 text tower (ONNX, manual wiring)#

flutter_edge_ai_embeddings ships a SigLIP 2 text profile, for putting text into the same space as SigLIP's vision tower — image↔text retrieval rather than document RAG. It is the only embedding profile here that is not installed through installEmbedder().

Text in, vectors out — the plugin does not run the vision tower. You embed the image side elsewhere (or offline) and query it with vectors this profile produces.

Its convention differs from every model above: no leading BOS, a single trailing <eos>, lowercased text, and a fixed 64-token width that lives in the token ids because the int8 export carries no attention_mask. It also ignores the TaskType prefix — the vision tower encodes an image with no prefix, so adding one moves the text vector off the space the two towers share.

SigLIP 2 reuses the Gemma BPE vocabulary, so a tokenizer.json cannot be told apart by its vocabulary alone, and the ONNX tokenizer loader refuses such a file rather than reading it with Gemma's convention and returning a plausible but wrong vector.

To tell whether an export is the one this profile expects, look at two blocks of its tokenizer.json — the same two the refusal keys on:

• "padding" declares a fixed width — "strategy": {"Fixed": …}, not "BatchLongest"
• "post_processor" appends <eos> and prepends no <bos>

The refusal needs both, so there are three outcomes rather than two. A file matching both is SigLIP 2's convention. One that prepends <bos> is an EmbeddingGemma-family file and belongs on the profiles above. A file failing either check — no padding block, "BatchLongest", or a post_processor that is missing, empty, or does not end in a special token — is not classified at all: it is read with Gemma's convention and nothing is raised.

So do not read the absence of an error as approval. A SigLIP 2 export that declares both blocks is caught, but one that dropped either — a re-export, or a tool that strips them — reaches the Gemma path and produces exactly the silently wrong vector this section warns about.

Wire it yourself:

import 'package:flutter_edge_ai_embeddings/embedding_tokenizer.dart'
    show loadSiglipSentencePieceEmbeddingTokenizer;

and pass that as the tokenizer factory of the ForwardPassDescriptor you give to CommonEmbeddingModel.create. That library is native-only. See the flutter_edge_ai_embeddings README for the full profile, and ONNX Runtime for why the factory declines to guess.