LogoFlutter Edge AI

Migration

Upgrade to Flutter Edge AI 2.7 (Hugging Face resolvers per role, the litertlm_dart runtime and its user_defines key), 2.6 (engine session/chat/turn SPI, decider SPI on InferenceModel, typed responses and failed-turn rollback), 2.5 (openModel and embedding providers), and earlier releases.

flutter_edge_ai 2.7.0: Hugging Face resolvers per role#

  • A HuggingFaceResolver implementation adds the role it serves: @override ModelManagementType get type => ModelManagementType.inference; for a language-model resolver (embedding, stt or tts for the others). initialize throws StateError for a provider whose HuggingFaceResolverSource resolver serves another role than the list it is passed in.
  • ResolvedHfModel.modelType is family, a ModelFamily (ModelType, SttModelType or TtsModelType). A resolver passes family:; an app that read the language-model family casts it: r.family as ModelType? ?? ModelType.general. The installer of installModel throws StateError for a family that is not a ModelType.
  • resolveHuggingFace takes type: (default inference); apps that call it for a language model change nothing.
  • getActiveEmbedder's supportImage and supportAudio are bool?: an omitted argument takes the defaults: value, then false as before. A subclass of EmbedderVariant, SttVariant or TtsVariant inherits a runtimeDefaults it may override.
  • A language-model directory install (installModel(...).fromHuggingFace) checks each member's size when the resolver gives one, as catalog and embedder installs do, and then also refuses to write over the files of the active model: close and uninstallModel the active model before installing another revision into the same directory. The ONNX resolver gives no sizes yet, so its installs are unchanged.

flutter_edge_ai_litertlm 1.21.0: the LiteRT-LM runtime is litertlm_dart#

dependencies:
  flutter_edge_ai: ^2.7.0
  flutter_edge_ai_litertlm: ^1.21.0   # requires flutter_edge_ai 2.7.x

flutter_edge_ai_litertlm 1.21.0 accepts core 2.7.x only, and flutter_edge_ai_onnx 0.10.0 and flutter_edge_ai_builtin_ai 0.4.3 also require core 2.7.0: upgrade them together.

The native library, its build hook and the Dart runtime under the engine moved to a new pure-Dart package, litertlm_dart, which flutter_edge_ai_litertlm depends on.

  • NPU flags move to the litertlm_dart: key. qualcomm_npu, google_tensor_npu, mediatek_npu and their settings (qualcomm_npu_maven_url / qualcomm_npu_aar / qualcomm_npu_qairt_zip, mediatek_npu_maven_url / mediatek_npu_aar) now go under hooks: user_defines: litertlm_dart: (the workspace root's pubspec, if the app is a workspace member):

    hooks:
      user_defines:
        litertlm_dart:
          qualcomm_npu: true
    

    1.21.0 still reads them under flutter_edge_ai_litertlm: and prints a warning naming each one; a key set under both takes the litertlm_dart: value. The old key is to be dropped with the next upstream LiteRT-LM version.

  • The native cache moved. The hook caches each release in its own directory, litertlm_dart/native/<release tag>/<platform> (native-v0.19.1-c for 1.21.0) under ~/Library/Caches (macOS), ~/.cache (Linux) or %LOCALAPPDATA% (Windows). To force a fresh download, delete litertlm_dart/native there and run flutter clean. The macOS Podfile post_install block still reads ~/Library/Caches/flutter_gemma/native/macos_arm64, which the hook keeps filled; keep the block as it is.

  • flutter_edge_ai_speech 0.6.2 needs flutter_edge_ai_litertlm 1.21.0. An older flutter_edge_ai_litertlm beside litertlm_dart would bundle libLiteRtLm a second time.

flutter_edge_ai 2.6.0: engine SPI#

Engine packages continue to import package:flutter_edge_ai/engine.dart. Core now owns openSession, openChat, sessions and both InferenceChat generation methods (@nonVirtual). Move an openSession override to openEngineSession(SessionRequest), an openChat override's defaults and validation to resolveChatRequest(ChatRequest), per-chat turn behavior to createChatTurnHooks(ResolvedChatRequest), and a sessions override to engineSessions.

openEngineSession serves both direct sessions and every initial or rebuilt chat session. If SessionRequest.responseFormat is non-null, core has accepted it and the engine must enforce it during every generation. resolveChatRequest fills only values the caller left unset, preserves caller values, and carries responseFormat over unchanged; core throws StateError if it changes.

A session that can report turn endings overrides startTurn() and returns EngineTurn(text:, end:); end never fails and completes after the text, including cancellation. Use end: null only when the engine cannot report a reliable end, and then keep StructuredOutputSupport.none. On core's turn path a failed turn commits nothing to core history, and the engine must mirror that take-back in its own context. Restore the engine context after any text error and after a stop that emitted no non-empty text; a refused too-long input is ContextLimitException with TurnEnd.contextLimit. A stop after text, outputLimit, and a decode that reaches contextLimit keep the partial exchange.

ChatTurnHooks.responses replaces the whole plain turn, including generation, parsing, history, token accounting, take-back and overflow rebuilds. Core never calls it for a structured chat; those turns always use startTurn and core's validation path.

implements InferenceModel#

An implementation adds openEngineSession, resolveChatRequest, createChatTurnHooks, engineSessions, structuredOutputSupport, refusalFor, engineStructuredOutputSupport, engineRefusalFor, readDecisionSection and openOptionScorer, and updates openSession/openChat for ResponseFormat? responseFormat. If it does not support independent sessions, option scoring or formats, implement the same refusing behavior as InferenceModel's defaults. A class that extends InferenceModel inherits those defaults. The removed OptionScoringModel mixin becomes overrides of readDecisionSection and openOptionScorer on the model.

implements InferenceModelSession#

Add startTurn(). If the engine cannot report a reliable ending, return EngineTurn(text: getResponseAsync(), end: null); a class that extends InferenceModelSession inherits that default.

flutter_edge_ai 2.6.0: typed responses#

openSession and openChat accept responseFormat:; createSession and createChat do not. Keep and close the independent session/chat returned by open*. Before opening it, inspect model.structuredOutputSupport and model.refusalFor(format), because support is specific to the loaded engine, platform and backend.

ResponseFormatUnsupportedException means the format was refused. If a structured generation's text closes normally with TurnEnd.outputLimit, contextLimit, stopped or closed, IncompleteStructuredOutputException carries that end and the partial text; a refused input that never fit the context fails with ContextLimitException instead. NonConformingStructuredOutputException means an engine accepted a format but ended normally with text that was not exactly one matching JSON value. Incremental engines emit their text before the final validation error; buffered engines emit one complete chunk. A format cannot be combined with tools or enableThinking: true.

Open sessions and chats#

createSession, createChat, session and chat are deprecated and will be removed in 3.0.0. Keep the handle returned by openSession or openChat, and close it when done:

final session = await model.openSession();
try {
  await session.addQueryChunk(Message(text: prompt, isUser: true));
  final reply = await session.getResponse();
} finally {
  await session.close();
}

Stuck on core 2.5.x after upgrading?#

The old package lines below require flutter_edge_ai <2.6.0: litertlm 1.19.x, mediapipe 1.2.x, builtin_ai 0.3.x, ONNX 0.8.x, llama 0.0.x, speech 0.5.x and genkit_flutter_edge_ai 0.11.x. Upgrade every installed package in this compatibility set with core 2.6.0; the 2.6-compatible lines are litertlm 1.20.0, mediapipe 1.3.0, builtin_ai 0.4.0, ONNX 0.9.0, llama 0.1.0, speech 0.6.0 and genkit_flutter_edge_ai 0.12.0.

flutter_edge_ai 2.5.0: several open models, removed and deprecated members, embedding providers#

Removed members#

RemovedUse
EmbeddingModel.generateEmbedding(t) embed(EmbeddingInput.text(t), taskType: TaskType.retrievalQuery)
EmbeddingModel.generateEmbeddings(ts) embedAll([for (final t in ts) EmbeddingInput.text(t)], taskType: TaskType.retrievalQuery)
EmbeddingModel.getDimension()dimension
InferenceModelSpec.fromLegacyUrl(name:, modelUrl:, loraUrl:) InferenceModelSpec(name: …, modelSource: …, loraSource: …, modelType: …)

The removed embedding methods embedded as TaskType.retrievalQuery unless given a taskType:; pass the same task to keep their vectors. genkit_flutter_edge_ai 0.9.0 and older call generateEmbeddings and do not compile against 2.5.0: upgrade it to 0.9.1 or later. For fromLegacyUrl, use the ModelSource that matches the URL: ModelSource.network(url) for http(s)://, ModelSource.asset(path) for asset:// or a bare path, ModelSource.bundled(name) for native://, ModelSource.file(path) for file://. The factory defaulted to modelType: ModelType.general.

FlutterEdgeAiPlugin.createModel is deprecated#

FlutterEdgeAiPlugin.instance.createModel(modelType:, fileType:, …) is deprecated and removed in 3.0.0. On Android, iOS and desktop it ignored modelType and fileType and returned the active model, which is what FlutterEdgeAi.getActiveModel() returns. The two ways to get a model are on FlutterEdgeAi:

BeforeAfter
FlutterEdgeAiPlugin.instance.createModel(modelType: …, maxTokens: …) FlutterEdgeAi.getActiveModel(maxTokens: …)
several models through src/ imports FlutterEdgeAi.openModel(spec, …) on a spec from install(activate: false) , with LiteRtLmEngine ( flutter_edge_ai_litertlm 1.16.0)

Embedding providers: .gguf without a tokenizer, accelerators#

Embedding specs accept .gguf without a separate tokenizer. For an opaque URL, use modelFromNetwork(url, filename: 'model.gguf'). Old .gguf plus tokenizer specs are now refused during installation and stored-identity restoration; no published backend produced those pairs. Other model/tokenizer pairs and existing .litertlm identity schemas keep their behavior. No GGUF embedding backend ships yet; a separate engine package is required to execute these files.

Follow-on: flutter_edge_ai_llama 0.0.3 adds the opt-in LlamaEmbeddingBackend for native EmbeddingGemma 2 GGUF text embeddings using the core 2.5.0 embedding SPI and llamadart 0.15.1. Register it through FlutterEdgeAi.initialize(embeddingBackends:) and install the GGUF alone. Keep a separate RAG vector profile from LiteRT; matching dimensions are not vector compatibility. flutter_edge_ai_embeddings remains tokenizer-only. The statement above describes availability at the core 2.5.0 release.

Engine authors declare explicit accelerator capabilities through EmbeddingBackendProvider.honouredBackends(spec); core computes cache identity. Automatic requests reuse only a non-NPU embedder for the same installed model and media; explicit NPU requests always retain a separate key. Repeated explicit CPU/GPU requests reuse their originating model, including a fallback, until a rebuild is needed or the model is closed. A fallback warning is printed once per new load, including reporting-provider NPU fallbacks. GPU-capable providers must report the loaded accelerator in EmbeddingPassInfo.activeBackend. Existing providers using the default capability set retain descriptor fallback, including the fixed WebGPU-only LiteRT-LM path.

maxInputTokens is the effective limit accepted by the loaded pass, including context and single-pass batch limits, task prefixes and special tokens. A null limit means unknown, not unlimited. GGUF's configured 1024-token context and micro-batch can be smaller than a model file's theoretical limit; advertising that larger limit would let RAG choose chunks the runtime cannot execute. Existing providers' numeric limits do not change. Inputs are never truncated.

flutter_edge_ai 2.4.1 / flutter_edge_ai_litertlm 1.15.0: npu on a generic bundle runs on the GPU#

dependencies:
  flutter_edge_ai: ^2.4.1
  flutter_edge_ai_litertlm: ^1.15.0   # requires flutter_edge_ai 2.4.1
  • PreferredBackend.npu on a bundle compiled for no NPU (a generic one) is never tried on the NPU: where the NPU was reachable, LiteRT ran such a bundle on the CPU while activeBackend reported npu. It runs on the GPU, then the CPU, with a notice; with LiteRtLmEngine(backendFallback: false) it is refused with a LiteRtLmBundleException (wrongBackend). To run on the NPU, install the bundle compiled for it.
  • Google Tensor G5/G6 bundles load on the NPU of a Pixel with that Tensor generation and Android 16 or later (not a BP2A build); run so far on G6 only. It is opt-in, as Qualcomm's is: google_tensor_npu: true in the same user_defines block. Without it such a bundle fails with a BackendInitException that says how to enable it.

See where a model runs.

flutter_edge_ai_agent 0.4.0: schedule_notification is opt-in#

dependencies:
  flutter_edge_ai_agent: ^0.4.0

NativeIntentExecutor() no longer schedules notifications: the intent returns an error until the app passes NotificationIntentSettings. Before, it scheduled whether or not the app had called initialize, and without it the app crashed: on macOS when the intent scheduled, on Android when the reminder fired with no small icon. An app that uses the intent:

  1. calls FlutterLocalNotificationsPlugin().initialize(...) once at startup, with its own callbacks, its default icon and settings for every platform it builds for (initialize throws without them) — the agent never does;
  2. passes NativeIntentExecutor(notifications: const NotificationIntentSettings());
  3. on Android, declares SCHEDULE_EXACT_ALARM and, on Android 14 and newer, asks for it with requestExactAlarmsPermission(), or passes androidExactAlarm: false for an inexact reminder.

The intent checks the permissions and refuses instead of asking for them, and it no longer loads the time zone database, which reset the app's tz.local to UTC. A schedule_notification entry in handlers: still replaces the built-in intent, but not together with notifications:.

Every schedule_notification call, an app's own handler included, is now refused before it runs when its date is partial (a day without year and month) or does not exist (February 31). Without a date, a time that has passed today moves to tomorrow at the same wall-clock time, across a clock change too. The mail and SMS intents write a space as %20 instead of +. See Agent → Setup.

flutter_edge_ai 2.4.0 / flutter_edge_ai_litertlm 1.14.0: backends follow the model file#

Upgrade these together: flutter_edge_ai 2.4.0, flutter_edge_ai_litertlm 1.14.0, flutter_edge_ai_agent 0.3.0 and genkit_flutter_edge_ai 0.10.0. litertlm 1.13.x and genkit 0.9.x compile against core 2.4.0 but ignore multimodalOrder, activeVisionBackend / activeAudioBackend and tool call ids. The other packages are unchanged.

dependencies:
  flutter_edge_ai: ^2.4.0
  flutter_edge_ai_litertlm: ^1.14.0
  flutter_edge_ai_agent: ^0.3.0       # if you use it
  genkit_flutter_edge_ai: ^0.10.0     # if you use it

implements InferenceModel#

A class that implements InferenceModel declares three new members: createDecider (fail it with DeciderUnsupportedException), and the getters activeVisionBackend and activeAudioBackend (null when the model has no such encoder or the engine does not say). A class that extends InferenceModel inherits all three.

Where a .litertlm runs#

Before it creates an engine, litertlm 1.14.0 reads the bundle's header and decides from it (where a model runs). What can change for an existing app:

  • A bundle compiled for an NPU (e.g. …_qualcomm_sm8750.litertlm) with no preferredBackend now runs on the NPU. With PreferredBackend.gpu or cpu it is refused with a LiteRtLmBundleException (wrongBackend); on a device whose NPU cannot be reached it fails with a BackendInitException that names the cause. It is never tried on the GPU or the CPU. A generic bundle with npu still falls back to GPU, then CPU.
  • An encoder you leave unset runs where its section says, reported in activeVisionBackend / activeAudioBackend: Gemma 4 vision and audio on the CPU, Gemma 3n vision on the GPU and its audio on the CPU.
  • preferredAudioBackend: PreferredBackend.gpu on Gemma 4 or Gemma 3n is refused with a LiteRtLmBundleException (wrongBackend): their audio encoders declare backend_constraint cpu. Remove it.
  • preferredVisionBackend: PreferredBackend.cpu on Gemma 3n is refused the same way: its vision encoder declares gpu only. Remove it.
  • An encoder backend left unset and an explicit PreferredBackend.cpu are different requests now: switching between them reloads the model.
  • A download that stopped early is refused before any engine (LiteRtLmBundleException, incomplete): install the file again.

New and optional: LiteRtLmEngine(backendFallback: false) makes one attempt and fails with LiteRT-LM's own reason, and readLiteRtLmBundleInfo(path) shows what the engine reads.

Gemma 4 gets its audio after the text#

litertlm 1.14.0 sends a Gemma 4 message's images, then its text, then its audio (MultimodalOrder.imagesTextAudio, the order of the model card); every other model keeps images, audio, text. Pass getActiveModel(multimodalOrder: MultimodalOrder.imagesAudioText) to keep the old order.

flutter_edge_ai_agent 0.3.0#

EmptyAnswerRetryEvent is a new AgentEvent, so an exhaustive switch over AgentEvent needs a case for it: the loop emits it when it asks again after an empty answer that followed loadSkill (emptyAfterSkillRetries, default 1). A turn that stays empty ends with EmptyAnswerEvent, a DoneEvent.

genkit_flutter_edge_ai 0.10.0#

Requires flutter_edge_ai ^2.4.0. Tool refs travel as call ids both ways: a tool request's ref is the call's id (call_0, call_1, … when it has none), and a tool response's ref becomes the toolCallId of the message it is sent back as.

flutter_edge_ai 2.3.0: the embedding API, EmbeddingGemma 2, RAG 1.1#

Upgrade every flutter_edge_ai package together (flutter pub upgrade, not flutter pub upgrade flutter_edge_ai). Minimum versions with core 2.3.0: flutter_edge_ai_litertlm 1.13.0 · flutter_edge_ai_embeddings 2.3.0 · flutter_edge_ai_onnx 0.5.8 · flutter_edge_ai_mediapipe 1.2.2 · flutter_edge_ai_builtin_ai 0.3.5 · flutter_edge_ai_speech 0.5.6 · flutter_edge_ai_agent 0.2.9 · flutter_edge_ai_rag 1.1.0 · flutter_edge_ai_sqlite 2.1.0 · flutter_edge_ai_qdrant 2.1.0. An older one fails to compile inside that package. genkit_flutter_edge_ai 0.9.0 still compiles; 0.9.1 calls the new API and re-resolves a closed embedder. flutter_edge_ai_diagnostics and genkit_hybrid are unaffected.

dependencies:
  flutter_edge_ai: ^2.3.0
  flutter_edge_ai_litertlm: ^1.13.0
  flutter_edge_ai_embeddings: ^2.3.0 # EmbeddingGemma 1 and other two-file models
  flutter_edge_ai_rag: ^1.1.0
  flutter_edge_ai_sqlite: ^2.1.0     # or flutter_edge_ai_qdrant: ^2.1.0

Embeddings#

EmbeddingModel takes an EmbeddingInput and a required TaskType. The three old methods stayed as deprecated forwarders until 2.5.0 removed them.

Before (2.2)After (2.3)
generateEmbedding(t) embed(EmbeddingInput.text(t), taskType: TaskType.retrievalQuery) . The task is required; TaskType.none embeds the text with no prefix.
generateEmbeddings(ts) embedAll([for (final t in ts) EmbeddingInput.text(t)], taskType: …) ; it also takes a cancelToken: .
await m.getDimension()m.dimension
a closed model threw StateError synchronously EmbedderClosedException ; every embedding error arrives as a failed future. Catch EmbeddingException for all of them.
inputs longer than the model's sequence length were truncated without a word (EmbeddingGemma 1 .tflite , ONNX static graphs, LiteRT.js on web, SigLIP) EmbeddingInputTooLongException(maxInputTokens, actualTokens) . Chunk your text below model.maxInputTokens — task prefix, BOS and EOS included.
TaskType { retrievalQuery, retrievalDocument } seven values appended: none , questionAnswering , factChecking , codeRetrieval , classification , clustering , sentenceSimilarity . Add the cases or a default branch to exhaustive switches; the first two keep their index.
getActiveEmbedder({preferredBackend}) adds supportImage and supportAudio : declare media at load. Media sent to a model that did not declare them throws.
FlutterEdgeAiPlugin.createEmbeddingModel(modelPath:, tokenizerPath:) the explicit paths are gone. Install the files with installEmbedder().modelFromFile(…) (and tokenizerFromFile(…) unless it is a .litertlm ), then call getActiveEmbedder() .
EmbeddingModelSpec(tokenizerSource:) was required tokenizerSource is null for a .litertlm embedder; check spec.isSelfContained . spec.files has one entry then.
installEmbedder() without a tokenizer always threw a .litertlm installs alone; a tokenizer for a .litertlm is an ArgumentError before any download, and, once an embedding backend is registered, a model none of them can run on this platform is an UnsupportedError before any download (with none registered the install goes ahead).

New and additive: EmbeddingInput.image/.audio, EmbeddingModality, EncodedMediaFormat, sniffEncodedMediaFormat, EmbeddingModel.modalities, maxInputTokens and the EmbeddingException family (EmbeddingLoadException, EmbeddingEngineException, …).

Test fakes and mocks: EmbeddingModel is an abstract base class now. A fake is final class FakeEmbedder extends EmbeddingModel (add with CloseNotifier from package:flutter_edge_ai/engine.dart for close listeners) implementing dimension, isClosed, embed, addCloseListener and close. implements EmbeddingModel, and therefore a mockito or mocktail mock of it, no longer compiles, and isClosed is no longer defaulted.

EmbeddingGemma 2#

flutter_edge_ai_litertlm 1.13.0 runs EmbeddingGemma 2 text: one .litertlm file that carries its own tokenizer, so it installs without tokenizerFrom…() and needs no flutter_edge_ai_embeddings. Android, iOS, macOS, Linux and Windows, CPU, at most 1024 input tokens — keep chunks at 512 or fewer. Images, the Web (text and images) and audio arrive in litertlm 1.13.1, audio and combined inputs on the Web in 1.13.2. See Embeddings & RAG for the install snippet and its disk and memory cost.

Web: copy the new litert_embeddings.js#

The LiteRT.js embedding bundle changed its API. Copy web/litert_embeddings.js from flutter_edge_ai_litertlm 1.13.0 (with the three files next to it) into your app's web/. A stale copy fails getActiveEmbedder() with an EmbeddingLoadException that names the file. Web EmbeddingGemma 1 now gets the same task prefix and the same token ids as native for every TaskType. In 1.13.0 a .litertlm embedder was refused on web at install(); since 1.13.1 it runs on WebGPU, with window.EmbeddingEngine = m.EmbeddingEngine; in web/index.html (see Installation → Web).

Imports: flutter_edge_ai.dart for apps, engine.dart for engines#

Core's internals moved under lib/src, which is private. The public libraries are flutter_edge_ai.dart (apps), engine.dart (engine and tokenizer packages; it re-exports the app library), engine_io.dart, engine_web.dart and genai.dart.

BeforeAfter
package:flutter_edge_ai/core/… , /mobile/… , /web/… , /desktop/… , /flutter_edge_ai_interface.dart , /model_file_manager_interface.dart package:flutter_edge_ai/flutter_edge_ai.dart ; engine packages package:flutter_edge_ai/engine.dart
EmbeddingTokenizerRegistry , SkillExecutorRegistry , ForwardResult , EmbeddingOutputContract , EmbeddingTokenizer , TokenizedInput , EmbeddingTokenizerFactory , meanPoolAndNormalize from the app library from package:flutter_edge_ai/engine.dart
ModelThinkingFilter through package:flutter_edge_ai/core/extensions.dart the app library
EmbeddingForwardPass , ForwardPassDescriptor , EmbeddingForwardPassFactory , CommonEmbeddingModel.create , EmbeddingBackendProvider.createModel EmbeddingPass , EmbeddingPassInfo , EmbeddingPassDescriptor<C> , TokenEmbeddingPass (if you tokenize in Dart) and EmbeddingBackendProvider.describe() , which returns EmbeddingPassDescriptor(engineTag:, factory:, config:, activeBackend:) . Core builds, hosts and closes the model.
CommonEmbeddingModel.inputSequenceLength, .outputDimension EmbeddingModel.maxInputTokens, .dimension
class MyBackend implements EmbeddingBackendProvider final class MyBackend extends EmbeddingBackendProvider (an abstract base class )
flutter_edge_ai_embeddings re-exported core's seam types it exports only GemmaEmbeddingTokenizers ; import the seam types from engine.dart . A SigLIP input over 63 tokens throws instead of truncating.
onnx OnnxEmbeddingForwardPass OnnxEmbeddingPass with OnnxEmbeddingConfig ; it extends TokenEmbeddingPass
litertlm WebEmbeddingModel (web_embedding_model.dart) removed: FlutterEdgeAi.getActiveEmbedder()

Removed deprecated members#

RemovedUse
FlutterEdgeAi.initialize(enableWebCache:)webStorageMode:
ModelFileManager.ensureModelReady(filename, url) ensureModelReadyFromSpec(spec)
installModelFromAsset, installModelFromAssetWithProgress FlutterEdgeAi.installModel(…).fromAsset(path), with .withProgress(…)
setModelPathFlutterEdgeAi.installModel(…).fromFile(path)
setLoraWeightsPath FlutterEdgeAi.installModel(…).withLoraFromFile(path)
deleteLoraWeightsinstall the model again without LoRA
deleteCurrentModeldeleteModel(spec)
FunctionCallParser.isJsonStart, isJsonComplete isFunctionCallStart, isFunctionCallComplete(buffer, modelType: …)
StopTokenFilter nothing: stop tokens have not reached the output since 0.14.0
FileSystemService.getTargetPath getReadTargetPath or getWriteTargetPath
InferenceModelSpec.modelUrl, loraUrl modelSource, loraSource
EmbeddingModelSpec.modelUrl, tokenizerUrl modelSource, tokenizerSource
EmbeddingModelSpec.fromLegacyUrl EmbeddingModelSpec(name: …, modelSource: ModelSource.network(url), tokenizerSource: ModelSource.network(tokenizerUrl))
MobileModelManager / WebModelManager.currentActiveModel , SmartDownloader.isHuggingFaceUrl these classes are internal now; read FlutterEdgeAi.activeModelSpec / activeEmbedderSpec

InferenceModelSpec.fromLegacyUrl was deprecated here and removed in 2.5.0. Build the spec with the ModelSource that matches the URL: ModelSource.network(url) for http(s)://, ModelSource.asset(path) for asset:// or a bare path, ModelSource.bundled(name) for native://, ModelSource.file(path) for file://. The factory defaulted to modelType: ModelType.general; pass that to keep its behaviour.

RAG: flutter_edge_ai_rag 1.1, flutter_edge_ai_sqlite and flutter_edge_ai_qdrant 2.1#

A location now binds one embedding profile and one document modality (VectorStoreSpec.documents: text, image or audio).

rag 1.0rag 1.1
open(spec:, activeEmbedderProfileId: id), profile bound on the first text operation open(spec:, embedder: FlutterEdgeAiActiveEmbedder(profileId: id)) — or .forActiveModel(namespace:) — resolves the profile (loading the model) and binds the location at once: install the embedder before open()
open(spec:, embeddingProfile: p) for precomputed vectors openVectors(spec:, profile: p)
index.addText(id:, content: c) index.add(id:, document: EmbeddingInput.text(c))
index.searchText(query: q) index.search(query: EmbeddingInput.text(q)) (rag 1.2.0's searchText(text:) is a different call: keyword search over an index opened with enableTextSearch , see Text and hybrid search )
index.embeddingProfile could be null never null; index.documents names the modality
RagEmbedder { profile, embedDocument(String), embedQuery(String) } { resolveProfile(), modalities, embedDocument(EmbeddingInput), embedQuery(EmbeddingInput, documentModality:) } : wrap strings in EmbeddingInput.text , return {EmbeddingModality.text} from modalities
a filter on an undeclared field was ignored ArgumentError : declare every filtered field in VectorStoreSpec.filterSchema
VectorStoreRepository ( implements ) with readEmbeddingProfile / bindEmbeddingProfile abstract base class ( extends ) with readBinding() / bind(VectorStoreBinding) ; the rag README describes how a store persists the binding
EmbeddingProfile ids could contain control characters U+0000–U+001F are refused, and EmbeddingProfile can no longer be implemented

flutter_edge_ai_rag re-exports EmbeddingInput, EmbeddingModality, TaskType and the EmbeddingException family, and every error of a Future-returning RagIndex or FlutterEdgeAiRag method arrives as a failed future.

Stores written by sqlite/qdrant 2.0.x open unchanged in 2.1.0 as text locations, and text locations move both ways. A location bound to images or audio cannot be opened by sqlite/qdrant 2.0.x.

flutter_edge_ai 2.2.0: sampling defaults come from the model#

createSession, openSession, createChat and openChat no longer default to temperature: 0.8, randomSeed: 1, topK: 1, which was greedy decoding for every model. The four parameters are nullable, and an unset one comes from:

  1. the sampler the model file ships — a native .litertlm bundle's, or an ONNX model's genai_config.json;
  2. the defaults its family publishes, SamplingParams.forModelType (Gemma: temperature: 1.0, topK: 64, topP: 0.95);
  3. temperature: 0.8, topK: 40, topP: 0.95, randomSeed: 1.

Output that was greedy is now sampled on LiteRT-LM and MediaPipe, except for the families whose publishers decode greedily (Phi, Hammer). ONNX leaves unset values to the model's config, on web too, where it used to decode greedily whatever the config said; built-in AI keeps temperature: 0.8, topK: 1. Pass topK: 1 or temperature: 0 to keep it greedy. Values you pass now also reach ONNX and web .litertlm, which used to drop them, and a value no engine can sample with (a negative temperature, topK below 1, topP outside (0, 1]) throws an ArgumentError. .litertlm keeps the sampler of an engine's first generation for every later conversation (google-ai-edge/LiteRT-LM#2080): close and reload the model to change it. genkit_flutter_edge_ai 0.8.1 and flutter_edge_ai_agent 0.2.8 pass unset values through the same way. What each engine reads, and what it ignores, is under Sampling.

Upgrade every flutter_edge_ai_* package together (flutter pub upgrade, not flutter pub upgrade flutter_edge_ai alone): earlier engine releases override the old signature and do not compile against core 2.2.0.

If you implement InferenceModel yourself, declare the parameters as double? temperature, int? randomSeed, int? topK, double? topP and fill unset ones with SamplingParams.resolve.

flutter_edge_ai 2.1.0: isThinking is enableThinking#

A rename with no dart fix rule, so the compiler finds every call site:

  • createSession, openSession, createChat and openChat take enableThinking: instead of isThinking:.
  • ModelRuntimeDefaults.isThinking is thinkingDeclared — what the model's manifest declares, which you pass on as enableThinking: r.runtime.thinkingDeclared ?? false.
  • genkit_flutter_edge_ai 0.7.1: the isThinking model option is enableThinking.

Requires flutter_edge_ai_litertlm 1.9.0 or later, which sends the flag to the runtime under the key chat templates read.

flutter_edge_ai_litertlm 1.10.0: Android NPU is opt-in#

Only apps that use PreferredBackend.npu on Android need to act. Qualcomm licenses its QNN runtime for redistribution inside an application only, so the package no longer ships it and every other Android app is about 83 MB smaller on the device. An app that wants the Qualcomm NPU asks for it in its pubspec.yaml (the workspace root's, if the app is a pub workspace member):

hooks:
  user_defines:
    litertlm_dart:
      qualcomm_npu: true

The key shown is the one flutter_edge_ai_litertlm 1.21.0 and later read; from 1.10.0 to 1.20.x it is flutter_edge_ai_litertlm:. The build hook downloads com.qualcomm.qti:qnn-runtime from Maven Central once per machine and caches it; offline builds set qualcomm_npu_maven_url to a mirror or qualcomm_npu_aar to the AAR itself. Setting the flag accepts Qualcomm's AI Stack License. It needs flutter_edge_ai 2.1.1. Without the flag a generic bundle still loads: npu falls back to GPU, then CPU. A bundle compiled for an NPU cannot run anywhere else, so from litertlm 1.14.0 its load fails instead, with a BackendInitException whose one attempt says the stack is not enabled. See LiteRT-LM.

Linux arm64 (flutter_edge_ai_litertlm 1.11.0). The same flag also bundles the Qualcomm NPU stack into Linux arm64 builds — a Qualcomm Linux board with a Hexagon V68–V81 DSP; Google publishes bundles for the QCS8275 (Arduino VENTUNO Q) only. The hook reads the QNN runtime out of Qualcomm's public QAIRT SDK zip (about 32 MB of it, by range request); qualcomm_npu_qairt_zip points it at a local copy. On the board the user must be in group fastrpc. A Linux x64 build ignores the flag.

Flutter Edge AI 1.x → 2.0: RAG leaves core#

Flutter Edge AI 2.0 keeps inference, embeddings, speech, installation, and model lifecycle in flutter_edge_ai, but moves RAG orchestration and all vector-store contracts to flutter_edge_ai_rag. RAG is instance-scoped and can use the active core embedder, a custom embedder, or precomputed vectors without initializing core. The snippets below use the current RAG API (flutter_edge_ai_rag 1.1, see the 2.3.0 section above).

dependencies:
  flutter_edge_ai: ^2.3.0
  flutter_edge_ai_rag: ^1.1.0
  flutter_edge_ai_sqlite: ^2.1.0 # or flutter_edge_ai_qdrant: ^2.1.0

Also breaking in 2.0:

  • Filter, FilterSchema, RetrievalResult, VectorStoreRepository and the other RAG types now import from package:flutter_edge_ai_rag/flutter_edge_ai_rag.dart, not from core.
  • ModelFileManager.setActiveModel is removed — use ensureModelReadyFromSpec.
  • The deprecated flutter_gemma name aliases (FlutterGemma, GemmaLogLevel, …) are removed. dart fix --apply still renames them.

Remove vectorStore: and filterSchema: from FlutterEdgeAi.initialize(). Register only AI runtimes there, then create and own RAG independently:

await FlutterEdgeAi.initialize(
  inferenceEngines: const [LiteRtLmEngine()],
  embeddingBackends: const [LiteRtEmbeddingBackend()],
  embeddingTokenizers: const [GemmaEmbeddingTokenizers()],
);

final rag = FlutterEdgeAiRag(
  providers: const [SqliteVectorStoreProvider()],
);
final index = await rag.open(
  spec: VectorStoreSpec(
    providerId: SqliteVectorStoreProvider.providerId,
    location: databasePath,
    filterSchema: FilterSchema(fields: [
      FilterField(name: 'category', type: FilterFieldType.string),
    ]),
  ),
  embedder: FlutterEdgeAiActiveEmbedder(
    profileId: 'embeddinggemma-300m-seq256-mp-rev-29888fcee321-'
        'retrieval-prefix-meanpool-l2-v1',
  ),
);

try {
  await index.add(id: 'doc-1', document: EmbeddingInput.text('Text to retrieve'));
  final hits = await index.search(
    query: EmbeddingInput.text('What should I retrieve?'),
  );
  await index.flush();
  print(hits);
} finally {
  await index.dispose();
}
1.x core RAGRagIndex
FlutterEdgeAi.rag.initialize(location) FlutterEdgeAiRag(...).open(spec: VectorStoreSpec(location: ...), embedder: ...)
addDocument(...) add(id: ..., document: EmbeddingInput.text(...))
addDocumentWithEmbedding(...)addVector(...)
searchSimilar(query: ...) search(query: EmbeddingInput.text(...))
low-level vector searchsearchVector(embedding: ...)
removeDocument(id: ...)remove(id: ...)
stats() / flush() / clear() same methods on the owned index
core/plugin teardownRagIndex.dispose() before core/embedder teardown

Embedding profiles and persistent data#

Every persistent location is durably bound to an EmbeddingProfile. Its ID must version the weights, tokenizer, pooling, normalization, and document/query prefix contract. For precomputed vectors, open a vector-only index with openVectors(spec:, profile:); if later text searches borrow the active core embedder, open with embedder: FlutterEdgeAiActiveEmbedder(profileId:) instead — addVector works on that index too. Independent text RAG passes a custom RagEmbedder whose resolveProfile() reports that stable identity.

A nonempty 1.11 store has no profile metadata. Prefer a new, profile-versioned location and re-index. If you can independently attest the old model and preprocessing, open once with allowLegacyProfileAdoption: true on the VectorStoreSpec and the verified profile passed to openVectors(profile:); the provider checks the stored dimension before binding it.

SQLite filter fields are physical vec0 columns, so a schema change needs a new schema-versioned location and re-index. On Web, keep one app-owned index per location because SQLite holds an exclusive Web Lock. Dispose RAG indexes before FlutterEdgeAi.dispose() or before disposing a custom embedder.

Historical: flutter_gemma → flutter_edge_ai (1.11.4)#

The project is now Flutter Edge AI. Every package moved to a new name; the code, the platforms and the on-device data are the same.

BeforeAfter
flutter_gemmaflutter_edge_ai 1.11.4
flutter_gemma_litertlmflutter_edge_ai_litertlm 1.8.6
flutter_gemma_mediapipeflutter_edge_ai_mediapipe 1.0.8
flutter_gemma_embeddingsflutter_edge_ai_embeddings 2.2.1
flutter_gemma_rag_sqliteflutter_edge_ai_sqlite 1.4.0
flutter_gemma_rag_qdrantflutter_edge_ai_qdrant 1.3.2
flutter_gemma_speechflutter_edge_ai_speech 0.5.2
flutter_gemma_agentflutter_edge_ai_agent 0.2.6
flutter_gemma_builtin_aiflutter_edge_ai_builtin_ai 0.3.0
flutter_gemma_onnxflutter_edge_ai_onnx 0.5.1
flutter_gemma_diagnostics flutter_edge_ai_diagnostics 0.1.0
genkit_flutter_gemmagenkit_flutter_edge_ai 0.6.2

The old flutter_gemma* packages stay on pub.dev as they are, so an app that has not moved yet keeps working.

To move:

  1. Replace each flutter_gemma* dependency in pubspec.yaml with its new name and the version from the table.
  2. In your import lines, replace flutter_gemma_rag_ with flutter_edge_ai_ first (flutter_gemma_rag_sqlite → flutter_edge_ai_sqlite, flutter_gemma_rag_qdrant → flutter_edge_ai_qdrant), then flutter_gemma with flutter_edge_ai everywhere — in the package name and in the file name: package:flutter_gemma/flutter_gemma.dart becomes package:flutter_edge_ai/flutter_edge_ai.dart. A project-wide search and replace does it.
  3. Run dart fix --apply (Flutter 3.44 or newer). It renames FlutterGemma, FlutterGemmaPlugin, FlutterGemmaDesktop, GemmaLogLevel, FlutterGemmaDiagnostics and the genkit names to their new spellings. In 1.11.4 they still compiled as deprecated aliases; flutter_edge_ai 2.0.0, flutter_edge_ai_diagnostics 0.2.0 and genkit_flutter_edge_ai 0.7.0 drop the aliases, and dart fix --apply renames the old names all the same.

If you installed the agent skills, run dart run skills@ get --all again and delete the old flutter-gemma-* skill directories: they still teach the old names.

What does not change:

  • Installed models, the model directory and the Web cache stay where they are, so nothing downloads again.
  • Existing Qdrant and SQLite vector stores opened as before in 1.11.4. Moving onward to 2.0 requires the embedding-profile migration above.
  • The Android package dev.flutterberlin.*, the platform channels and the macOS post_install snippet in your Podfile are unchanged.

Genkit: model and embedder ids are now flutter-edge-ai/<name>, and the context-window middleware is registered as flutter-edge-ai-context-window. Code that uses flutterEdgeAi.model(...) and trimContext() picks this up; a hard-coded 'flutter-gemma/<name>' string has to change. The old Dart names were deprecated aliases in 0.6.2; 0.7.0 drops them, and dart fix --apply renames them.

Move every package at once: an app that keeps a flutter_gemma_X next to flutter_edge_ai_X gets the same native libraries and Android classes twice, and the build fails. An old satellite you did not move (say flutter_gemma_speech) pulls the old engine back in the same way.

flutter_edge_ai_sqlite needs Flutter 3.47. An app on Flutter 3.44 that uses the SQLite store upgrades Flutter first.

Historical: flutter_gemma 0.x → 1.0#

1.0 split the monolithic flutter_gemma plugin into a small core package plus opt-in packages, so your app only ships the native weight it actually uses. This is the only breaking change: you add the packages you need and one initialize(...) call. Model/session/chat/embedding APIs stayed compatible in that release; RAG later changed in 2.0 as documented above.

TL;DR#

  1. Add the opt-in packages for the formats/features you use (see table below).
  2. Call await FlutterEdgeAi.initialize(inferenceEngines: [...], ...) once in main(), passing the engines/backends from the packages you added.
  3. Then apply the 2.0–2.7 sections above: RAG moved to flutter_edge_ai_rag (2.0), isThinking became enableThinking (2.1), sampling defaults come from the model (2.2), the embedding API changed (2.3), backends follow the .litertlm file (2.4, litertlm 1.14 and 1.15), members were removed and deprecated (2.5), sessions and chats come from openSession/openChat (2.6), NPU flags go under litertlm_dart: (litertlm 1.21.0), and, if you use the agent, schedule_notification is opt-in (flutter_edge_ai_agent 0.4).

1. pubspec.yaml#

Before (0.16.x):

dependencies:
  flutter_gemma: ^0.16.3

After (current 2.7 set):

dependencies:
  flutter_edge_ai: ^2.7.0                 # core — always required
  flutter_edge_ai_litertlm: ^1.21.0       # add if you run .litertlm models (also provides LiteRtEmbeddingBackend)
  flutter_edge_ai_mediapipe: ^1.3.1       # add if you run .task / .bin models
  flutter_edge_ai_embeddings: ^2.3.3      # add if you compute embeddings with a tokenizer file (needs a backend, see above)
  flutter_edge_ai_rag: ^1.2.0             # add for on-device RAG (RagIndex) + one store below
  flutter_edge_ai_qdrant: ^2.1.1          # native on-device RAG store (qdrant)
  flutter_edge_ai_sqlite: ^2.2.0          # RAG store (sqlite-vec; all platforms incl. web) — needs Flutter 3.47

Pick by what you actually used in 0.16.x:

In 0.16.x you used…Add
.litertlm models (any desktop model included) flutter_edge_ai_litertlm
.task / .bin models flutter_edge_ai_mediapipe
generateEmbedding() / installEmbedder() flutter_edge_ai_embeddings + flutter_edge_ai_litertlm ( LiteRtEmbeddingBackend ). generateEmbedding is embed since 2.3.0
RAG (addDocument / searchSimilar), fastest on native flutter_edge_ai_rag + flutter_edge_ai_qdrant
RAG on web (or a portable store on any platform) flutter_edge_ai_rag + flutter_edge_ai_sqlite

Not sure which format your models are? In 0.16.x desktop was always .litertlm (flutter_edge_ai_litertlm). On mobile/web check the file extension you install. You can add both engine packages and let the registry route each model by its file type.

New opt-in packages since 1.2 (not migration targets from the 0.16.x monolith — they add new capabilities): flutter_edge_ai_agent (on-device agent skills — SKILL.md + tool-calling loop), flutter_edge_ai_builtin_ai (OS system models — Gemini Nano on Android and Web, Apple Foundation Models on iOS/macOS, Windows AI Foundry on Windows), flutter_edge_ai_speech (STT, TTS and a voice loop), flutter_edge_ai_diagnostics (memory a model costs, read from the OS), flutter_edge_ai_onnx (ONNX Runtime — ORT-GenAI text generation + plain-ORT embeddings via dart:ffi on native, + Web via Transformers.js / onnxruntime-web), and flutter_edge_ai_llama (GGUF chat models and EmbeddingGemma 2 GGUF text embeddings through llama.cpp, via llamadart; see llama.cpp). Add any of them only if you want that feature. See Getting Started.

Breaking: embeddings 2.0.0 — LiteRtEmbeddingBackend moved#

flutter_gemma_embeddings 2.0.0 is a breaking change, independent of the 0.16.x → 1.0 migration above. As of flutter_gemma_litertlm 1.5.0, flutter_gemma_embeddings no longer ships a concrete embedding backend — LiteRtEmbeddingBackend moved to flutter_edge_ai_litertlm. Since 2.2.0 the package holds only the embedding tokenizer implementations; the pipeline, pooling and isolate worker live in core.

If your app registers LiteRtEmbeddingBackend(), fix the import and bump both dependencies:

// Before (< 2.0.0):
import 'package:flutter_gemma_embeddings/flutter_gemma_embeddings.dart';

// After (>= 2.0.0):
import 'package:flutter_edge_ai_litertlm/flutter_edge_ai_litertlm.dart';
dependencies:
  flutter_edge_ai_embeddings: ^2.3.0   # tokenizer implementations (still required)
  flutter_edge_ai_litertlm: ^1.13.0     # now provides LiteRtEmbeddingBackend

LiteRtEmbeddingBackend() itself is unchanged — only where the class is imported from. Since litertlm 1.8.0 it also needs a tokenizer registered beside it for .tflite models (an EmbeddingGemma 2 .litertlm carries its own): add flutter_edge_ai_embeddings to your pubspec and pass embeddingTokenizers: [GemmaEmbeddingTokenizers()], or getActiveEmbedder() fails with an EmbeddingLoadException whose cause is a StateError naming that step. You still depend on flutter_edge_ai_embeddings (it owns the tokenizers); you just no longer import a backend class from it. If you'd rather run embeddings over an ONNX/ORT model instead, flutter_edge_ai_onnx's OnnxEmbeddingBackend is a drop-in alternative — see Packages.

Breaking: builtin_ai 0.3.0 — the native layer moved to flutter_local_ai#

flutter_edge_ai_builtin_ai 0.3.0 (and flutter_gemma_builtin_ai 0.3.0 before it) is no longer a Flutter plugin. It ships no Kotlin/Swift/C++ and no pigeon; every OS backend now comes from flutter_local_ai, which it depends on. No Dart code changes — BuiltInAi, BuiltInAiEngine, BuiltInAiModels, BuiltInAiAvailability, BuiltInAiUnavailableException and BuiltInAiHuggingFaceResolver keep their names and signatures — but three build-level things move.

  1. pub get regenerates the plugin registrants and Podfile.lock: this package leaves them, flutter_local_ai enters. CI that runs a frozen pod install --deployment fails until you re-commit the lockfile.
  2. The macOS deployment floor rises from 10.15 to 12.0. A macOS 11 target fails resolution with a message naming the flutter_local_ai pod, not the package you added. iOS is unaffected — flutter_local_ai builds from 13.0 and core flutter_edge_ai still requires 15.0.
  3. package:flutter_gemma_builtin_ai/pigeon.g.dart is gone with the channel it wrapped. It was generated plumbing that the documented API never used.

In exchange, Windows joins the supported platforms (AI Foundry / Phi Silica), and BuiltInAiModels gains windowsAiFoundry, chromePromptApi, all and forCurrentPlatform. Requesting vision on a backend that has none now throws at model creation instead of dropping images mid-conversation. See Built-in AI.

Breaking: rag_sqlite 1.1.0 — the index does not carry over#

flutter_gemma_rag_sqlite 1.1.0 replaced the Dart brute-force/HNSW store with in-SQLite vec0 KNN, and with it the table the index lives in: documents became vec_documents. An index written by 1.0.x is not read by 1.1.0+. This shipped as a minor version with no note — if you upgraded and your RAG answers went vague, this is why.

Nothing errors. initialize() succeeds, getStats() reports 0 documents, searchSimilar() returns no hits, and your rows are still sitting in the old documents table, unread. The model then answers without the context it used to have, which reads as the model getting worse rather than as a migration you missed.

Your data is recoverable. Unlike the qdrant break below, nothing is lost: 1.0.x stored id, content, the embedding as a Float32 BLOB and metadata in a plain table, all still readable. Move it once at startup — no re-embedding, no model needed:

Before copying anything, identify the exact embedder that produced the old vectors. The profile ID must version its weights, tokenizer, pooling, normalization, and document/query prefixes; the dimension must equal that embedder's output dimension. Do not infer identity from the database path or reuse the example values below without verifying them.

import 'dart:typed_data';
import 'package:flutter_edge_ai_rag/flutter_edge_ai_rag.dart';
import 'package:flutter_edge_ai_sqlite/flutter_edge_ai_sqlite.dart';
import 'package:path_provider/path_provider.dart';
import 'package:sqlite3/sqlite3.dart';   // add sqlite3 to your own pubspec

final documentsDirectory = await getApplicationDocumentsDirectory();
final databasePath = '${documentsDirectory.path}/rag.db';

// This example is exact only for the pinned EmbeddingGemma pipeline named
// here. Replace both constants if another embedder produced the legacy rows.
const sourceEmbeddingProfileId =
    'embeddinggemma-300m-seq256-mp-rev-29888fcee321-'
    'retrieval-prefix-meanpool-l2-v1';
const sourceEmbeddingDimension = 768;
final sourceEmbeddingProfile = EmbeddingProfile(
  id: sourceEmbeddingProfileId,
  dimension: sourceEmbeddingDimension,
);

final store = SqliteVectorStore();
await store.initialize(databasePath);

final db = sqlite3.open(databasePath);
try {
  final hasLegacy = db
      .select("SELECT name FROM sqlite_master "
              "WHERE type='table' AND name='documents'")
      .isNotEmpty;

  if (hasLegacy) {
    final legacyRows = db.select(
      'SELECT id, content, embedding, metadata FROM documents',
    );
    if (legacyRows.isNotEmpty) {
      final firstEmbedding = legacyRows.first['embedding'] as Uint8List;
      final storedDimension = firstEmbedding.lengthInBytes ~/ 4;
      if (firstEmbedding.lengthInBytes % 4 != 0 ||
          storedDimension != sourceEmbeddingDimension) {
        throw StateError(
          'Legacy vectors do not match $sourceEmbeddingProfile',
        );
      }
      // Bind the verified space before the first addDocument call.
      await store.bind(VectorStoreBinding(profile: sourceEmbeddingProfile));
    }

    for (final row in legacyRows) {
      // 1.0.x wrote each element with setFloat32(..., Endian.little); read it
      // back the same way. ByteData.sublistView needs no 4-byte alignment,
      // which a raw asFloat32List view of the BLOB would.
      final bytes = ByteData.sublistView(row['embedding'] as Uint8List);
      if (bytes.lengthInBytes % 4 != 0 ||
          bytes.lengthInBytes ~/ 4 != sourceEmbeddingDimension) {
        throw StateError('Legacy row ${row['id']} has the wrong dimension');
      }
      await store.addDocument(
        id: row['id'] as String,
        content: row['content'] as String,
        embedding: List<double>.generate(
          bytes.lengthInBytes ~/ 4,
          (i) => bytes.getFloat32(i * 4, Endian.little),
        ),
        metadata: row['metadata'] as String?,
      );
    }
    db.execute('DROP TABLE documents');   // only after the loop succeeds
  }
} finally {
  db.close();
  await store.close();
}

Guard it with your own "already migrated" flag if you prefer, but the sqlite_master check is enough: dropping the table is what makes the block a no-op on every later launch.

There is no built-in migration call — this is a one-time fix for an upgrade that has already happened, not an ongoing API.

Breaking: rag_qdrant 1.3.0 — the on-disk store is not readable#

flutter_gemma_rag_qdrant 1.3.0 moves onto the official qdrant_edge UniFFI SDK, and an index written by 1.2 or earlier cannot be read. This is a data change, not an API change: your addDocument / searchSimilar calls are unchanged, but the documents already on the device are not.

An upgraded app finds no documents where its corpus used to be. 1.3.0 refuses loudly rather than starting empty — initialize() throws a QdrantLegacyStoreException naming the old store — so this shows up the first time the store opens, not as silently unanswered questions later.

Remove the old store's files once, then re-index. 1.3.0 will not do it for you: it never deletes data it cannot read, and the three entries a 1.x shard owns (edge_config.json, wal/, segments/) may sit beside files of your own.

import 'package:flutter_edge_ai_qdrant/flutter_edge_ai_qdrant.dart';

final store = QdrantVectorStore();
try {
  await store.initialize(path);
} on QdrantLegacyStoreException catch (e) {
  // e.message names the three entries a 1.x shard owns. Remove them with the
  // file APIs you already use for `path`, then initialize() again.
  rethrow;
}
// ...then re-add your documents, and flush() — on qdrant, points live in the
// shard's in-RAM segment until then, so a background kill loses the re-index.

Catch QdrantLegacyStoreException, not the base VectorStoreException. initialize() also throws the base type when a current-layout (1.3.0 and later) shard is present but will not open right now — a WAL held by another store, a permission problem — and treating that as "the old format is here" is how a recovery step can act on a store that is perfectly fine.

clear() no longer deletes anything: it empties the shard in place, and it refuses when a 1.x layout is present rather than removing files it cannot read.

If your app has no re-indexing path of its own, do the re-index behind the same progress UI you use for the first run — from the user's side this is a rebuild of the index, not a migration they can be asked to wait through silently.

2. main.dart — current 2.0 composition#

Before (0.16.x): engines were bundled into core; initialize() was optional.

void main() {
  WidgetsFlutterBinding.ensureInitialized();
  // (initialize was optional — only for HF token / retries)
  runApp(MyApp());
}

Current (2.0): register only AI runtimes with core. Construct RAG with its own provider registry and own the returned index separately.

import 'package:flutter/widgets.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:flutter_edge_ai_embeddings/flutter_edge_ai_embeddings.dart';
import 'package:flutter_edge_ai_litertlm/flutter_edge_ai_litertlm.dart';
import 'package:flutter_edge_ai_mediapipe/flutter_edge_ai_mediapipe.dart';
import 'package:flutter_edge_ai_qdrant/flutter_edge_ai_qdrant.dart';
import 'package:flutter_edge_ai_rag/flutter_edge_ai_rag.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();

  await FlutterEdgeAi.initialize(
    inferenceEngines: const [LiteRtLmEngine(), MediaPipeEngine()],
    embeddingBackends: const [LiteRtEmbeddingBackend()], // flutter_edge_ai_litertlm
    embeddingTokenizers: const [GemmaEmbeddingTokenizers()], // flutter_edge_ai_embeddings
    // '' when the define is absent — an empty token still sends a bare
    // `Authorization: Bearer` header, so pass null instead.
    huggingFaceToken: const String.fromEnvironment('HUGGINGFACE_TOKEN').isNotEmpty
        ? const String.fromEnvironment('HUGGINGFACE_TOKEN')
        : null,
  );

  final rag = FlutterEdgeAiRag(
    providers: const [QdrantVectorStoreProvider()],
  );
  final index = await rag.open(
    spec: VectorStoreSpec(
      providerId: 'qdrant',
      location: ragDirectory,
    ),
    // open() loads the embedder: install it before this call.
    embedder: FlutterEdgeAiActiveEmbedder(
      profileId: 'embeddinggemma-300m-seq256-mp-rev-29888fcee321-'
          'retrieval-prefix-meanpool-l2-v1',
    ),
  );

  runApp(MyApp(ragIndex: index));
}

Only list what you ship. If you don't do embeddings, omit embeddingBackends; if you don't do RAG, do not create an index. A vector-only or custom-embedder RAG pipeline can run without initializing core at all. Dispose every RagIndex before FlutterEdgeAi.dispose() or before disposing its custom embedder.

3. Model/chat stays compatible; RAG uses RagIndex#

Model, session, chat, and embedding installation calls keep their API (the embedding calls themselves changed in 2.3.0, see above). Replace the removed singleton RAG facade with calls on the app-owned index:

// install + run a model
await FlutterEdgeAi.installModel(
    modelType: ModelType.gemma4, fileType: ModelFileType.litertlm)
    .fromNetwork(url, token: token).install();
final model = await FlutterEdgeAi.getActiveModel(maxTokens: 2048);
final chat  = await model.openChat();
await chat.addQueryChunk(Message.text(text: 'Hello', isUser: true));
await for (final r in chat.generateChatResponseAsync()) { /* r is a ModelResponse */ }

// embeddings + RAG
await FlutterEdgeAi.installEmbedder()
    .modelFromNetwork(modelUrl, token: token)
    .tokenizerFromNetwork(tokenizerUrl, token: token)
    .install();
await index.add(id: 'doc-1', document: EmbeddingInput.text(document));
await index.flush(); // required for qdrant; Web SQLite durability fence
final hits = await index.search(query: EmbeddingInput.text(query), topK: 5);

// Shutdown order: the index borrows its embedder.
await index.dispose();
await FlutterEdgeAi.dispose(); // resets core's registries; closes no model

What you'll see if you forget step 2#

  • Calling getActiveModel() with no matching inferenceEngines registered throws a StateError naming the model's ModelFileType and the engines that are registered — add the engine package for that file type.
  • FlutterEdgeAi.getActiveEmbedder() or default text RAG with no matching embeddingBackends throws a clear error naming the runtime package to add.
  • FlutterEdgeAiRag.open() with no matching registered provider throws and lists the registered provider IDs. Text RAG with the default active embedder needs embedder: FlutterEdgeAiActiveEmbedder(profileId: …) with a stable id; raw vectors need openVectors(profile: …).

Platform setup#

Native setup moved to the package that owns it:

  • MediaPipe Gradle / Pod deps + the @mediapipe/tasks-genai web CDN are now in flutter_edge_ai_mediapipe (bundled automatically on Android/iOS; add the CDN <script> for web).
  • The .litertlm native library + the @litert-lm/core web CDN are in flutter_edge_ai_litertlm (the native library through its litertlm_dart dependency from 1.21.0).
  • The sqlite-vec web loader (sqlite3.wasm with sqlite-vec statically linked) is in flutter_edge_ai_sqlite.

The iOS/Android entitlements and manifest entries still apply when you ship an inference engine. See the full Installation guide.

Troubleshooting#

dlopen "library not found" after upgrading from flutter_gemma_*: run flutter clean and delete ~/Library/Caches/flutter_gemma/native (Linux: ~/.cache/flutter_gemma/native, Windows: %LOCALAPPDATA%\flutter_gemma\native), then flutter pub get. Since LiteRT-LM moved to litertlm_dart (the flutter_edge_ai_litertlm released with core 2.7.0), its own cache is litertlm_dart/native in the same places. litertlm_dart (under flutter_edge_ai_litertlm from that release) owns the native LiteRT library; flutter_edge_ai_litertlm and flutter_edge_ai_speech consume it through it — neither has a Native-Assets hook of its own. flutter_edge_ai_embeddings has no native code at all.