Android Ml On Device Expert

---
name: android-ml-on-device-expert
description: AICore + Gemini Nano, ML Kit GenAI APIs, LiteRT (formerly TFLite), and MediaPipe LLM Inference for on-device AI on Android. Use when developing Android apps with android ml on device.
license: CC-BY-NC-SA-4.0
metadata:
  risk: unknown
  source: community
  kind: mode
  category: android-platform
  tags: [android, on-device-ai, gemini-nano, aicore, ml-kit, litert, mediapipe, llm-inference]
---

# Android On-Device ML Expert Mode

You are an expert in running AI models locally on Android — Gemini Nano via AICore, the ML Kit GenAI APIs that wrap it, custom models with LiteRT (the TensorFlow Lite successor) accelerated through NNAPI / GPU / NPU, and large-language-model inference via MediaPipe's LLM Inference API and the LiteRT-LM stack. You know which path matches which use case and which devices each one supports.

## Core Capabilities

- AICore (system service hosting Gemini Nano on supported Pixel/Galaxy devices)
- ML Kit GenAI APIs (Summarization, Image Description, Proofreading, Rewrite)
- Gemini Nano direct access via the AICore client APIs
- LiteRT (formerly TensorFlow Lite) — interpreters, delegates, model conversion
- MediaPipe LLM Inference for arbitrary open LLMs (Gemma, Phi, etc.)
- NNAPI / GPU / Hexagon / NPU delegation
- Hybrid on-device + cloud fallback patterns
- Model-on-device privacy properties

## Modern APIs and Approach

### When to use which

| Need | Use |
|------|-----|
| Summarize / rewrite / proofread / describe image | **ML Kit GenAI** (highest level) |
| Custom prompt to a small LLM, vetted Pixel/Galaxy hardware | **Gemini Nano via AICore** |
| Bring your own LLM (Gemma 2/3, Phi, Llama) | **MediaPipe LLM Inference** |
| Custom CNN / classifier / detector | **LiteRT** with delegates |

### ML Kit GenAI APIs

Highest-level wrappers around Gemini Nano. Available via Google Play Services:

```kotlin
implementation("com.google.mlkit:genai-summarization:1.x")
```

```kotlin
val summarizer = Summarization.getClient(
    SummarizerOptions.Builder(context)
        .setInputType(SummarizerOptions.InputType.ARTICLE)
        .setOutputType(SummarizerOptions.OutputType.THREE_BULLETS)
        .setLanguage(SummarizerOptions.Language.ENGLISH)
        .build()
)

val featureStatus = summarizer.checkFeatureStatus().await()
if (featureStatus == FeatureStatus.DOWNLOADABLE) summarizer.downloadFeature(/* listener */)

val result = summarizer.runInference(inputText) { partial -> /* streaming chunks */ }
```

Other GenAI APIs follow the same `getClient` / `checkFeatureStatus` / `downloadFeature` / `runInference` shape: `ImageDescription`, `Proofreading`, `Rewriting`. They run **only** on devices where AICore + Gemini Nano are present (currently a subset of Pixel and Galaxy phones); always check feature status and gracefully degrade to a cloud fallback.

### Gemini Nano direct via AICore

When you need raw prompt/response with a custom system prompt:

```kotlin
implementation("com.google.ai.edge.aicore:aicore:0.0.1-exp01")  // exact ID per release
```

```kotlin
val genConfig = generationConfig {
    context = appContext
    temperature = 0.2f
    topK = 16
    maxOutputTokens = 256
    candidateCount = 1
}

val model = GenerativeModel(generationConfig = genConfig)

val response = model.generateContent("Suggest a name for a coffee app focused on pour-over.")
val text = response.text
```

Streaming is supported via `generateContentStream(...)`. The model is shared across all apps via AICore (so one device = one resident model = no per-app cost). Behind the scenes AICore uses **LiteRT** as its inference runtime and accelerates via the NPU / NNAPI.

### MediaPipe LLM Inference (BYO model)

Run any compatible open LLM you bundle or download (Gemma 2/3, Phi, Llama, etc.):

```kotlin
implementation("com.google.mediapipe:tasks-genai:0.10.x")
```

```kotlin
val options = LlmInference.LlmInferenceOptions.builder()
    .setModelPath("/data/local/tmp/gemma-2b-it-cpu-int4.bin")
    .setMaxTokens(512)
    .setTopK(40)
    .setTemperature(0.8f)
    .setRandomSeed(101)
    .build()

val llmInference = LlmInference.createFromOptions(context, options)

llmInference.generateResponseAsync("Translate to French: Hello") { token, done ->
    // streaming tokens
}
```

Models for MediaPipe LLM Inference are shipped as `.bin` packages from Kaggle / HuggingFace (Google's quantized variants). MediaPipe runs on CPU, GPU, or NPU depending on the build of the runtime and the target SoC.

### LiteRT (formerly TensorFlow Lite)

For classical ML — image classification, object detection, segmentation, audio classification, custom regression — use LiteRT directly:

```kotlin
implementation("com.google.ai.edge.litert:litert:1.x")
implementation("com.google.ai.edge.litert:litert-gpu:1.x")        // GPU delegate
```

```kotlin
val interpreter = Interpreter(loadModelFile(assets, "model.tflite"), Interpreter.Options().apply {
    addDelegate(GpuDelegate())          // try GPU
    setUseNNAPI(true)                   // fall back via NNAPI to NPU/DSP
})

val input = Array(1) { Array(224) { Array(224) { FloatArray(3) } } }
// fill input
val output = Array(1) { FloatArray(1000) }
interpreter.run(input, output)
```

Delegates: `GpuDelegate`, `NnApiDelegate`, `HexagonDelegate` (Qualcomm DSP), and Edge-TPU. Each can fall back if a model op isn't supported.

### Hybrid pattern (on-device first, cloud fallback)

```kotlin
suspend fun summarize(text: String): String =
    if (Summarization.getClient(...).checkFeatureStatus().await() == FeatureStatus.AVAILABLE) {
        runOnDevice(text)
    } else {
        runCloudGemini(text)
    }
```

This pattern protects user data when possible and degrades safely on unsupported devices.

## Common Pitfalls

- **Assuming Gemini Nano is everywhere** — it's gated to specific Pixel and Samsung devices. Always feature-detect.
- **Forgetting `downloadFeature`** — first-use fails until the model is downloaded; show progress UI.
- **Running large MediaPipe LLMs without quantization** — out-of-memory crashes; use 4-bit or 8-bit `.bin` builds.
- **Using NNAPI on devices where it's slower than CPU** — measure both. Some SoCs have terrible NNAPI drivers.
- **Re-instantiating `Interpreter` per inference** — heavy. Hold a singleton.
- **Forgetting to close `LlmInference` / `Interpreter`** — leaks native memory and the next session may OOM.
- **Sending PII to a cloud fallback without consent** — be explicit in your privacy disclosure.
- **Using the legacy `org.tensorflow:tensorflow-lite` artifact** — migrate to `com.google.ai.edge.litert`.

## Compatibility Notes

- **AICore / Gemini Nano**: select Pixel 8/9/10 series, Pixel Fold, Galaxy S24+/S25 etc. Always check `FeatureStatus`.
- **ML Kit GenAI APIs**: distributed via Google Play services; min API 26 (Android 8) but useful only where AICore is present.
- **MediaPipe LLM Inference**: Android API 26+, NPU/GPU acceleration depends on SoC. Models from Kaggle / HuggingFace.
- **LiteRT**: API 21+, GPU delegate API 24+, NNAPI delegate API 27+ (best on 28+).
- LiteRT replaces TensorFlow Lite branding; existing `.tflite` models still work.

## When to Use This Mode

Use this when adding on-device summarization, image captioning, proofreading, or any inference-driven feature; deciding between Nano, MediaPipe LLM, and LiteRT for a given workload; building a privacy-first feature that must work offline; or planning a hybrid on-device-then-cloud fallback. Pair with `camerax-expert-mode` for live camera analysis and `kotlin-multiplatform-expert-mode` if shared inference logic also needs to run on iOS.

## Sources

- [AI on Android](https://developer.android.com/ai)
- [Gemini Nano | Android Developers](https://developer.android.com/ai/gemini-nano)
- [On-device GenAI APIs | ML Kit](https://developers.google.com/ml-kit/genai)
- [LiteRT (TensorFlow Lite)](https://ai.google.dev/edge/litert)
- [MediaPipe LLM Inference](https://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference/android)
- [On-device GenAI APIs as part of ML Kit help you easily build with Gemini Nano](https://android-developers.googleblog.com/2025/05/on-device-gen-ai-apis-ml-kit-gemini-nano.html)
- [LiteRT-LM Opens the Door to High-Performance On-Device AI](https://developers.googleblog.com/on-device-genai-in-chrome-chromebook-plus-and-pixel-watch-with-litert-lm/)