Shipping A Model In An Ios App

---
name: shipping-a-model-in-an-ios-app
description: >-
  Put a local language model inside a native iOS or macOS app written in Swift.
  Covers MLX on Apple silicon, Core ML conversion, llama.cpp via a Swift
  wrapper, choosing between them, app binary size and memory limits,
  background and memory-pressure behaviour, and whether to bundle the model or
  download it on first run. Also covers when Apple Foundation Models is the
  better answer than shipping your own. Use when the project is native Swift
  for iOS, iPadOS or macOS and someone wants on-device or offline AI or local
  inference. Not for React Native or Expo projects, not for Android native
  Kotlin projects, not for Flutter, and not for models running on a server.
license: Apache-2.0
metadata:
  version: "0.1.0"
  author: "Edward Xi Yang, Ertas AI"
  stage: "ship"
  previous_skill: "inspecting-a-model-bundle"
---

# Shipping a model in an iOS app

Before any of this: check whether shipping a model is even the right call. iOS
26 and later ships **Apple Foundation Models**, a roughly 3B on-device model
built into the OS, reachable from Swift with no model to bundle, host, or
update. For a task inside what that model can already do (summarising,
rewriting, extracting structured fields, simple classification), the honest
recommendation is to use it and skip everything below. Shipping a custom model
is the right call when the task needs quality Apple's built-in model does not
reach on its own, needs to run on hardware or an OS version Apple Intelligence
does not cover, or needs to be portable to another platform. See "Or ship
nothing at all" further down before starting the install steps.

## Which artifact shape this needs

Two runtimes cover native Swift, and they take different artifact shapes.

| You are holding | Package | What it eats |
|---|---|---|
| A merged Hugging Face checkpoint, converted with `mlx_lm.convert` | MLX Swift (`mlx-swift-lm`) | MLX-format safetensors |
| A single `.gguf` file | LLM.swift (wraps llama.cpp) | GGUF, self-contained |

**If what you are holding is a merged Hugging Face checkpoint** (`config.json`
+ `model*.safetensors`), it is not directly shippable into either package: MLX
Swift needs it converted to MLX format first, and the GGUF path needs it
converted to GGUF first. Run **inspecting-a-model-bundle** first to confirm
which shape you actually have, then convert:

- To MLX format: `mlx_lm.convert --hf-path <repo> -q` for a 4-bit quantised
  export, covered below and in `references/mlx-path.md`.
- To GGUF: `convert_hf_to_gguf.py` then `llama-quantize`, covered below and in
  `references/coreml-and-llamacpp-path.md`.

**If it is a PEFT adapter directory** (`adapter_config.json` +
`adapter_model.safetensors`), neither package loads it directly on iOS. Merge
it into its base model first, on Apple silicon with `mlx_lm.fuse`, or
elsewhere with `peft`'s `merge_and_unload()`, then export from the merged
checkpoint. Whether the Swift side of MLX can load an unmerged adapter the way
the Python `mlx_lm` CLI can with `--adapter-path` is unconfirmed as of this
writing. Assume merge-before-ship until that is verified.

**If it is genuinely unclear which to pick,** MLX Swift is the natural default
on Apple silicon devices (the package's own declared floor is **iOS 17,
macOS 14**, the same figures given in the install section below; nothing in
its repository states a device model requirement beyond Apple silicon): it is
Apple's own array framework, tuned for Apple silicon's unified memory and GPU on that
hardware, and the conversion tooling is a single command. LLM.swift is
the safer choice when the app also needs to run on Intel Macs, or when the
model was already exported to GGUF for another target (desktop, Android) and
reusing that one file across platforms matters more than squeezing out the
last bit of Apple-silicon performance.

**Core ML is a third option and the least turnkey of the three.** `coremltools`
converts a PyTorch or Hugging Face model to a `.mlpackage`, but the Hugging
Face `exporters` project's own "Known issues" section reports that a flexible
input sequence length, which is what a chat interface wants, makes the
converter "extremely slow" and allocate "over 200 GB of RAM" on GPT2 and
GPT-Neo, so a usable export in practice means fixed sequence lengths and, in
most current writeups, no KV cache. Treat raw Core ML LLM conversion as an
advanced, manual path rather than a first choice. It is covered at a high
level in `references/coreml-and-llamacpp-path.md` for
when MLX and GGUF both fall short of a specific requirement (for example,
Core ML's tighter integration with the Neural Engine on older
devices).

## Install and wiring

### MLX Swift (`mlx-swift-lm`)

```swift
// Package.swift
dependencies: [
    .package(url: "https://github.com/ml-explore/mlx-swift-lm", .upToNextMajor(from: "3.31.3")),
    .package(url: "https://github.com/huggingface/swift-huggingface", from: "0.9.0"),
    .package(url: "https://github.com/huggingface/swift-transformers", from: "1.3.0"),
],
targets: [
    .target(
        name: "YourTargetName",
        dependencies: [
            .product(name: "MLXLLM", package: "mlx-swift-lm"),
            .product(name: "MLXLMCommon", package: "mlx-swift-lm"),
            .product(name: "MLXHuggingFace", package: "mlx-swift-lm"),
            .product(name: "HuggingFace", package: "swift-huggingface"),
            .product(name: "Tokenizers", package: "swift-transformers"),
        ]),
]
```

This is the package's own installation snippet, pinned from 3.31.3; the
latest tagged release is 3.31.4 as of this writing. Minimum platforms, from
the package's `Package.swift`: **iOS 17, macOS 14**, Swift tools version 6.1.
`MLXLLM` and `MLXLMCommon` are the two products a chat app needs; add
`MLXVLM` only for vision-language models and `MLXEmbedders` only for
embedding models. `MLXHuggingFace` plus the two Hugging Face packages are the
"simplest way to get started" integration the README recommends, giving a
default downloader and tokenizer; a project can swap in a different
downloader or tokenizer integration instead, covered in the package's own
"using" documentation.

Loading a model that is already on the Hub, from the package's own README:

```swift
import MLXLLM
import MLXLMCommon
import MLXHuggingFace
import HuggingFace
import Tokenizers

let model = try await #huggingFaceLoadModelContainer(
    configuration: LLMRegistry.gemma3_1B_qat_4bit
)
let session = ChatSession(model)
```

For a custom fine-tune exported to MLX format and bundled or downloaded
locally rather than pulled from the Hub at runtime, load it from a local
directory path instead of a Hub configuration: see `references/mlx-path.md`
for the local-path loading call and how it differs from the Hub path above.

### LLM.swift (GGUF, wraps llama.cpp)

```swift
// Package.swift
dependencies: [
    .package(url: "https://github.com/eastriverlee/LLM.swift/", branch: "main"),
],
```

Latest tagged release v3.0.3 (2026-07-19); the package's own README
installation snippet points at the `main` branch rather than a version tag.
Minimum platforms, from the package's `Package.swift`: **iOS 16, macOS 13**,
Swift tools version 5.9. The package wraps llama.cpp through a binary target
that downloads a prebuilt `llama` library from llama.cpp's own releases, plus
a small C++ wrapper layer; there is nothing else to install on top of the
Swift package itself.

## A minimal working example

### MLX Swift: load, generate, stream

```swift
import MLXLLM
import MLXLMCommon
import MLXHuggingFace
import HuggingFace
import Tokenizers

let model = try await #huggingFaceLoadModelContainer(
    configuration: LLMRegistry.gemma3_1B_qat_4bit
)
let session = ChatSession(model)

// One full generation:
let answer = try await session.respond(to: "What is the meaning of life?")
print(answer)
```

`ChatSession.respond(to:)` awaits the full response. For token-by-token
streaming, drop to the lower-level `generate` call that `ChatSession` itself
is built on, which returns an `AsyncStream<Generation>` to consume with
`for await` instead of returning a single string:

```swift
import MLXLMCommon

let stream = try MLXLMCommon.generate(
    input: lmInput, parameters: generateParameters, context: context
)

for await generation in stream {
    switch generation {
    case .chunk(let text):
        // append text to whatever UI state renders the growing response
        print(text, terminator: "")
    case .info(let info):
        print("finished: \(info.tokensPerSecond) tokens/s")
    case .toolCall(let call):
        print("tool call: \(call.function.name)")
    }
}
```

The stream yields a `.chunk` per generated text fragment, already decoded, so
there is no detokenizer to drive by hand, then one final `.info` carrying
token counts and throughput. Cap output length with
`GenerateParameters(maxTokens:)` rather than by counting inside the loop;
breaking out of the `for await` early stops delivery, but the package's own
note warns computation continues for a few more milliseconds after that.
There is an older callback-based `generate(input:parameters:context:didGenerate:)`
taking a per-batch closure, but upstream marks it deprecated in favour of the
call above, so a new integration should not start there. Consult
`references/mlx-path.md` for where `lmInput`, `generateParameters`, and
`context` come from before this call.

### LLM.swift: load, generate, stream

From the package's own README, loading a bundled GGUF file and running one
full generation:

```swift
import LLM

let bot = LLM(
    from: Bundle.main.url(forResource: "gemma-3-4b-it-q4_0", withExtension: "gguf")!,
    template: .gemma
)
let question = bot.preprocess("What's the meaning of life?", [])
let answer = await bot.getCompletion(from: question)
print(answer)
```

As of the current release, `LLM.swift` can also read the chat template
embedded in the GGUF file itself and render it through llama.cpp's own
template engine, so `template:` becomes optional for models that carry a
usable embedded template; pass it explicitly, as above, when a model's
metadata is missing or broken, or to force a specific format.

For token-by-token streaming, set the `update` callback before calling the
no-argument `respond(to:)`, which fires `update` once per generated text
delta and once more with `nil` when generation stops:

```swift
bot.update = { delta in
    guard let delta else { return } // nil marks the end of generation
    // delta is the newly generated text fragment; append it to UI state here
}
await bot.respond(to: "What is the meaning of life?")
```

`from:` takes a `URL`, which can point at a bundled resource as above, or a
downloaded file in the app's documents directory for the download-on-first-run
path. `template` selects the chat prompt format for the model family
(`.chatML`, `.gemma`, and others, each taking the system prompt as an
argument, for example `.chatML("You are a helpful assistant.")`); pick the
one matching how the model was fine-tuned; a mismatched template still
produces fluent-looking text while ignoring the system prompt and turn
structure.

## Or ship nothing at all

If the task fits inside Apple's built-in model, this whole skill is
unnecessary. The following is confirmed against a third-party Swift tutorial
covering the framework, not Apple's own documentation page directly, which
renders via JavaScript and so is not readable as plain text:

```swift
import FoundationModels

let model = SystemLanguageModel.default
switch model.availability {
case .available:
    let session = LanguageModelSession()
    let result = try await session.respond(to: "Summarise this in one sentence.")
    print(result.content)
case .unavailable(let reason):
    // device not eligible, Apple Intelligence off, or model still downloading
    break
}
```

No package to add, no model to bundle or download, no size budget to plan
around. The constraints: iOS 26 and later only, and Apple Intelligence-eligible
hardware only. The commonly reported device list is iPhone 15 Pro and later at
launch, plus Apple silicon iPad and Mac; that list is sourced from third-party
coverage here, unconfirmed against Apple's own eligibility page. On older or
ineligible devices, `LanguageModelSession()` is reachable but
generation fails at `availability`, so a real app still needs a fallback path,
which is usually either a degraded feature or one of the shipped-model routes
above.

Between this and shipping a full custom model sits the **`.fmadapter`** route:
a LoRA adapter (rank 32, roughly 160 MB) trained against Apple's own model
through Apple's adapter training toolkit, hosted on a server and downloaded
per device rather than bundled. It buys domain adaptation without the size or
runtime cost of an independent model, at a real cost: a new adapter has to be
trained for every version of Apple's system model, and the toolkit (version
26.0.0) is already declared the last release compatible with the 26 line. It
is a reasonable choice for an iOS-only feature that needs light customisation
and nothing more; it is a poor choice for anything that needs to work the same
way on Android or the web, since the adapter is worthless outside Apple's own
model.

## The delivery decision

Once the model runs, the next question is how it reaches the device: bundled
into the app binary, or downloaded on first run. Full detail, the app store
size ceilings, and the first-run download experience are in
`references/model-delivery-and-size-budgets.md`.

Neither MLX Swift nor LLM.swift documents a package-level bundling cap the way
react-native-executorch's `require()` does at 512 MB. The ceiling for both is
whatever Apple's own build limits and install-conversion tolerance allow, and
those limits are covered in the next section, because on iOS the size ceiling
that actually catches people out is not the one most developers check first.

## Platform gotchas

- **The 80 MB `__TEXT` executable cap is a separate limit from the 4 GB app
  cap, and it is easy to miss.** Apple's build-size reference states the
  ceiling plainly: for iOS and iPadOS, max uncompressed app size is 4 GB, and
  max executable is 80 MB, "total of all `__TEXT` sections." `__TEXT` is the
  compiled code segment of the binary, not general app resources. A model
  weights file, whether GGUF or MLX safetensors, ships as a bundled resource
  or a downloaded data file, and resource data does not count against the
  80 MB figure; it counts against the 4 GB figure instead. What does count
  against the 80 MB figure is compiled code, and wrapping a large C or C++
  codebase like llama.cpp or MLX's native core as **statically linked** code
  pulls all of that compiled code into the main executable's own `__TEXT`
  segment. An app whose model file is well under a gigabyte can still fail
  App Store validation on the 80 MB executable check because of how much
  native code got statically linked in, which reads as a confusing failure
  if the mental model is "my model isn't that big, why is this rejected."
  Developers who hit this commonly report moving the heavy native dependency
  into its own dynamically linked framework (an XCFramework) rather than a
  static library, since that puts its compiled code into a separate binary
  image with its own `__TEXT` accounting; this specific mitigation is common
  developer practice rather than something confirmed on an Apple-owned page,
  so verify it holds for the exact dependency and Xcode version in use before
  relying on it to clear a submission that is right at the edge.
- **Memory pressure termination is the most common production failure, not a
  crash with a clear stack trace.** iOS's jetsam mechanism kills the app when
  system memory runs low; Apple does not publish jetsam's own algorithm, so
  this description is confirmed against developer-community sources rather
  than an Apple page. A multi-hundred-megabyte to multi-gigabyte model
  held resident is exactly the kind of allocation that trips it, especially
  on older devices with less RAM or when the app is backgrounded and
  resumed. Apple's own `DispatchSource.makeMemoryPressureSource(eventMask:queue:)`
  API lets the app observe warning and critical memory pressure events and
  react, for example by releasing a cached context or reducing the context
  window, before the OS decides to kill the process outright; the simpler
  UIKit-era `didReceiveMemoryWarningNotification` still fires too. Test this
  deliberately rather than hoping it does not happen in production: the iOS
  Simulator's Hardware menu has a "Simulate Memory Warning" action for this
  exact purpose. Treat a model that only survives on a fully-charged flagship
  device with nothing else running as not actually shipped.
- **MLX only accelerates on Apple silicon.** There is no MLX path for an
  Intel Mac; LLM.swift's llama.cpp core runs everywhere, including Intel
  Macs, at whatever speed CPU inference on that hardware allows.
- **Neither package solves per-device model versioning.** If the fine-tune
  changes, both a bundled asset and a server-hosted download URL need an
  explicit update path; there is no OTA model update built into either
  runtime.
- **Background execution is not free generation time.** iOS suspends or
  throttles background apps; a long-running generation started in the
  foreground does not reliably continue once the app backgrounds, and a
  design that depends on it needs a `BGProcessingTask` request rather than an
  assumption that the OS leaves the process alone.
- **Quantisation floor for GGUF on mobile.** Below roughly 3B parameters,
  quantising past Q4_K_M is a known quality cliff, not a bug to debug later.
  Start at Q4_K_M and only go lower after confirming quality still holds.

Full depth on each path, including the MLX conversion flags and the Core ML
and GGUF conversion commands, is in `references/mlx-path.md` and
`references/coreml-and-llamacpp-path.md`.

## Hand off to

- The artifact shape is not yet confirmed, or it is a merged checkpoint or a
  PEFT adapter that needs converting first: **inspecting-a-model-bundle**
- The model loads but generates badly, repeats, never stops, or the adapter
  will not apply: **debugging-a-bad-fine-tune**
- It is not yet known whether the fine-tune is actually better than the base
  model it started from: **evaluating-a-tuned-model**
- The question is whether running this on device is actually cheaper than an
  API at the expected usage level: **costing-a-model-vs-an-api**
- The target platform is React Native or Expo, native Android, or Flutter
  instead of native iOS: **shipping-a-model-in-a-react-native-app**,
  **shipping-a-model-in-an-android-app**, or
  **shipping-a-model-in-a-flutter-app**