Win Ai Local

---
name: win-ai-local
description: Local AI inference via Ollama — run LLMs on-device, manage models, detect NPU/GPU/DirectML hardware, zero cloud dependencies.
metadata:
  {
    "openclaw":
      {
        "emoji": "🧠",
        "os": ["win32"],
        "requires": { "bins": ["ollama"] },
      },
  }
---

# win-ai-local

Run LLMs locally on Windows via Ollama. Manage models, generate text, chat, and detect AI hardware (NPU, GPU, DirectML, WinML).

Requires [Ollama](https://ollama.com) installed. Works on Windows 10/11.

## Detect AI Hardware

```powershell
powershell.exe -NoProfile -Command "
Write-Host '--- NPU ---'
Get-CimInstance Win32_PnPEntity | Where-Object { $_.Name -match 'NPU|Neural|AI Boost|Movidius|VPU' } | ForEach-Object {
    Write-Host ('{0} [{1}]' -f $_.Name, $_.Status)
}
Write-Host ''
Write-Host '--- GPU ---'
Get-CimInstance Win32_VideoController | Where-Object { $_.Status -eq 'OK' } | ForEach-Object {
    Write-Host ('{0} (VRAM: {1} GB)' -f $_.Name, [math]::Round($_.AdapterRAM/1GB,1))
}
Write-Host ''
Write-Host '--- ML Runtimes ---'
$dml = Get-Item 'C:\Windows\System32\DirectML.dll' -ErrorAction SilentlyContinue
if ($dml) { Write-Host ('DirectML: {0}' -f $dml.VersionInfo.FileVersion) }
else { Write-Host 'DirectML: Not found' }
$winml = Get-Item 'C:\Windows\System32\windows.ai.machinelearning.dll' -ErrorAction SilentlyContinue
if ($winml) { Write-Host ('WinML: {0}' -f $winml.VersionInfo.FileVersion) }
else { Write-Host 'WinML: Not found' }
"
```

## Ollama Status

```powershell
powershell.exe -NoProfile -Command "
$proc = Get-Process -Name 'ollama*' -ErrorAction SilentlyContinue
if ($proc) {
    Write-Host ('Ollama: Running (PID {0})' -f ($proc.Id -join ', '))
} else {
    Write-Host 'Ollama: Not running'
}
$ver = & ollama --version 2>&1
Write-Host $ver
"
```

## Start Ollama

```powershell
powershell.exe -NoProfile -Command "
Start-Process ollama -ArgumentList 'serve' -WindowStyle Hidden
Write-Host 'Ollama server starting...'
Start-Sleep -Seconds 2
$proc = Get-Process -Name 'ollama*' -ErrorAction SilentlyContinue
if ($proc) { Write-Host 'Ollama is running.' }
else { Write-Host 'Failed to start. Try running ollama serve manually.' }
"
```

## Stop Ollama

```powershell
powershell.exe -NoProfile -Command "
Stop-Process -Name 'ollama*' -Force -ErrorAction SilentlyContinue
Write-Host 'Ollama stopped.'
"
```

## List Models

```powershell
powershell.exe -NoProfile -Command "
ollama list
"
```

## Pull a Model

```powershell
powershell.exe -NoProfile -Command "
ollama pull MODEL_NAME
"
```

Replace `MODEL_NAME` with the model to download (e.g., `gemma3:4B`, `llama3.2:3b`, `phi4-mini`, `qwen2.5:7b`).

Recommended models for AIPC (8-32 GB RAM):

| Model | Size | Good for |
|-------|------|----------|
| `gemma3:4B` | 3.3 GB | General, fast |
| `phi4-mini` | 2.5 GB | Reasoning, compact |
| `llama3.2:3b` | 2.0 GB | General, lightweight |
| `qwen2.5:7b` | 4.7 GB | Multilingual (Chinese) |
| `deepseek-r1:8b` | 4.9 GB | Coding, reasoning |

## Delete a Model

```powershell
powershell.exe -NoProfile -Command "
ollama rm MODEL_NAME
"
```

## Show Model Info

```powershell
powershell.exe -NoProfile -Command "
ollama show MODEL_NAME
"
```

## Generate Text

Single prompt, single response (no conversation):

```powershell
powershell.exe -NoProfile -Command "
[Console]::OutputEncoding = [System.Text.Encoding]::UTF8
$body = @{
    model = 'MODEL_NAME'
    prompt = 'PROMPT_TEXT'
    stream = $false
} | ConvertTo-Json -Depth 3
$response = Invoke-RestMethod -Uri 'http://localhost:11434/api/generate' -Method POST -Body $body -ContentType 'application/json'
Write-Host $response.response
Write-Host ''
Write-Host ('Tokens: {0} | Duration: {1:N1}s | Speed: {2:N1} tok/s' -f $response.eval_count, ($response.total_duration / 1e9), ($response.eval_count / ($response.eval_duration / 1e9)))
"
```

- `MODEL_NAME`: model to use (e.g., `gemma3:4B`)
- `PROMPT_TEXT`: the prompt to send

### With System Prompt

```powershell
powershell.exe -NoProfile -Command "
[Console]::OutputEncoding = [System.Text.Encoding]::UTF8
$body = @{
    model = 'MODEL_NAME'
    prompt = 'PROMPT_TEXT'
    system = 'SYSTEM_PROMPT'
    stream = $false
} | ConvertTo-Json -Depth 3
$response = Invoke-RestMethod -Uri 'http://localhost:11434/api/generate' -Method POST -Body $body -ContentType 'application/json'
Write-Host $response.response
"
```

### With Parameters (Temperature, Max Tokens)

```powershell
powershell.exe -NoProfile -Command "
[Console]::OutputEncoding = [System.Text.Encoding]::UTF8
$body = @{
    model = 'MODEL_NAME'
    prompt = 'PROMPT_TEXT'
    stream = $false
    options = @{
        temperature = 0.7
        num_predict = 500
    }
} | ConvertTo-Json -Depth 3
$response = Invoke-RestMethod -Uri 'http://localhost:11434/api/generate' -Method POST -Body $body -ContentType 'application/json'
Write-Host $response.response
"
```

- `temperature`: 0.0 (deterministic) to 2.0 (creative), default 0.8
- `num_predict`: max tokens to generate, default 128

## Chat (Multi-Turn)

```powershell
powershell.exe -NoProfile -Command "
[Console]::OutputEncoding = [System.Text.Encoding]::UTF8
$body = @{
    model = 'MODEL_NAME'
    messages = @(
        @{ role = 'system'; content = 'SYSTEM_PROMPT' }
        @{ role = 'user'; content = 'USER_MESSAGE' }
    )
    stream = $false
} | ConvertTo-Json -Depth 5
$response = Invoke-RestMethod -Uri 'http://localhost:11434/api/chat' -Method POST -Body $body -ContentType 'application/json'
Write-Host $response.message.content
Write-Host ''
Write-Host ('Tokens: {0} | Speed: {1:N1} tok/s' -f $response.eval_count, ($response.eval_count / ($response.eval_duration / 1e9)))
"
```

### Multi-Turn Conversation

```powershell
powershell.exe -NoProfile -Command "
[Console]::OutputEncoding = [System.Text.Encoding]::UTF8
$body = @{
    model = 'MODEL_NAME'
    messages = @(
        @{ role = 'user'; content = 'FIRST_MESSAGE' }
        @{ role = 'assistant'; content = 'FIRST_RESPONSE' }
        @{ role = 'user'; content = 'SECOND_MESSAGE' }
    )
    stream = $false
} | ConvertTo-Json -Depth 5
$response = Invoke-RestMethod -Uri 'http://localhost:11434/api/chat' -Method POST -Body $body -ContentType 'application/json'
Write-Host $response.message.content
"
```

## Embeddings

Generate text embeddings for semantic search or RAG:

```powershell
powershell.exe -NoProfile -Command "
$body = @{
    model = 'MODEL_NAME'
    input = 'TEXT_TO_EMBED'
} | ConvertTo-Json -Depth 3
$response = Invoke-RestMethod -Uri 'http://localhost:11434/api/embed' -Method POST -Body $body -ContentType 'application/json'
Write-Host ('Dimensions: {0}' -f $response.embeddings[0].Count)
Write-Host ('First 5: {0}' -f ($response.embeddings[0][0..4] -join ', '))
"
```

Use an embedding model like `nomic-embed-text` or `mxbai-embed-large`.

## Common Workflows

### Quick local inference (no API key needed)

```
1. win-ai-local: check Ollama status (start if needed)
2. win-ai-local: generate text with a prompt
3. Use the result in your workflow
```

### Translate text locally

```
1. win-ai-local: generate with system prompt "Translate to English"
2. No data leaves the machine — fully private
```

### Summarize a document privately

```
1. Read the document content
2. win-ai-local: generate with prompt "Summarize: {content}"
3. Confidential data stays local
```

### Agent delegates to local LLM

```
1. Main agent identifies a sub-task suitable for local inference
2. win-ai-local: chat with appropriate system prompt
3. Main agent uses the response
4. Saves API costs for simple tasks
```

### On-device RAG pipeline

```
1. win-ai-local: generate embeddings for documents
2. Store embeddings locally
3. On query, embed the query and find similar docs
4. win-ai-local: chat with retrieved context
```

## Notes

- Requires [Ollama](https://ollama.com) installed and running (`ollama serve`).
- Models are stored at `%USERPROFILE%\.ollama\models\`.
- Ollama API runs at `http://localhost:11434` — fully local, no data leaves the machine.
- GPU acceleration: Ollama auto-detects and uses available GPU (Intel Arc, NVIDIA, AMD).
- NPU acceleration: not yet supported by Ollama directly. Use OpenVINO or WinML for NPU inference.
- For Chinese text, use `qwen2.5` models which have strong multilingual support.
- Inference speed depends on model size and hardware. On Intel Arc 140V: ~5-10 tok/s for 4B models.
- `stream = $false` returns the full response at once. Set to `$true` for streaming (requires different parsing).
- Max context window varies by model (typically 2K-128K tokens). Check with `ollama show MODEL_NAME`.
- For production use, consider running Ollama as a Windows service for auto-start.