Splunk Ai Ml Toolkit Setup

---
name: splunk-ai-ml-toolkit-setup
description: "Use when the user asks about MLTK, Splunk AI Toolkit, Machine Learning Toolkit, PSC, Python for
  Scientific Computing, DSDL, Deep Learning Toolkit, Splunk anomaly detection assistants, AI Toolkit Agent
  Launchpad or the earlier Agent Builder preview, the `aiagent` command, Cisco Time Series Model, Cisco Deep
  Time Series Model, Smart Alerts
  Assistant, or AI/ML product coverage outside Splunk AI Assistant, including Cisco Data Fabric requests
  about AI Toolkit or machine-data model workflows. Install, render, validate, and audit Cisco Data Fabric
  AI Toolkit and Splunk-owned AI and machine-learning workflows beyond Splunk AI Assistant: Splunk AI
  Toolkit / MLTK (`Splunk_ML_Toolkit`), Python for Scientific Computing (PSC), Splunk App for Data Science
  and Deep Learning (DSDL), MLTK anomaly workflows, LLM/`ai` command readiness, external model runtimes,
  and legacy anomaly app migration."
compatibility: "Splunk Cloud Platform 10.5.2605: conditional. Follow documented package, entitlement, topology, and customer-managed runtime guardrails; self-managed paths remain on the public 10.4 baseline."
metadata:
  splunk_cloud_10_5: "conditional"
  compatibility_verified: "2026-08-20"
---

# Splunk AI/ML Toolkit Setup

## Prerequisites

| Tool or access | Purpose | Verify |
|---|---|---|
| Bash and Python 3 | Run bundled setup and validation helpers | `bash --version && python3 --version` |
| Required product/platform access | Inspect or configure the selected target | Complete the documented preflight |
| Credential files for live modes | Keep secrets out of chat | Verify paths only |

## Workflow Overview

```text
┌───────────┐   ┌───────────────┐   ┌───────────────┐   ┌─────────────────┐
│ Preflight │ → │ Render/review │ → │ Apply/handoff │ → │ Validate evidence │
└───────────┘   └───────────────┘   └───────────────┘   └─────────────────┘
```

## When to Activate

- MLTK, Splunk AI Toolkit, Machine Learning Toolkit, PSC, Python for Scientific Computing, DSDL, Deep Learning
  Toolkit, Splunk anomaly detection assistants, AI Toolkit Agent Launchpad, the `aiagent` command, Cisco Time
  Series Model, Cisco Deep.
- Preview and review the splunk ai ml toolkit setup workflow before any live apply phase.
- Diagnose failed prerequisites, generated assets, configuration, or validation evidence.

## Scope

Follow the documented read-only or render-first path whenever it is available.
This skill does not imply permission to mutate live systems. Require explicit
apply flags, protected credentials, and operator review for state changes.

## Examples

Inspect the supported setup modes before selecting one:

```bash
bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh --help
```

Expected output: usage, supported modes, and required arguments are displayed
without changing the target environment.

Inspect validation modes before running completion checks:

```bash
bash skills/splunk-ai-ml-toolkit-setup/scripts/validate.sh --help
```

Expected output: offline, live, and completion options are displayed when the
skill supports them; help exits without mutation.

## Troubleshooting

| Issue | Cause | Resolution |
|---|---|---|
| Preflight fails | A required tool or access path is missing | Resolve it before rendering or applying |
| Rendered assets are incomplete | Required non-secret inputs are absent | Complete intake and render again |
| Apply is blocked | Review, credentials, or explicit acceptance is missing | Use the documented handoff |
| Validation is incomplete | Live evidence is unavailable | Record the gap and keep completion open |

## Shared add-on completion gate

Whenever this workflow installs, configures, or hands off a registry-listed
Splunk app or add-on, follow the
[shared completion gate](../shared/ta_completion_gate.md). Package delivery
alone is not success; capture applicable configuration, data/readiness, and
shipped-view evidence, or explicit package evidence that no dashboards ship.

Use this skill for Splunk-owned AI and machine-learning platform workflows
that are not Splunk AI Assistant. It owns coverage reporting, install
orchestration, compatibility validation, DSDL runtime handoffs, and migration
guidance for legacy anomaly apps.

For newer Cisco Data Fabric wording, this is the AI Toolkit / model-workflow
route. Federated search, edge/ingest pipelines, and MCP server setup remain in
their dedicated skills.

AI Toolkit Agent Launchpad is owned here. It became generally available in AI
Toolkit `6.0.0`, replacing the earlier Agent Builder feature preview. It is not
Cisco Cloud Control Studio Agent Builder, which belongs to
`cisco-cloud-control-setup`.

## Coverage Boundary

This skill covers Splunk-owned and Splunk-supported AI/ML products:

- Splunk AI Toolkit / MLTK (`Splunk_ML_Toolkit`, Splunkbase `2890`)
- Python for Scientific Computing (PSC) add-ons:
  - Linux 64-bit (`2882`, `Splunk_SA_Scientific_Python_linux_x86_64`)
  - Windows 64-bit (`2883`, `Splunk_SA_Scientific_Python_windows_x86_64`)
  - Mac Intel (`2881`, `Splunk_SA_Scientific_Python_darwin_x86_64`)
  - Mac Apple Silicon (`6785`, `Splunk_SA_Scientific_Python_darwin_arm64`)
  - Linux 32-bit (`2884`) as legacy migration/blocking coverage only
- Splunk App for Data Science and Deep Learning / DSDL (`4607`,
  package id `mltk-container`)
- AI Toolkit Smart Assistants, ML-SPL commands, model management, ONNX apply,
  LLM `ai` command readiness, Connections tab, Container Management tab,
  external LLM/provider connection handoffs, ML alerting, and Cisco Deep Time
  Series forecasting/anomaly detection readiness
- AI Toolkit Agent Launchpad readiness, generally available since `6.0.0`,
  including supported LLM and MCP providers, Agent Skills, the
  `edit_agent_connections` and `run_agents` capabilities, the `aiagent` ML-SPL
  command, the in-product run-history surface, and the Splunk Cloud region and
  egress-allowlist prerequisites
- Cisco Time Series Model 1.0 as an available Apache-2.0 open-weight model,
  kept distinct from the AI Toolkit-integrated Cisco Deep Time Series Model
  and its hosted Splunk Cloud or self-hosted Enterprise paths
- Hosted foundation model readiness where available in the Splunk Platform
  boundary, including Foundation-Sec and GPT-OSS review handoffs; CDTSM is not
  represented as an LLM connection, and this skill never renders external
  model API keys
- Legacy Splunk App for Anomaly Detection (`6843`) and Smart Alerts Assistant
  beta (`6415`) as audit and migration-only coverage

Third-party AI-tagged Splunkbase apps are out of scope unless another skill
explicitly routes them.

## Splunk 10.5 Legacy Package Guardrail

Splunkbase does not list Splunk 10.5 support for PSC Linux 32-bit (`2884`),
Splunk App for Anomaly Detection (`6843`), or Smart Alerts Assistant beta
(`6415`). Do not install any of these three packages on a new or upgraded
Splunk 10.5 deployment. They remain in this skill only so an existing estate
can inventory dependencies and render a migration plan. Use a supported
64-bit PSC package and current Splunk AI Toolkit workflows for replacement
coverage.

## Safety Rules

- Never ask for Splunk passwords, Splunkbase passwords, HEC tokens, LLM API
  keys, cloud provider secrets, DSDL container credentials, or model registry
  tokens in chat.
- Never pass secrets on the command line or as environment-variable prefixes.
- LLM provider credentials, HEC tokens, Splunk access tokens, Docker registry
  secrets, Kubernetes kubeconfigs, and TLS key material must be file-backed or
  delegated to the owning setup skill.
- Do not install legacy EOL/beta anomaly apps by default, and never install
  `2884`, `6415`, or `6843` on Splunk 10.5. Audit and migrate them to current
  AI Toolkit workflows.
- Do not claim DSDL runtime automation for Docker, Kubernetes, OpenShift, HPC,
  GPU, air-gapped images, Jupyter notebooks, or model governance unless the
  workflow is rendered as a handoff or an owning runtime skill applies it.
- Do not report Agent Launchpad as ready from a package install alone. The
  public package ships the views and the `aiagent` command, but Splunk Cloud
  still needs a supported AWS region plus the region's egress IP in the stack
  allowlist, and Splunk Enterprise needs the Splunk Cloud Connect app.
- Do not pair AI Toolkit `6.0.2` with a PSC release below `4.3.4`, and do not
  downgrade the audited AI Toolkit/PSC pair based on older documentation.
- Do not conflate the open Cisco Time Series Model 1.0 release with the Cisco
  Deep Time Series Model integration. Both are available, but the open model
  weights, the model service, and the AI Toolkit experience remain separately
  validated layers.
- Do not create or redirect the agent run-history index, and do not save
  knowledge-base, MCP, LLM, or model-server credentials from this render-only
  handoff.

## Primary Workflow

Render and validate a complete coverage plan:

```bash
bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
  --render --validate \
  --spec skills/splunk-ai-ml-toolkit-setup/template.example \
  --output-dir splunk-ai-ml-toolkit-rendered
```

Install or update AI Toolkit with the right PSC add-on:

```bash
bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
  --install \
  --psc-target linux64
```

Include DSDL package delivery and runtime handoff artifacts:

```bash
bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
  --render --validate \
  --include-dsdl \
  --dsdl-runtime kubernetes \
  --output-dir splunk-ai-ml-toolkit-rendered
```

Audit legacy anomaly apps without installing them:

```bash
bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
  --doctor \
  --legacy-anomaly-audit
```

## Apply Model

- `--install` renders, runs the offline structural check, delegates package
  delivery, then runs a **live** post-install validation (skipped under
  `--dry-run`) that confirms the PSC prerequisite is installed (a missing PSC
  add-on fails) and that AI Toolkit — and DSDL when planned — are present in
  Splunk. Live validation requires Splunk credentials.
- `--validate` always renders before validating, so the offline structural
  check never runs against a stale or missing rendered directory.
- Package delivery delegates to `splunk-app-install`.
- Package delivery intentionally omits `--app-version` so Splunkbase/ACS pulls
  the latest compatible release; audited version metadata is used for reports
  and regression checks, not as a live install pin.
- AI Toolkit and PSC belong on the search tier/search head cluster only.
- Install order is PSC first, AI Toolkit second, optional DSDL third.
- DSDL external runtimes are rendered handoffs by default:
  `docker`, `kubernetes`, `openshift`, `hpc`, `gpu`, `airgap`, or `handoff`.
- Legacy Anomaly Detection and Smart Alerts beta are never part of the default
  install plan; the skill emits migration reports instead.

## Validation Rules

Validation must fail for:

- Unknown coverage statuses in `coverage-report.json`
- AI Toolkit install plans without a selected compatible PSC target
- PSC Linux 32-bit as a new install target; on Splunk 10.5 it is inventory and
  migration coverage only
- DSDL requested without AI Toolkit and PSC coverage in the same plan
- Direct-secret flags such as `--token`, `--api-token`, `--password`,
  `--client-secret`, or `--llm-api-key`
- Missing or unsupported upstream `product_stage` values in the rendered
  coverage report, including any report that fails to keep the Agent Launchpad
  surfaces and integrated CDTSM at `ga` and open CTSM 1.0 at `available` for
  the verified 2026-08-20 source baseline

Validation must warn for:

- DSDL Docker runtime in production because TLS, image provenance, and network
  isolation must be handled by the operator
- AI Toolkit/PSC versions lower than the latest audited compatibility pair
- Any attempted Splunk 10.5 install of legacy Anomaly Detection or Smart
  Alerts Assistant beta
- MLTK model objects created before the MLTK 5.3 compatibility break, which
  may need retraining
- Agent Launchpad requests without a confirmed supported Splunk Cloud region
  and egress allowlist entry, or without Splunk Cloud Connect on Enterprise

## References

- Read `reference.md` before changing product coverage, compatibility rules,
  generated artifacts, or live install behavior.
- Use `scripts/render_assets.py --discover` to print the built-in product
  catalog and coverage surface.