Nutanix

---
name: nutanix
description: "Nutanix AHV hyperconverged infrastructure expertise across all AOS versions: AHV (KVM-based hypervisor), Controller VM (CVM) architecture, Distributed Storage Fabric (DSF), Prism Element/Central management, acli/ncli CLI operations, Flow microsegmentation, Protection Domains, NearSync, Metro Availability, Leap DR orchestration, NCC health checks, storage containers, erasure coding, and cluster lifecycle management. Use for \"Nutanix\", \"AHV\", \"Prism\", \"acli\", \"ncli\", \"CVM\", \"DSF\", \"Nutanix cluster\", \"Protection Domain\"."
license: MIT
---

# Nutanix AHV

This skill covers Nutanix AHV hyperconverged infrastructure across all AOS versions. It covers:

- AHV hypervisor (KVM/QEMU-based with Nutanix management layer)
- Controller VM (CVM) architecture and service stack (Stargate, Cassandra, Curator, Zookeeper)
- Distributed Storage Fabric (DSF) with data locality, tiering, and replication
- Prism Element (single-cluster) and Prism Central (multi-cluster) management planes
- acli (AHV CLI) for VM lifecycle, migration, snapshots, and networking
- ncli (Nutanix CLI) for cluster, container, protection domain, and disk management
- nuclei and v3 REST API for Prism Central automation
- Storage containers (RF2/RF3, compression, dedup, erasure coding)
- Open vSwitch networking, IPAM, VLANs, and bond modes
- Flow microsegmentation (category-based policies, quarantine mode)
- Protection Domains (async DR, NearSync, Metro Availability)
- Leap DR orchestration (recovery plans, test failover, runbooks)
- NCC (Nutanix Cluster Check) health diagnostics
- Life Cycle Manager (LCM) for firmware and software upgrades

## How to Approach Tasks

When you receive a request:

1. **Classify** the request type:
   - **Troubleshooting** -- Load `references/diagnostics.md`
   - **Optimization** -- Load `references/best-practices.md`
   - **Architecture** -- Load `references/architecture.md`
   - **Administration** -- Follow the admin guidance below
   - **Data Protection** -- Apply Protection Domain and Leap expertise
   - **Scripting** -- Reference acli/ncli patterns and CVM shell access

2. **Identify version** -- Determine which AOS and AHV versions the user is running. Version affects feature availability (e.g., Flow requires Prism Central, NearSync requires AOS 5.x+).

3. **Load context** -- Read the relevant reference file for deep knowledge.

4. **Analyze** -- Apply Nutanix-specific reasoning, not generic KVM or virtualization advice.

5. **Recommend** -- Provide actionable guidance with acli, ncli, or Prism UI steps.

6. **Verify** -- Suggest validation steps (NCC checks, ncli queries, log review).

## Core Expertise

### Node Architecture

Each Nutanix node runs three software layers:

```
User VMs (QEMU/KVM guests)
Controller VM (CVM) -- Nutanix storage services
AHV Host (KVM + QEMU + OVS) -- Hypervisor layer
Physical Hardware (NVMe/SSD/HDD, NIC, CPU)
```

The CVM handles all storage I/O -- user VMs issue iSCSI/NFS I/O to the local CVM, which manages disk access via DSF. CVM has reserved CPU and memory (typically 32-48 GB RAM on production nodes).

### acli -- AHV CLI

acli is the primary CLI for AHV hypervisor operations. Run from any CVM or AHV host.

```bash
# VM listing and power operations
acli vm.list
acli vm.get <vm_name>
acli vm.on <vm_name>
acli vm.off <vm_name>
acli vm.shutdown <vm_name>          # graceful (requires NGT)

# Create and clone VMs
acli vm.create <vm_name> memory=4G num_vcpus=2 num_cores_per_vcpu=1
acli vm.clone <vm_name> clone_name=<new_name>

# Live migrate to a specific host
acli vm.migrate <vm_name> host_name=<ahv_host_name>

# Host maintenance mode
acli host.enter_maintenance_mode <host_ip>
acli host.exit_maintenance_mode <host_ip>

# Add disk and NIC
acli vm.disk_create <vm_name> container=<container_name> size=50G
acli vm.nic_create <vm_name> network=<network_name>

# Snapshots
acli vm.snapshot_create <vm_name> snapshot_name=<snap_name>
acli vm.snapshot_revert <vm_name> snapshot_name=<snap_name>
acli vm.snapshot_delete <vm_name> snapshot_name=<snap_name>
```

### ncli -- Nutanix CLI

ncli is the cluster-level management CLI for storage, protection domains, and health. Run from any CVM.

```bash
# Cluster overview
ncli cluster info
ncli cluster get-storage-info
ncli host list

# Storage containers
ncli container list
ncli container create name=<name> replication-factor=2 compression-enabled=true
ncli container edit name=<name> compression-enabled=true

# Protection domains
ncli protection-domain list
ncli protection-domain create name=<pd_name>
ncli protection-domain add-vms name=<pd_name> vms=<vm1>,<vm2>
ncli protection-domain add-schedule name=<pd_name> every=4hours retain=24

# Snapshots and replication
ncli snapshot list protection-domain-name=<pd_name>
ncli remote-site list

# Disk management
ncli disk list
ncli disk remove-start id=<disk_id>

# Alerts and health
ncli alert list
ncli cluster health-summary-get
```

### Storage -- Distributed Storage Fabric (DSF)

DSF provides data locality -- VM data resides on the same node where the VM runs. Writes go to the local CVM first, then replicate to a remote CVM. Reads served locally do not cross the network.

| Feature | Behavior |
|---------|----------|
| Replication Factor | RF2 (2 copies, 1 failure) or RF3 (3 copies, 2 failures) |
| Inline compression | LZ4 (fast) or ZSTD (high ratio) on first write |
| Post-process dedup | Fingerprint inline; dedup map applied post-process |
| Erasure Coding (EC-X) | Background re-encoding of cold data for capacity savings |
| Snapshots | Redirect-on-write -- instantaneous, no data copy |
| Clones | Space-efficient linked clones via block map sharing |

Storage tiers: NVMe/SSD (hot data, random I/O), HDD (cold data, sequential), S3-compatible (archive via Intelligent Tiering).

### Networking

AHV uses Open vSwitch (OVS) with `br0` as the default bridge. Physical NICs are bonded and uplinked into OVS bridges.

```bash
# Network management
acli net.list
acli net.create <name> vlan=100
acli net.create <name> vlan=200 ip_config=192.168.100.0/24 \
    ip_config.default_gateway=192.168.100.1 \
    ip_config.pool.0.range="192.168.100.50 192.168.100.200"
```

Bond modes: `active-backup` (failover), `balance-slb` (MAC hash), `LACP` (802.3ad).

Flow microsegmentation (Prism Central): stateful L4 policies enforced at the hypervisor vNIC level. Category-based (not subnet-based). Includes quarantine mode for instant VM isolation.

### Data Protection

| Mode | RPO | Mechanism |
|------|-----|-----------|
| Async DR | 1 hour minimum | Scheduled snapshot replication |
| NearSync | 1 minute (20-sec internal) | Shadow clone-based change tracking |
| Metro Availability | 0 (synchronous) | Synchronous write commit at both sites |

**Leap** (Prism Central) orchestrates failover and failback:
- Recovery Plans with ordered VM boot sequences and network mapping
- One-click failover/failback driven by Prism Central
- Non-disruptive test failover in isolated network bubble
- Runbook automation via Calm

### Prism Management Planes

| Plane | Scope | API |
|-------|-------|-----|
| Prism Element (PE) | Single cluster | v2 REST API |
| Prism Central (PC) | Multi-cluster | v3 REST API (intent-based) |

Prism Central adds Flow microsegmentation, Calm automation, Karbon Kubernetes, and multi-cluster analytics.

## Common Pitfalls

**1. Undersizing CVM resources**
CVM memory below 32 GB on production nodes causes storage performance degradation. Follow Nutanix sizing guidelines based on workload type and VM density.

**2. Ignoring data locality after VM migration**
Migration disrupts data locality. Curator re-converges data in the background, but heavy migration churn causes sustained cross-network I/O. Minimize unnecessary migrations.

**3. Using RF2 for production databases**
RF2 tolerates only one node failure. A second failure during rebuild causes data loss. Use RF3 for critical workloads, especially databases.

**4. Skipping NCC before upgrades**
Always run `ncc health_checks run_all` before any AOS, AHV, or firmware upgrade. Upgrading with unresolved warnings risks failure mid-upgrade.

**5. Mixing node hardware configurations within a cluster**
Non-uniform nodes cause unbalanced storage distribution. All nodes in a cluster should match hardware configuration (CPU, RAM, disk count/type).

**6. Not monitoring CVM service health**
Use `genesis status` and `allssh "genesis status"` to verify all services are running across all CVMs. A failed Stargate or Cassandra service degrades the entire cluster.

**7. Forgetting to configure remote sites before creating protection domain schedules**
Protection domain replication requires a configured remote site. Create the remote site first, then add schedules with retention policies.

**8. Running NearSync without sufficient bandwidth**
NearSync generates continuous replication traffic. Ensure dedicated replication network with adequate bandwidth or RPO targets will not be met.

**9. Not using categories for Flow policies**
Flow policies should use categories (tag-based VM grouping), not individual VM references. Category-based policies survive VM replacement and scaling.

**10. Ignoring LCM compatibility matrix**
AOS, AHV, NCC, and Prism Central versions must be compatible. Check the Nutanix compatibility matrix before upgrading any component.

## Reference Files

Load these when you need deep knowledge for a specific area:

- `references/architecture.md` -- AHV/CVM node architecture, DSF storage internals, networking/Flow, data protection (Protection Domains, NearSync, Metro, Leap), Prism management planes, CVM service stack. Read for "how does X work" questions.
- `references/diagnostics.md` -- acli/ncli diagnostic commands, NCC health checks, CVM service management, log locations, common issues table, upgrade troubleshooting. Read when troubleshooting errors or performance.
- `references/best-practices.md` -- Sizing guidance, RF selection, data locality optimization, NCC usage, upgrade order, container configuration, network design, backup strategy. Read for design and operations questions.