Scylladb

---
name: scylladb
description: "ScyllaDB technology expert covering ALL versions. Deep expertise in shard-per-core architecture, CQL compatibility, performance tuning, cluster operations, and Cassandra migration. WHEN: \"ScyllaDB\", \"Scylla\", \"scylla.yaml\", \"nodetool scylla\", \"shard-per-core\", \"seastar\", \"ScyllaDB Cloud\", \"ScyllaDB Enterprise\", \"Alternator\", \"scylla-manager\", \"CQL Scylla\", \"tablets scylladb\", \"sctool\", \"scylla_setup\"."
license: MIT
---

# ScyllaDB

This skill covers ScyllaDB across all supported versions (2025.1 LTS through 2026.1 LTS, plus legacy 6.x open-source). It covers ScyllaDB's shard-per-core architecture (Seastar framework), CQL compatibility with Apache Cassandra, Alternator (DynamoDB-compatible API), data modeling, compaction strategies, cluster management, ScyllaDB Manager, ScyllaDB Monitoring Stack, performance tuning, and Cassandra-to-Scylla migration. For version-specific detail, see the matching file under `references/versions/`.

## When to Use This Skill vs. Version-Specific Guidance

**Use this agent when the question spans versions or is version-agnostic:**
- "How does ScyllaDB's shard-per-core architecture work?"
- "Design a data model for a time-series workload on Scylla"
- "Tune compaction for a write-heavy Scylla cluster"
- "Set up multi-datacenter replication in ScyllaDB"
- "Migrate from Cassandra to ScyllaDB"
- "Explain ScyllaDB scheduling groups and workload prioritization"
- "Compare Alternator vs native CQL for my use case"

**See the matching version reference when the question is version-specific:**
- "ScyllaDB 2026.1 vector search with filtering" --> `references/versions/2026.1.md`
- "ScyllaDB 2026.1 native backup to GCS" --> `references/versions/2026.1.md`
- "ScyllaDB 2025.1 tablets default behavior" --> `references/versions/2025.1.md`
- "ScyllaDB 2025.1 strongly consistent topology" --> `references/versions/2025.1.md`

## How to Approach Tasks

When you receive a request:

1. **Classify** the request:
   - **Architecture/internals** -- Load `references/architecture.md`
   - **Performance diagnostics** -- Load `references/diagnostics.md`
   - **Configuration/operations** -- Load `references/best-practices.md`
   - **Version-specific feature** -- See the matching `references/versions/<v>.md` file
   - **Comparison with other databases** -- see the `overview` skill

2. **Determine version** -- Ask if unclear. Behavior differs significantly across versions (e.g., tablets default-on in 2025.1+, unified codebase in 2025.1+, vector search filtering only in 2026.1+).

3. **Analyze** -- Apply ScyllaDB-specific reasoning. Reference the shard-per-core model, the per-shard resource ownership, scheduling groups, compaction mechanics, and consistency trade-offs as relevant. Remember: ScyllaDB is C++ native -- there is no JVM, no garbage collection pauses, no heap tuning.

4. **Recommend** -- Provide actionable guidance with specific scylla.yaml parameters, CQL statements, nodetool commands, REST API calls, or sctool commands.

5. **Verify** -- Suggest validation steps (nodetool tablestats, REST API queries, CQL system table queries, Grafana dashboard checks, Prometheus metrics).

## Core Expertise

### Shard-Per-Core Architecture (Seastar Framework)

ScyllaDB is built on the Seastar framework, a C++ asynchronous programming framework that implements a shared-nothing, shard-per-core design. This is the fundamental architectural difference from Cassandra (JVM-based):

- **One application thread per CPU core** -- Each core runs an independent shard. No shared memory between shards. No locks, no context switches, no contention.
- **Each shard owns dedicated resources:**
  - Its own memory allocation (no shared heap, no GC pauses)
  - Its own memtables and SSTables
  - Its own cache partition
  - Its own network connections (via DPDK or POSIX)
  - Its own I/O scheduler queue
- **Inter-shard communication** via explicit message passing (futures/promises), never shared state.
- **Userspace I/O scheduling** -- Seastar bypasses the Linux page cache with direct I/O (O_DIRECT) and manages its own read/write queues per shard, enabling precise I/O prioritization.
- **Userspace task scheduler** -- Cooperative multitasking with continuation-passing. No OS thread scheduling overhead.
- **Linear scalability** -- Doubling cores produces approximately double throughput, since shards are independent.

**Why this matters vs. Cassandra:**
| Aspect | Cassandra (JVM) | ScyllaDB (Seastar/C++) |
|---|---|---|
| GC pauses | Yes (G1, ZGC, Shenandoah) | None -- no GC |
| Memory management | JVM heap + off-heap | Direct memory control per shard |
| Thread model | Thread pool, shared state | One thread per core, shared nothing |
| I/O model | JVM NIO, OS page cache | Direct I/O, userspace scheduler |
| CPU utilization | Often < 50% due to GC, locks | Near 100% utilization typical |
| Tail latency (p99) | Higher due to GC pauses | Consistently low |
| Capacity planning | 3-10x Scylla nodes for same throughput | Fewer nodes needed |

### Data Distribution: Tablets vs Vnodes

ScyllaDB supports two data distribution mechanisms:

**Vnodes (legacy, pre-6.0 default):**
- Each node owns multiple token ranges on a consistent hash ring
- Default `num_tokens: 256` (recommended: 256)
- Topology changes require streaming large amounts of data
- Same concept as Cassandra vnodes

**Tablets (default in 2025.1+):**
- Dynamic, fine-grained data distribution units
- A table is split into tablets, each tablet is a unit of replication and migration
- Tablets can split (when a table grows) and merge (when a table shrinks)
- Tablet migration is fast -- individual tablets move between nodes independently
- Enables true elastic scaling -- add/remove nodes with minimal data movement
- Each tablet has its own replication group
- Counter support added in 2026.1

**Tablets vs Vnodes:**
| Aspect | Vnodes | Tablets |
|---|---|---|
| Granularity | Token ranges per node | Per-table tablet units |
| Scaling speed | Slow (stream all ranges) | Fast (migrate individual tablets) |
| Add multiple nodes | Sequential | Simultaneous |
| Mixed instance types | Difficult | Supported (tablets auto-balance) |
| Topology changes | Heavy streaming | Lightweight tablet migration |
| Counter support | Yes | Yes (2026.1+) |

### CQL Compatibility with Cassandra

ScyllaDB implements CQL (Cassandra Query Language) and is wire-protocol compatible with Apache Cassandra. Existing Cassandra drivers work with ScyllaDB with minimal or no changes:

**Fully compatible:**
- CQL data types (all standard types, UDTs, collections, frozen types)
- Primary key structure (partition key + clustering columns)
- Consistency levels (ONE, QUORUM, LOCAL_QUORUM, ALL, etc.)
- Lightweight transactions (IF NOT EXISTS, IF conditions)
- Batches (logged and unlogged)
- TTL and tombstone mechanics
- Keyspace replication strategies (SimpleStrategy, NetworkTopologyStrategy)
- Prepared statements
- Paging

**ScyllaDB-specific CQL extensions:**
- `BYPASS CACHE` -- Read without polluting the cache
- `USING TIMEOUT` -- Per-query timeout override
- Workload prioritization via service levels
- `PRUNE MATERIALIZED VIEW` -- Remove orphan MV rows

**Key behavioral differences from Cassandra:**
- Tombstone handling is per-shard, compaction is per-shard
- No JBOD support -- ScyllaDB uses RAID-0 across disks (or one disk per shard)
- Hinted handoff is managed differently (hints stored per-shard)
- Repair is row-level (not token-range-level like Cassandra pre-4.0)

### Alternator (DynamoDB-Compatible API)

Alternator is ScyllaDB's DynamoDB-compatible API, allowing DynamoDB applications to run on ScyllaDB with no code changes:

**Supported operations:**
- CreateTable, DeleteTable, DescribeTable, UpdateTable, ListTables
- PutItem, GetItem, UpdateItem, DeleteItem
- BatchGetItem, BatchWriteItem
- Query, Scan with FilterExpressions
- Global Secondary Indexes (GSI), Local Secondary Indexes (LSI)
- DynamoDB Streams (experimental)
- Conditional expressions, projection expressions, update expressions
- TTL (Time to Live)
- Tagging

**Configuration (scylla.yaml):**
```yaml
alternator_port: 8000            # HTTP port
alternator_https_port: 8043      # HTTPS port
alternator_address: 0.0.0.0      # Listen address
alternator_write_isolation: always_use_lwt  # or forbid, only_rmw_uses_lwt
```

**When to use Alternator vs CQL:**
- Use Alternator when migrating an existing DynamoDB application to self-hosted ScyllaDB
- Use CQL for new applications -- it provides full ScyllaDB feature access and lower overhead
- Alternator has slightly higher overhead due to JSON parsing and DynamoDB protocol translation

### Data Modeling

ScyllaDB data modeling follows the same query-driven methodology as Cassandra, with additional ScyllaDB-specific considerations:

**Methodology:**
1. Identify all access patterns (queries)
2. Design one table per query pattern
3. Choose partition key for even distribution and query satisfaction
4. Choose clustering columns for sort order within partition
5. Denormalize aggressively

**ScyllaDB-specific partition sizing guidance:**
- Target < 100MB per partition (same as Cassandra)
- But ScyllaDB handles large partitions better due to per-shard processing
- Monitor with `nodetool tablestats` -- look at `Maximum partition size`
- Large partition warnings in scylla.yaml:
  ```yaml
  compaction_large_partition_warning_threshold_mb: 100
  compaction_large_row_warning_threshold_mb: 10
  compaction_large_cell_warning_threshold_mb: 1
  ```

**Time-series example with time-bucketing:**
```cql
CREATE TABLE metrics.readings (
    sensor_id    text,
    day          date,
    reading_time timestamp,
    value        double,
    PRIMARY KEY ((sensor_id, day), reading_time)
) WITH CLUSTERING ORDER BY (reading_time DESC)
  AND compaction = {'class': 'TimeWindowCompactionStrategy',
                    'compaction_window_size': 1,
                    'compaction_window_unit': 'DAYS'}
  AND default_time_to_live = 7776000;  -- 90 days
```

### Compaction Strategies

ScyllaDB supports four compaction strategies. Unlike Cassandra, compaction runs per-shard, which eliminates cross-shard coordination:

| Strategy | Best For | Space Overhead | Write Amp | Read Amp |
|---|---|---|---|---|
| **STCS** (SizeTiered) | General write-heavy | Up to 2x temp | Low | Higher |
| **ICS** (Incremental) | Replaces STCS (recommended) | ~10-15% temp | Low | Lower than STCS |
| **LCS** (Leveled) | Read-heavy, update-heavy | ~10% overhead | Higher | Low |
| **TWCS** (TimeWindow) | Time-series with TTL | Low | Lowest | Low |

**Incremental Compaction Strategy (ICS) -- ScyllaDB-specific, recommended over STCS:**
- Breaks large SSTables into sorted runs of fixed-size fragments (default 1GB)
- Same read/write amplification as STCS but only ~10-15% temporary space (vs STCS 2x)
- Compacts incrementally -- a large SSTable does not need to be fully rewritten
- Always prefer ICS over STCS for new deployments

```cql
-- Set ICS on a table
ALTER TABLE my_ks.my_table WITH compaction = {
    'class': 'IncrementalCompactionStrategy',
    'sstable_size_in_mb': 1024
};

-- Set TWCS for time-series
ALTER TABLE my_ks.timeseries WITH compaction = {
    'class': 'TimeWindowCompactionStrategy',
    'compaction_window_size': 1,
    'compaction_window_unit': 'HOURS'
};
```

**Selection guidance:**
```
Write-heavy, general purpose        --> ICS (not STCS)
Read-heavy, frequent updates        --> LCS
Time-series with TTL                --> TWCS
Legacy or migrating from Cassandra  --> STCS (then migrate to ICS)
```

### Cluster Management

**Cluster configuration (scylla.yaml):**
```yaml
cluster_name: 'my_cluster'
listen_address: 10.0.1.1
rpc_address: 10.0.1.1
seed_provider:
  - class_name: org.apache.cassandra.locator.SimpleSeedProvider
    parameters:
      - seeds: "10.0.1.1,10.0.1.2,10.0.1.3"
endpoint_snitch: GossipingPropertyFileSnitch
num_tokens: 256
data_file_directories:
  - /var/lib/scylla/data
commitlog_directory: /var/lib/scylla/commitlog

# Scylla-specific
api_port: 10000                     # REST API port
api_address: 127.0.0.1              # REST API listen address
prometheus_port: 9180               # Prometheus metrics port
developer_mode: false               # NEVER true in production
```

**Topology operations:**
```bash
# Add a node -- start scylla with correct seeds and cluster_name
# For tablets-enabled clusters (2025.1+), multiple nodes can join simultaneously

# Decommission a node
nodetool decommission

# Remove a dead node (from a live node)
nodetool removenode <host-id>

# Check cluster status
nodetool status

# Check topology operation progress
nodetool describecluster
```

### ScyllaDB Manager (sctool)

ScyllaDB Manager automates repair and backup operations across clusters:

**Cluster management:**
```bash
# Add a cluster to manager
sctool cluster add --name my-cluster --host 10.0.1.1 --auth-token <token>

# List managed clusters
sctool cluster list

# Check cluster status
sctool status -c my-cluster
```

**Repair operations:**
```bash
# Create a recurring repair task
sctool repair -c my-cluster --interval 7d

# Create repair for specific keyspace
sctool repair -c my-cluster -K my_keyspace --interval 7d

# List repair tasks
sctool task list -c my-cluster --type repair

# Monitor repair progress
sctool task progress -c my-cluster <task-id>

# Update repair parameters
sctool repair update -c my-cluster <task-id> --parallel 2
```

**Backup operations:**
```bash
# Create a backup task
sctool backup -c my-cluster --location s3:my-bucket --interval 24h --retention 7

# Backup specific keyspace
sctool backup -c my-cluster -K my_keyspace --location s3:my-bucket

# List backups
sctool backup list -c my-cluster --location s3:my-bucket

# Restore from backup
sctool restore -c my-cluster --location s3:my-bucket --snapshot-tag <tag>

# Delete old backups
sctool backup delete -c my-cluster --location s3:my-bucket --snapshot-tag <tag>
```

### ScyllaDB Monitoring Stack

See `references/operations.md#scylladb-monitoring-stack` — Prometheus + Grafana + Loki container stack, key Grafana dashboards, and setup steps.

### Performance Tuning (No JVM -- C++ Native)

ScyllaDB auto-tunes aggressively on startup via `scylla_setup` and `perftune.py`. There is no JVM heap sizing, no GC tuning, no jvm.options file. Key tuning areas:

**System preparation (scylla_setup):**
```bash
# Run the setup wizard -- handles kernel parameters, IRQ affinity, disk config
sudo scylla_setup

# Or run individual components:
sudo /opt/scylladb/scripts/scylla_sysconfig_setup  # sysconfig
sudo /opt/scylladb/scripts/scylla_io_setup          # I/O scheduler calibration
sudo /opt/scylladb/scripts/perftune.py              # IRQ affinity, NIC tuning
```

**Key scylla.yaml performance parameters:**
```yaml
# Memory allocation (per node, Scylla auto-calculates per shard)
# By default Scylla takes all available RAM minus ~1.5GB for OS
# Override only if needed:
# --memory <amount> in scylla.args or SCYLLA_ARGS

# I/O properties (auto-detected by scylla_io_setup)
# /etc/scylla.d/io.conf or /etc/scylla.d/io_properties.yaml

# Compaction throughput (MB/s per shard, 0 = unlimited)
compaction_throughput_mb_per_sec: 0

# Compaction enforcement
compaction_enforce_min_threshold: false

# Concurrent reads/writes (auto-tuned, rarely need override)
# These are per-shard limits

# Row cache -- enabled by default, uses available memory
# Scylla manages cache memory automatically per shard
```

**Scheduling groups (workload prioritization):**
ScyllaDB groups internal tasks into scheduling groups with configurable CPU shares:
- `statement` -- CQL reads and writes (default priority)
- `compaction` -- Compaction tasks
- `streaming` -- Repair streaming, bootstrap streaming
- `gossip` -- Gossip protocol
- `memtable_to_cache` -- Memtable flush to cache

Workload prioritization via service levels:
```cql
-- Create service levels (requires ScyllaDB Enterprise / 2025.1+)
CREATE SERVICE LEVEL gold WITH timeout = '5s' AND workload_type = 'interactive';
CREATE SERVICE LEVEL silver WITH timeout = '30s' AND workload_type = 'batch';

-- Attach service level to a role
ATTACH SERVICE LEVEL gold TO 'app_user';
ATTACH SERVICE LEVEL silver TO 'analytics_user';
```

### Cassandra-to-Scylla Migration

See `references/operations.md#cassandra-to-scylla-migration` — migration paths (SSTableLoader, Spark Migrator, dual-write), pre-migration checklist, and shard-aware driver configuration.

### ScyllaDB Cloud

ScyllaDB Cloud is the fully managed service:
- **Serverless** -- Pay per operation, auto-scaling
- **Dedicated** -- Reserved instances, full control over configuration
- Available on AWS, GCP, Azure
- Supports both CQL and Alternator APIs
- Managed backups, repairs, monitoring
- VPC peering for private connectivity

### Security

**Authentication:**
```yaml
# scylla.yaml
authenticator: PasswordAuthenticator
```
```cql
CREATE ROLE admin WITH PASSWORD = 'strongpassword' AND LOGIN = true AND SUPERUSER = true;
CREATE ROLE app_user WITH PASSWORD = 'apppass' AND LOGIN = true;
```

**Authorization:**
```yaml
# scylla.yaml
authorizer: CassandraAuthorizer
```
```cql
GRANT SELECT ON KEYSPACE my_ks TO app_user;
GRANT MODIFY ON TABLE my_ks.events TO app_user;
```

**Encryption:**
```yaml
# Client-to-node encryption
client_encryption_options:
    enabled: true
    certificate: /etc/scylla/db.crt
    keyfile: /etc/scylla/db.key
    truststore: /etc/scylla/ca.crt

# Node-to-node encryption
server_encryption_options:
    internode_encryption: all
    certificate: /etc/scylla/db.crt
    keyfile: /etc/scylla/db.key
    truststore: /etc/scylla/ca.crt
```

**Note:** ScyllaDB uses PEM-format certificates (not Java keystores like Cassandra).

### Consistency Levels

ScyllaDB supports the same consistency levels as Cassandra:

| Consistency Level | Behavior | Use Case |
|---|---|---|
| `ONE` | 1 replica responds | Logging, low-latency reads |
| `QUORUM` | RF/2 + 1 respond | Strong consistency |
| `LOCAL_QUORUM` | Majority in local DC | Multi-DC standard |
| `EACH_QUORUM` | Majority in each DC | Cross-DC strong (writes only) |
| `ALL` | All replicas | Rarely used |
| `SERIAL` / `LOCAL_SERIAL` | Paxos quorum | LWT reads |

**Strong consistency formula:** `R + W > RF` (identical to Cassandra).

## Common Pitfalls

1. **Running scylla_setup in developer mode** -- `developer_mode: true` disables I/O tuning, memory locking, and other production optimizations. Never use in production.

2. **Not using shard-aware drivers** -- Standard Cassandra drivers work but route all requests through one shard per connection. ScyllaDB-specific drivers route directly to the owning shard, reducing latency by avoiding inter-shard forwarding.

3. **Using STCS instead of ICS** -- ICS provides the same write amplification as STCS but with dramatically less temporary space overhead. Always use ICS for new ScyllaDB deployments.

4. **JVM tuning habits from Cassandra** -- There is no JVM. No heap sizing, no GC tuning, no jvm.options. ScyllaDB manages memory per-shard automatically.

5. **Not running scylla_setup** -- ScyllaDB requires kernel tuning (IRQ affinity, NIC queue assignment, I/O scheduler calibration). The `scylla_setup` script automates this. Skipping it causes severe performance degradation.

6. **Hot partitions** -- Same as Cassandra. Design for high-cardinality partition keys. Monitor with `nodetool tablestats` and the Scylla Monitoring per-table dashboard.

7. **Ignoring scheduling groups** -- Batch analytics queries competing with interactive reads. Use service levels and workload prioritization to isolate workloads.

8. **Not monitoring with the Scylla Monitoring Stack** -- The built-in dashboards and advisor provide critical insights. Running without monitoring is operating blind.

9. **Large partitions** -- Partitions > 100MB cause compaction pressure and high per-shard memory usage. ScyllaDB logs warnings for partitions exceeding `compaction_large_partition_warning_threshold_mb`.

10. **Neglecting repair** -- Same as Cassandra. Use ScyllaDB Manager to schedule recurring repairs within gc_grace_seconds.

## Version-specific guidance

| Version | Status | Key Features | Reference |
|---|---|---|---|
| **ScyllaDB 2026.1** | Current LTS | Vector search filtering/quantization, counter tablets, native GCS backup, native S3 restore | `references/versions/2026.1.md` |
| **ScyllaDB 2025.1** | Supported LTS | Tablets default, tablet merge, strongly consistent topology/auth/service levels, unified codebase, source-available | `references/versions/2025.1.md` |
| **ScyllaDB 6.x** | Legacy (open-source EOL) | Tablets introduced (6.0), zero-token nodes (6.2), Alternator RBAC (6.2) | Not covered (upgrade recommended) |

## Reference Files

Load these when you need deep knowledge for a specific area:

- `references/architecture.md` -- Shard-per-core internals, Seastar framework, write path, read path, SSTable format, memory management, I/O scheduler, scheduling groups, tablets architecture, gossip protocol, failure detection, hinted handoff, repair, streaming. Read for "how does ScyllaDB work internally" questions.
- `references/diagnostics.md` -- 100+ nodetool commands, REST API diagnostics, CQL system table queries, sctool commands, Prometheus metrics, Grafana dashboard queries, per-shard diagnostics, troubleshooting playbooks. Read when troubleshooting performance, cluster health, or operational issues.
- `references/best-practices.md` -- Data modeling methodology, partition sizing, cluster sizing, hardware selection, scylla.yaml tuning, compaction strategy selection, repair scheduling, backup strategies, monitoring setup, security hardening, multi-DC deployment, Cassandra migration, ScyllaDB Cloud configuration. Read for configuration and operational guidance.