Glusterfs

---
name: glusterfs
description: "GlusterFS distributed filesystem: volume types (distributed, replicated, dispersed, arbiter), bricks, translators, self-heal, geo-replication, and Kubernetes integration via Kadalu. Use for \"GlusterFS\", \"Gluster\", \"gluster volume\", \"gluster peer\", \"brick\", \"self-heal\", \"split-brain\", \"geo-replication\", \"glusterd\", \"glusterfsd\", \"dispersed volume\", \"arbiter\", \"DHT\", \"AFR\", \"Kadalu\", \"gluster heal\"."
license: MIT
---

# GlusterFS

This skill covers GlusterFS distributed filesystem (current stable: 11.2), including:

- Volume types: distributed, replicated (2-way, 3-way), arbiter (2+1), dispersed (erasure coded), and combinations
- Bricks, Trusted Storage Pool (TSP), and peer management
- Translator (xlator) stack: DHT, AFR, EC, performance translators, POSIX
- Self-heal daemon, split-brain detection and resolution, GFID management
- Geo-replication via changelog-based gsync daemons
- FUSE, NFS-Ganesha, and SMB (Samba + vfs_glusterfs) access methods
- Volume snapshots (LVM thin and ZFS), quotas, and security
- Performance tuning for small-file, large-file, and metadata workloads
- Kubernetes integration via Kadalu operator
- Project status: community-maintained, Red Hat Gluster Storage EOL Dec 2024

For cross-platform storage comparisons, see the `overview` skill.

## How to Approach Tasks

1. **Classify** the request:
   - **Troubleshooting** -- Load `references/diagnostics.md` for heal status, split-brain resolution, brick failures, performance profiling, and log analysis
   - **Architecture / design** -- Load `references/architecture.md` for TSP, bricks, volume types, translators, FUSE, NFS/SMB gateways, geo-replication
   - **Best practices** -- Load `references/best-practices.md` for volume design, brick layout, performance tuning, monitoring, geo-rep setup, K8s integration

2. **Assess project status** -- GlusterFS is in maintenance mode. Red Hat Gluster Storage EOL December 2024. Core is community-maintained with security fixes. Recommend evaluating Ceph for new deployments requiring active development.

3. **Load context** -- Read the relevant reference file.

4. **Analyze** -- Consider volume type, replica count, brick layout, workload profile (small vs large files, random vs sequential).

5. **Recommend** -- Provide actionable guidance with gluster CLI commands.

## Core Architecture

### No Central Metadata Server

GlusterFS distributes metadata across bricks using DHT (consistent hashing) and extended attributes. No single point of failure for metadata.

### Volume Types

| Type | Description | Use Case |
|---|---|---|
| Distributed | Files spread across bricks via hash | Maximum capacity, no redundancy |
| Replicated (3-way) | Every file on all replicas | High availability, critical data |
| Arbiter (2+1) | 2 data + 1 metadata-only brick | HA with ~2x overhead (not 3x) |
| Dispersed (EC) | Reed-Solomon erasure coding | Storage efficiency + fault tolerance |
| Distributed Replicated | Distribution + replication | Most common production topology |
| Distributed Dispersed | Distribution + erasure coding | Scale + efficiency |

### Key Components

| Component | Role |
|---|---|
| glusterd | Management daemon; peer probing, volume config |
| glusterfsd | Brick process; one per brick per volume |
| glusterfs (client) | FUSE mount; full translator stack in userspace |
| Self-Heal Daemon (shd) | Repairs files missed during brick outage |
| gsync | Geo-replication daemon; changelog-based async replication |

### Translator Stack

```
Application -> VFS -> FUSE -> [Client: DHT -> AFR/EC -> Protocol/Client]
  -- network -->
[Server: Protocol/Server -> POSIX -> Underlying filesystem (XFS)]
```

## Critical Rules

- **Always use replica 3 (or arbiter 2+1) over replica 2** -- replica 2 cannot resolve split-brain automatically
- **Format bricks with XFS and isize=512** -- required for extended attribute storage
- **Brick order matters** in distributed-replicated volumes -- bricks grouped sequentially into replica sets
- **Never place two bricks of the same replica set on the same server**

## Reference Files

- `references/architecture.md` -- TSP, bricks, volume types, translators, FUSE mount, NFS/SMB gateways, geo-replication, self-heal, network ports
- `references/best-practices.md` -- Volume design, brick layout, performance tuning, monitoring with Prometheus, geo-rep setup, Kubernetes integration
- `references/diagnostics.md` -- Heal status, split-brain resolution, brick failures, performance profiling, log analysis, common error patterns