Neptune

---
name: neptune
description: "Amazon Neptune expert. Deep expertise in property graphs, RDF, Gremlin, SPARQL, openCypher, Neptune Analytics, Neptune ML, and graph query optimization. WHEN: \"Neptune\", \"Amazon Neptune\", \"Neptune Database\", \"Neptune Analytics\", \"Gremlin\", \"SPARQL\", \"openCypher\", \"Neptune ML\", \"Neptune Serverless\", \"Neptune graph\", \"RDF triple store\", \"Neptune notebook\"."
license: MIT
---

# Amazon Neptune

This skill covers Amazon Neptune with deep knowledge of property graph and RDF graph models, the Gremlin/openCypher/SPARQL query languages, Neptune Database (transactional workloads), Neptune Analytics (analytical workloads), Neptune ML (graph neural networks), data modeling for graph databases, bulk loading, streams (CDC), full-text search integration, and operational tuning. Neptune is a fully managed, purpose-built graph database engine optimized for storing billions of relationships and querying the graph with millisecond latency.

## How to Approach Tasks

When you receive a request:

1. **Classify** the request:
   - **Architecture/internals** -- Load `references/architecture.md`
   - **Performance diagnostics** -- Load `references/diagnostics.md`
   - **Configuration/operations** -- Load `references/best-practices.md`
   - **Comparison with other databases** -- see the `overview` skill

2. **Determine scope** -- Identify whether the question is about data modeling, query language (Gremlin/openCypher/SPARQL), Neptune Database vs. Neptune Analytics, ML integration, bulk loading, streaming, security, or operational troubleshooting.

3. **Analyze** -- Apply Neptune-specific reasoning. Reference the dual-graph model (property graph + RDF), query engine internals, storage architecture, replication model, and cost implications as relevant.

4. **Recommend** -- Provide actionable guidance with specific Gremlin traversals, openCypher queries, SPARQL queries, AWS CLI commands, HTTP API calls, or SDK patterns.

5. **Verify** -- Suggest validation steps (status endpoints, CloudWatch metrics, explain/profile, slow query logs).

## Core Expertise

### Dual Graph Model

Neptune supports two graph data models with three query languages:

**Property Graph Model:**
- Vertices (nodes) with labels and key-value properties
- Edges (relationships) with labels, direction, and key-value properties
- Queried via **Apache TinkerPop Gremlin** (traversal-based) or **openCypher** (declarative, pattern-matching)
- Data format for loading: CSV (Gremlin format) with `~id`, `~label`, `~from`, `~to` columns

**RDF (Resource Description Framework) Model:**
- Data expressed as triples: subject-predicate-object (SPO)
- Supports named graphs (quads: subject-predicate-object-graph)
- Queried via **W3C SPARQL 1.1**
- Data format for loading: N-Triples, N-Quads, Turtle, RDF/XML

**Choosing between models:**

| Factor | Property Graph | RDF |
|---|---|---|
| Data structure | Application-centric entities/relationships | Standards-based, linked data |
| Query style | Traversal / pattern matching | Declarative SPARQL, federated queries |
| Schema | Schema-optional | Ontology-driven (OWL/RDFS) |
| Interoperability | Application-specific | W3C standards, semantic web |
| Best for | Social networks, fraud, recommendations, knowledge graphs | Linked open data, regulatory ontologies, data integration |

**Important constraint:** Property graph and RDF data in the same Neptune cluster occupy separate storage spaces. You cannot query across models -- Gremlin/openCypher cannot read RDF triples and SPARQL cannot read property graph data.

### Gremlin Query Language

Gremlin is the traversal language of Apache TinkerPop. Neptune supports TinkerPop 3.6.x/3.7.x features.

**Core traversal steps:**

```groovy
// Add a vertex
g.addV('Person').property(id, 'p1').property('name', 'Alice').property('age', 30)

// Add an edge
g.addE('KNOWS').from(__.V('p1')).to(__.V('p2')).property('since', 2020)

// Pattern match: friends of friends
g.V('p1').out('KNOWS').out('KNOWS').dedup().where(neq('p1')).values('name')

// Shortest path
g.V('p1').repeat(out().simplePath()).until(hasId('p5')).path().limit(1)

// Aggregation: count vertices by label
g.V().groupCount().by(label)

// Conditional upsert (add or update)
g.V().has('Person', 'email', 'alice@example.com')
  .fold()
  .coalesce(
    unfold(),
    addV('Person').property('email', 'alice@example.com')
  )
  .property(single, 'name', 'Alice')
  .property(single, 'lastSeen', datetime())

// Subgraph extraction
g.V('p1').outE('KNOWS').subgraph('sg').inV().outE('KNOWS').subgraph('sg').cap('sg')

// Path with edge properties
g.V('p1').outE('KNOWS').inV().path().by('name').by('since')
```

**Neptune Gremlin specifics:**
- Neptune uses `id` as a string (not auto-generated integers). Always supply `property(id, 'your-id')` for predictable behavior.
- Neptune supports `datetime()` for temporal values (ISO 8601 strings internally).
- HTTP endpoint: `https://<cluster-endpoint>:8182/gremlin`
- WebSocket endpoint: `wss://<cluster-endpoint>:8182/gremlin`
- Sessions (transactions): Use WebSocket sessions for multi-statement transactions with `session` parameter.
- `g.tx().commit()` / `g.tx().rollback()` for explicit transaction control within sessions.
- Neptune does NOT support Gremlin `lambda` steps (security restriction).
- Neptune does NOT support `TinkerGraph` in-memory operations.

### openCypher Query Language

Neptune supports openCypher specification (based on Neo4j's Cypher). openCypher provides declarative pattern-matching syntax.

```cypher
-- Create nodes and relationships
CREATE (a:Person {name: 'Alice', age: 30})
CREATE (b:Person {name: 'Bob', age: 25})
CREATE (a)-[:KNOWS {since: 2020}]->(b)

-- Pattern match: friends of friends
MATCH (me:Person {name: 'Alice'})-[:KNOWS]->(friend)-[:KNOWS]->(foaf)
WHERE NOT (me)-[:KNOWS]->(foaf) AND me <> foaf
RETURN DISTINCT foaf.name

-- Variable-length path (1 to 5 hops)
MATCH p = (a:Person {name: 'Alice'})-[:KNOWS*1..5]->(b:Person {name: 'Eve'})
RETURN p

-- Aggregation
MATCH (p:Person)-[:WORKS_AT]->(c:Company)
RETURN c.name, count(p) AS employees
ORDER BY employees DESC

-- MERGE (upsert)
MERGE (p:Person {email: 'alice@example.com'})
ON CREATE SET p.name = 'Alice', p.created = datetime()
ON MATCH SET p.lastLogin = datetime()
RETURN p

-- UNWIND for batch operations
UNWIND [{name: 'Alice'}, {name: 'Bob'}] AS person
CREATE (p:Person) SET p = person

-- Shortest path
MATCH p = shortestPath((a:Person {name: 'Alice'})-[:KNOWS*..10]-(b:Person {name: 'Eve'}))
RETURN p
```

**Neptune openCypher specifics:**
- HTTP endpoint: `https://<cluster-endpoint>:8182/openCypher`
- POST with `query` parameter in application/x-www-form-urlencoded body
- Parameterized queries supported: `MATCH (n:Person {name: $name}) RETURN n` with `parameters` JSON
- Read/write queries routed appropriately in cluster mode
- Bolt protocol supported on port 8182
- Neptune supports `EXPLAIN` for openCypher queries (returns the query plan)
- Some Neo4j-specific Cypher features are NOT supported (APOC, GDS, full-text index syntax, CALL {} subqueries with side effects)

### SPARQL Query Language

See `references/sparql.md#sparql-query-language` — SPARQL 1.1 query examples (INSERT, SELECT, CONSTRUCT, named graphs, aggregation) and Neptune-specific SPARQL endpoint details.

### Neptune Database vs. Neptune Analytics

**Neptune Database** (the original service):
- Fully managed, transactional graph database cluster
- Instance-based: db.r5, db.r6g, db.x2g, db.serverless instance classes
- Storage: Auto-scaling up to 128 TB, 6-way replicated across 3 AZs
- Up to 15 read replicas for read scaling
- ACID transactions with read-committed isolation
- Millisecond latency for transactional graph queries
- Supports Gremlin, openCypher, SPARQL
- Use case: OLTP graph workloads (real-time recommendations, fraud detection, identity graphs)

**Neptune Analytics:**
- Serverless, in-memory analytical graph engine
- No instances to manage; specify memory (graph memory provisioned in GBs)
- Purpose-built for running graph algorithms and vector similarity search at scale
- Load data from Neptune Database snapshots, Amazon S3, or via openCypher queries
- Supports openCypher with built-in graph algorithm functions
- Built-in algorithms: PageRank, connected components, shortest path, community detection (Louvain, label propagation), betweenness centrality, closeness centrality, degree centrality, triangle count, weakly/strongly connected components
- Vector similarity search: store vector embeddings alongside graph data, KNN queries
- Use case: Graph analytics, algorithm execution, vector search on graph data

**Neptune Analytics algorithm syntax (openCypher extensions):**
```cypher
-- PageRank
CALL neptune.algo.pageRank()
YIELD node, score
RETURN node, score ORDER BY score DESC LIMIT 10

-- Community detection (Louvain)
CALL neptune.algo.louvain()
YIELD node, community
RETURN community, count(node) AS size ORDER BY size DESC

-- Shortest path (weighted)
CALL neptune.algo.shortestPath(
  {source: 'person/alice', target: 'person/bob', edgeWeightProperty: 'distance'}
)
YIELD path, totalWeight
RETURN path, totalWeight

-- Connected components
CALL neptune.algo.connectedComponents()
YIELD node, component
RETURN component, count(node) AS size ORDER BY size DESC

-- Vector similarity search
CALL neptune.algo.vectors.topKByNode(
  {node: 'product/123', k: 10, concurrency: 4}
)
YIELD node, score
RETURN node, score

-- Betweenness centrality
CALL neptune.algo.betweennessCentrality({samplingSize: 100})
YIELD node, score
RETURN node, score ORDER BY score DESC LIMIT 20
```

### Neptune Serverless

Neptune Serverless automatically scales compute capacity based on workload:

- **Neptune Capacity Units (NCUs):** Scaling unit. 1 NCU provides approximately 2 GiB of memory and associated CPU.
- **Configuration:** Set minimum and maximum NCU range (e.g., 1.0 to 128.0 NCUs).
- **Scaling speed:** Scales in seconds based on query demand.
- **Writer and readers scale independently.** Reader instances can scale to different NCU levels than the writer.
- **Cost model:** Pay per NCU-hour consumed. No charge when idle (scales to minimum NCUs).
- **When to use:** Variable or unpredictable workloads, development/test environments, applications with idle periods.
- **When NOT to use:** Steady-state high throughput (provisioned instances are more cost-effective), workloads sensitive to cold-start latency.

### Neptune ML

Neptune ML enables machine learning predictions on graph data using Graph Neural Networks (GNNs):

**Capabilities:**
- **Node classification** -- Predict labels/categories for nodes (e.g., fraudulent account detection)
- **Node regression** -- Predict numerical properties of nodes
- **Edge classification** -- Predict types/properties of edges
- **Edge regression** -- Predict numerical properties of edges
- **Link prediction** -- Predict missing or future edges (e.g., friend recommendations, drug interactions)

**Architecture:**
1. **Export** -- Neptune exports graph data to S3 using the Neptune Export service
2. **Data processing** -- SageMaker Processing job transforms graph data into GNN training features
3. **Model training** -- SageMaker trains a GNN model (DGL framework -- Deep Graph Library)
4. **Inference endpoint** -- SageMaker hosts the trained model
5. **Query integration** -- Gremlin/openCypher queries call the ML endpoint via `Neptune#ml.classification`, `Neptune#ml.regression`, or `Neptune#ml.linkPrediction` predicates

**Gremlin ML query examples:**
```groovy
// Node classification: predict fraud probability
g.V('account-123').properties('fraud_score')
  .with('Neptune#ml.classification')

// Link prediction: who might this user connect to?
g.V('user-456').out('KNOWS')
  .with('Neptune#ml.linkPrediction')
  .hasLabel('Person')
  .limit(10)

// Node regression: predict property value
g.V().hasLabel('House').has('neighborhood', 'downtown')
  .properties('predicted_price')
  .with('Neptune#ml.regression')
  .limit(5)
```

### Graph Data Modeling for Neptune

**Property graph modeling principles:**

1. **Nodes represent entities** -- Person, Product, Account, Transaction
2. **Edges represent relationships** -- KNOWS, PURCHASED, TRANSFERRED_TO
3. **Properties store attributes** -- name, amount, timestamp
4. **Labels categorize** -- Use node labels for entity types, edge labels for relationship types

**Common patterns:**

| Pattern | Description | Example |
|---|---|---|
| **Hub-and-spoke** | Central entity connected to many related entities | Customer -> Orders, Addresses, Payments |
| **Bipartite graph** | Two distinct node types connected by edges | Users -- PURCHASED --> Products |
| **Temporal edges** | Edges with timestamp properties for time-based traversals | TRANSFERRED_TO {amount: 500, date: '2025-03-15'} |
| **Hierarchical** | Tree or DAG structure using parent-child edges | Org chart, category taxonomy |
| **Hyperedge via node** | Represent N-ary relationships as intermediate nodes | Transaction node connecting sender, receiver, bank |

**Anti-patterns to avoid:**
- Storing large text/binary data as vertex properties (use S3 references instead)
- Using a single "generic" edge label for all relationships (e.g., "RELATED_TO")
- Creating supernodes with millions of edges without a partitioning strategy
- Modeling everything as properties when relationships would enable better traversals
- Using sequential numeric IDs for vertex IDs (use meaningful composite IDs like `person:alice`)

**RDF modeling principles:**
- Use well-known ontologies (FOAF, Schema.org, Dublin Core) when applicable
- Define a namespace for your domain: `@prefix myapp: <http://example.org/myapp/>`
- Use `rdfs:subClassOf` and `owl:ObjectProperty` for schema-level modeling
- Prefer IRIs over blank nodes for entities that need stable identity
- Use `xsd` datatypes for typed literals

### Bulk Loading

Neptune Bulk Loader is the fastest way to load large datasets:

**Property graph CSV format:**
```csv
# vertices.csv
~id,~label,name:String,age:Int
person:alice,Person,Alice,30
person:bob,Person,Bob,25

# edges.csv
~id,~from,~to,~label,since:Int
edge:e1,person:alice,person:bob,KNOWS,2020
```

**Loader API:**
```bash
# Start a bulk load job
curl -X POST "https://<cluster-endpoint>:8182/loader" \
  -H "Content-Type: application/json" \
  -d '{
    "source": "s3://my-bucket/neptune-data/",
    "format": "csv",
    "iamRoleArn": "arn:aws:iam::123456789012:role/NeptuneLoadFromS3",
    "region": "us-east-1",
    "failOnError": "FALSE",
    "parallelism": "OVERSUBSCRIBE",
    "updateSingleCardinalityProperties": "TRUE"
  }'

# Check load status
curl "https://<cluster-endpoint>:8182/loader/<load-id>"

# Cancel a load
curl -X DELETE "https://<cluster-endpoint>:8182/loader/<load-id>"
```

**Load formats supported:** CSV (Gremlin), N-Triples, N-Quads, Turtle, RDF/XML
**Source:** Amazon S3 only (must be same region as the Neptune cluster)
**Authentication:** IAM role attached to Neptune cluster with S3 read access

### Neptune Streams (Change Data Capture)

Neptune Streams captures a log of all changes to graph data:

- **Property graph stream:** `https://<endpoint>:8182/propertygraph/stream`
- **SPARQL stream:** `https://<endpoint>:8182/sparql/stream`
- Captures add/remove operations for vertices, edges, properties, and triples
- Events include commit timestamp and sequence numbers for ordering
- Retention: up to 7 days (configurable)
- Common integration: Lambda function polling the stream endpoint for CDC

**Stream response fields:**
- `commitTimestamp` -- When the transaction committed
- `eventId` -- Unique event identifier with commit number and operation number
- `op` -- Operation type: `ADD` or `REMOVE`
- `data` -- The changed element (vertex, edge, property, or triple)

### Full-Text Search Integration

Neptune integrates with Amazon OpenSearch Service for full-text search:

- Neptune streams data to OpenSearch automatically when configured
- Enables Gremlin/SPARQL queries that include full-text search predicates
- Gremlin: `has('description', TextP.containing('graph database'))`
- SPARQL: `neptune-fts:search` predicate
- Requires VPC peering between Neptune and OpenSearch, plus IAM configuration
- Supports fuzzy matching, wildcard queries, phrase matching, scoring

### Security

**Network isolation:**
- Neptune clusters run exclusively in Amazon VPC (no public endpoint)
- Access controlled via security groups and NACLs
- Cross-VPC access via VPC peering, Transit Gateway, or PrivateLink

**Authentication:**
- **IAM authentication** -- Recommended. All requests signed with SigV4.
- IAM policies control access at the cluster/action level (e.g., `neptune-db:ReadDataViaQuery`, `neptune-db:WriteDataViaQuery`)
- Condition keys: `neptune-db:QueryLanguage` (Gremlin, SPARQL, OpenCypher) for per-language access control

**Encryption:**
- **At rest:** AWS KMS encryption enabled at cluster creation (cannot be changed after)
- **In transit:** TLS/SSL enforced by default on all endpoints (port 8182)

**Audit logging:**
- Neptune audit logs can be published to CloudWatch Logs
- Captures all Gremlin, openCypher, and SPARQL queries with timestamps, source IP, query text, and execution time
- Enable via cluster parameter group: `neptune_enable_audit_log = 1`

### Neptune Notebooks

Neptune provides Jupyter notebook integration for interactive graph exploration:

- **Neptune Workbench** -- Managed Jupyter notebooks with pre-installed graph libraries
- `%%gremlin` magic command for Gremlin traversals
- `%%sparql` magic command for SPARQL queries
- `%%opencypher` / `%%oc` magic command for openCypher queries
- Built-in graph visualization using the `%%graph_notebook_vis_options` magic
- Support for `graph-notebook` Python library with `%seed`, `%graph_notebook_config`, `%load`, `%status`
- Network visualization with vis.js
- Can connect to Neptune from SageMaker notebooks as well

## Key Limits and Constraints

| Resource | Limit |
|---|---|
| Maximum cluster storage | 128 TB |
| Maximum read replicas | 15 |
| Maximum vertex/edge properties | No hard limit (practical: ~100K per element) |
| Maximum property value size | 55 MB |
| Maximum query timeout | 120 minutes (configurable) |
| Maximum bulk load file size | No limit (S3 multipart) |
| Maximum concurrent queries | Instance-dependent (hundreds on large instances) |
| Maximum parameters per query | 64 KB total parameter size |
| Minimum NCU (Serverless) | 1.0 NCU (2 GiB) |
| Maximum NCU (Serverless) | 128.0 NCU (256 GiB) |
| Neptune Analytics max memory | 4,096 GB |
| Supported engine versions | 1.2.x, 1.3.x (check current latest) |

## Quick Reference: Endpoint URLs

```
Status:              https://<endpoint>:8182/status
Gremlin:             https://<endpoint>:8182/gremlin      (POST JSON or WebSocket)
Gremlin Status:      https://<endpoint>:8182/gremlin/status
openCypher:          https://<endpoint>:8182/openCypher   (POST form-encoded)
openCypher Status:   https://<endpoint>:8182/openCypher/status
SPARQL:              https://<endpoint>:8182/sparql        (POST form-encoded)
SPARQL Status:       https://<endpoint>:8182/sparql/status
Loader:              https://<endpoint>:8182/loader        (POST JSON)
Stream (PG):         https://<endpoint>:8182/propertygraph/stream
Stream (RDF):        https://<endpoint>:8182/sparql/stream
System:              https://<endpoint>:8182/system
```

## When to Choose Neptune

**Choose Neptune when:**
- Your data is highly connected and you need to traverse relationships efficiently
- You need real-time graph queries (sub-second traversals across many hops)
- You need both transactional graph queries AND analytical graph algorithms
- You need a managed, multi-AZ, fully replicated graph database
- You need RDF/SPARQL compliance for linked data or ontology-driven applications

**Consider alternatives when:**
- Simple key-value or document lookups (DynamoDB, DocumentDB)
- Full-text search is the primary use case (OpenSearch)
- You need a single-server embedded graph (Neo4j Community)
- Analytics on tabular/columnar data (Redshift, Athena)
- Your graph fits in memory on a single machine and you want open-source flexibility (Neo4j, JanusGraph)