Fulcra Csv

---
name: fulcra-csv
description: Import any CSV stream into a Fulcra account as annotations — body weight, mood scores, expenses, sleep, media plays, anything timestamped. Use when the user wants to ingest a CSV they got from somewhere (a wearable, a Pipedream/IFTTT workflow, a hand-rolled spreadsheet) into Fulcra.
---

# fulcra-csv — Import any CSV into Fulcra

`fulcra-csv` is a small CLI that maps CSV columns onto Fulcra annotation events. It's the foundation `fulcra-media-helpers` uses for its `generic-csv` importer, but it works standalone for **any** kind of data: weights, moods, expenses, water intake, custom logs.

This skill teaches you (the AI agent) how to import a CSV into the right Fulcra annotation type for a given user data shape.

---

## Where to start — the re-entrancy probes

Before running an import, probe how far this user already got. The states are a prefix of
the import flow — **authed? → target def exists? → anything-to-import? → already landed?** — so
enter at the **first probe that fails** (per the repo's skill-quality pattern,
`docs/skill-quality-pattern.md`). Every state is safely re-enterable: re-running an import produces
the same deterministic source_ids, and the importer dedups against a readback of existing source_ids
before posting (client-side, in `run_import`; see "Critical invariant" below), so there is never a
penalty for re-probing or re-importing.

| Probe (run in order) | Command | Passes when | If it fails, enter at |
|---|---|---|---|
| Authed? | `fulcra auth print-access-token` | exits 0 and prints a non-empty token (the CLI mints/refreshes it; `FULCRA_ACCESS_TOKEN` in the env also satisfies this) | **AUTH** — tell the user to run `fulcra auth login` (interactive browser flow); see "Fulcra Life API auth" below |
| Target picked + def exists? | for a user-defined/generic annotation, list the account's live annotation definitions and confirm the intended UUID is present: `curl --oauth2-bearer "$(fulcra auth print-access-token)" https://api.fulcradynamics.com/user/v1alpha1/annotation` (this is the GET `annotations_catalog()` reads; `fulcra catalog` lists *data types*, not user-defined annotation defs, so don't use it here). For a built-in type (`--data-type BodyMass`, etc.), no def is needed — this probe is N/A | the intended `--definition-id` UUID appears in the JSON with `deleted_at` null, OR the user is targeting a built-in type | **BOOTSTRAP** — mint a def with `fulcra-csv bootstrap …` (prints the UUID), then import against it. Do NOT bootstrap if the UUID is already present — that mints a duplicate def and splits the data |
| Anything to import? | `fulcra-csv import <csv-path> --dry-run` **with a target flag** — `--definition-id <uuid>` (user-defined) or `--data-type <Name>` (built-in), plus the same column flags you'll use for the real run. `--dry-run` skips auth/ingest but the CLI still rejects the run with a UsageError if neither `--definition-id` nor `--data-type` is present (there'd be no target), so pass one even in dry-run | prints `parsed N events …` with N > 0 and the sampled rows look right (pure parse — no auth or ingest) | **EXPORT/COLLECT** — the file is empty or the column mapping is wrong; fix the flags or get a better file before importing |
| Already landed? | `fulcra-csv export --definition-id <uuid> --start "30 days ago" --columns start_time,source_id` (built-in target: pass `--data-type <Name>` instead) | rows come back that match the CSV you're about to import (compare `source_id` hashes or timestamps) | **IMPORT** — nothing landed yet; run `fulcra-csv import … ` for real (drop `--dry-run`) |

All probes pass → the data is already imported; tell the user and point them at
[Context Web](https://context.fulcradynamics.com) to browse it. A brand-new user fails the first
probe. Note the `fulcra auth print-access-token` command (probe 1, and reused inside the probe-2
curl) belongs to the separate [fulcra-api](https://github.com/fulcradynamics/fulcra-api-python) CLI;
`fulcra-csv` has no auth subcommand of its own. Probe 2 hits the Fulcra Life API annotation endpoint
(`GET /user/v1alpha1/annotation` on `https://api.fulcradynamics.com`, the host the fulcra-api client
is what this repo's fulcra-common client resolves from `FULCRA_API_BASE`; the fulcra-api lib derives the same host from its OIDC audience) directly with `curl` — the same GET the CLI's
`annotations_catalog()` performs — because neither `fulcra` nor `fulcra-csv` exposes a subcommand
that lists user-defined annotation definitions.

---

## Three target modes — pick first

Every import targets ONE of three places. Decide first which you're using:

### 1. User-defined annotation (most common)

You have (or will create) a custom Fulcra annotation definition. The CLI writes events under it. Pass `--definition-id <uuid>`.

```bash
fulcra-csv bootstrap --name "Mood" --description "Mood self-reports" \
  --annotation-type instant --value-type int --tag mood
# → prints the new UUID; save it
fulcra-csv import mood.csv --definition-id <uuid> \
  --annotation-type instant --ts-col timestamp --value-col score \
  --value-type int --note-col context
```

### 2. Built-in Fulcra type (BodyMass, HeartRate, etc.) — no definition-id

When the user wants the data to land in Fulcra's *native* time series (the same place HealthKit imports go), pass `--data-type <Name>` and SKIP `--definition-id`. The CLI doesn't append an annotation-def source to the record, so dedup is purely source-id-based and CSV imports can coexist with HealthKit imports of the same kind without duplicating.

```bash
fulcra-csv import weights.csv --data-type BodyMass \
  --annotation-type instant --ts-col date --value-col kg --unit kg \
  --tag manual-scale
```

⚠️ Built-in-type writes assume the receiving schema matches. Check `fulcra catalog` for known data types. As of this skill's writing, the data-type write API is forthcoming — when shipped, BodyMass/HeartRate/StepCount/etc. are first-class targets.

### 3. Generic DurationAnnotation / InstantAnnotation — custom definition

Create a simple annotation definition with `bootstrap`, then import against it with `--definition-id`. The CLI writes plain DurationAnnotation events (or InstantAnnotation with `--annotation-type instant`). Useful for "throw a CSV in and forget" cases where the data does not belong to a built-in Fulcra type.

```bash
fulcra-csv bootstrap --name "Imported CSV"
# -> save UUID as $CSV_UUID
fulcra-csv import random.csv --definition-id $CSV_UUID
```

---

## Annotation type — duration vs instant

`--annotation-type duration` (default) — events have `start_time` and `end_time`. For watches, listens, workouts (anything with a span). The CLI uses `--end-col` or `--duration-col`; falls back to a 1-second sentinel when neither is given (Fulcra silently drops zero-duration events).

`--annotation-type instant` — point-in-time. For weights, moods, single readings. `recorded_at` only has `start_time`. The `--end-col` / `--duration-col` flags error out if you pass them with `--annotation-type instant`.

---

## Value column — for measurements

If the row has a numeric reading (weight, score, count), use `--value-col <name>` to lift it into `data.value`. Coerce with `--value-type {float,int,str,bool}` (default float). Pair with `--unit <string>` to add a constant `data.unit`.

```bash
# Body weight in kg
fulcra-csv import weights.csv \
  --data-type BodyMass --annotation-type instant \
  --ts-col date --value-col kg --value-type float --unit kg \
  --tag manual-scale
```

Empty value cells become `None` (not zero, not the string `""`).

---

## Column mapping cheatsheet

| Flag | Purpose | Required? |
|---|---|---|
| `--ts-col` | Timestamp column header | Yes (default `timestamp`) |
| `--title-col` | Title column header | No (default `title`); used for both `note` and `title` |
| `--subtitle-col` | Subtitle (e.g. artist for music) | No; if set, `note` becomes `subtitle – title` |
| `--note-col` | Override note column | No |
| `--value-col` | Measurement value column | Only for value-bearing rows |
| `--value-type` | float/int/str/bool | Default `float`; only matters with `--value-col` |
| `--unit` | Constant unit string | Optional |
| `--end-col` | Explicit end-time column | Optional (duration only) |
| `--duration-col` | Duration-in-seconds column | Optional (duration only) |
| `--source-id-col` | Per-content id column (mixed into source-id hash) | Optional |
| `--tag-col` | Per-row tag column | Optional |
| `--tag` | Default tag for all rows | Optional |
| `--data-field COL=KEY` | Lift CSV column into `data.<key>` | Repeatable |
| `--extra COL=KEY` | Lift CSV column into `data.external_ids[<key>]` | Repeatable |
| `--tz` | IANA tz for naive timestamps | Default `UTC` |
| `--source-id-prefix` | Override deterministic id prefix | Default `com.fulcradynamics.csv.v1` |
| `--dry-run` | Parse + print first 5 rows; don't ingest | Optional |

---

## Bootstrap a new annotation definition

If the user wants a custom def, mint it first:

```bash
fulcra-csv bootstrap \
  --name "Concerts attended" \
  --description "Live music I went to" \
  --tag music --tag tickets
# → prints UUID
```

For measurement-bearing annotations, set `--annotation-type` and `--value-type`:

```bash
fulcra-csv bootstrap \
  --name "Resting Heart Rate (manual)" --description "Daily morning RHR" \
  --annotation-type instant --value-type int --unit bpm
```

Tags are auto-created if they don't exist.

---

## Soft-delete a definition

```bash
fulcra-csv soft-delete <uuid> --confirm
```

⚠️ **Fulcra has no per-event delete.** Soft-deleting a definition removes the def from the user's account but its events stay visible in queries with their `source_id` pointing at the deleted def. For a true "reset," soft-delete + create a new def with a different `source_id_prefix` so future imports namespace cleanly. The `fulcra-media` sibling has a `reset` command that wraps this for the four media defs (Watched/Listened/Activity/Read).

---

## Critical invariant: source_ids always include the timestamp

When `--source-id-col` is set, the column value is mixed into the hash **with** the timestamp, not used verbatim. Two plays of the same Spotify track at different times produce distinct events. Don't try to "preserve" content IDs in source_ids — they're hashed, idempotency is per-row, not per-content. (Content-level identity belongs in `--extra content_fingerprint=fp`.)

This is why **re-running the same import is always safe** — same input rows produce the same source_ids, and before posting each chunk `run_import` reads back the existing source_ids in that time window and skips the ones already present (client-side dedup, not a server-side guarantee). You don't need to track "have I imported this yet?"

---

## Recipes — common shapes

### Body weight (HealthKit-compatible)

```csv
date,kg
2026-05-01,82.4
2026-05-02,82.1
```

```bash
fulcra-csv import weights.csv \
  --data-type BodyMass --annotation-type instant \
  --ts-col date --value-col kg --value-type float --unit kg \
  --tag manual-scale
```

### Mood entries

```csv
timestamp,score,note
2026-05-01T09:00:00Z,7,morning coffee good
2026-05-01T22:00:00Z,5,long day
```

```bash
fulcra-csv bootstrap --name "Mood" --annotation-type instant \
  --value-type int --tag mood
# → save UUID as $MOOD_UUID

fulcra-csv import mood.csv --definition-id $MOOD_UUID \
  --annotation-type instant --ts-col timestamp \
  --value-col score --value-type int --note-col note
```

### Expenses with category tags

```csv
date,amount,merchant,category
2026-05-01,12.50,Blue Bottle,coffee
2026-05-01,38.00,Whole Foods,groceries
```

```bash
fulcra-csv bootstrap --name "Expenses" --annotation-type instant \
  --value-type float --unit usd --tag finance
# → save UUID

fulcra-csv import expenses.csv --definition-id $UUID \
  --annotation-type instant --ts-col date --value-col amount \
  --value-type float --unit usd \
  --title-col merchant --tag-col category
```

### Pipedream / IFTTT play log (music)

```csv
ts,track,artist,track_id,url
2026-05-01T09:00:00Z,Reelin' In The Years,Steely Dan,1I7zHEdDx8Ny5RxzYPqsU2,https://...
```

```bash
fulcra-csv import plays.csv --definition-id $LISTENED_UUID \
  --ts-col ts --title-col track --subtitle-col artist \
  --source-id-col track_id --tag spotify --extra url=spotify_url
```

### Sleep durations

```csv
start,end,quality
2026-05-01T23:00:00Z,2026-05-02T07:30:00Z,8
```

```bash
fulcra-csv bootstrap --name "Sleep" --tag sleep
fulcra-csv import sleep.csv --definition-id $UUID \
  --ts-col start --end-col end --value-col quality --value-type int
```

---

## Export — round-tripping annotations back to CSV

`fulcra-csv export` is the inverse of `import`. Reach for it when the user wants to:

- **Round-trip a CSV** — re-export what an import landed to verify columns/values look right, or to hand the data to another tool that expects CSV.
- **Audit a recent import** — pull the last day/week of a definition and eyeball it instead of running ad-hoc API queries.
- **Slice for a downstream tool** — pull a configurable subset of fields (well-known + `data.<key>` + `external_ids.<key>`) into a tidy CSV, optionally with epoch timestamps.

It is NOT a sync/backfill mechanism — it's a one-shot read. If the user wants ongoing sync, point them at a scheduled job that runs export on a window.

### Target — same model as import

Pass ONE of:

- `--definition-id <uuid>` — scope to a user-defined annotation. The CLI fetches the underlying data type (default `DurationAnnotation`) and filters records whose `sources` array references the target def. This mirrors the importer's dedup-readback, so what you see in export is what import would dedup against.
- `--data-type <Name>` — pull a built-in time series (`BodyMass`, `HeartRate`, `DurationAnnotation`, etc.).

`--start` is required (ISO-8601 or relative — `"1 week ago"`, `"yesterday"`). `--end` defaults to `now`.

### Column model — `--columns col1,col2,...`

Default: `start_time,end_time,tag,note,value`. Each entry resolves against the record in one of three ways:

| Form | Source |
|---|---|
| Well-known field (`start_time`, `end_time`, `note`, `title`, `value`, `unit`, `tag`, `tags`, `category`, `source_id`, `definition_id`, ...) | Top-level on the record, or normalised from `recorded_at` (timestamps), `tag_names` (tags), `sources` (source_id / definition_id). |
| `data.<key>` | The (possibly JSON-encoded) `data` payload — the place `--data-field COL=KEY` lifts to on import. |
| `external_ids.<key>` | `data.external_ids[<key>]` — symmetric to import's `--extra COL=KEY`. |

`source_id` is special: it returns the first non-definition source (the per-row dedup key), so you can round-trip dedup hashes back through.

### Other knobs

- `--date-format iso|epoch|local` (default `iso`, always UTC with `Z` suffix). `epoch` writes integer seconds. `local` honors `--tz`.
- `--tz <iana>` — used for parsing relative `--start`/`--end` and for `--date-format local`.
- `--out <path>` — file output. Omit for stdout.

### CSV-injection guard

By default, cells starting with `= + - @ \t \r` are prefixed with a single quote (`'`) so Excel/Sheets/Numbers don't interpret them as formulas. This is OWASP-grade defense-in-depth and on by default. The library exposes `ExportOptions.guard_formulas=False` for callers feeding CSV into a downstream parser that doesn't need it; the CLI does NOT expose a flag — keep it on for spreadsheets. Booleans render as lowercase `"true"`/`"false"` so they round-trip through `coerce_value`.

### Worked example — audit yesterday's mood imports

The user just ran `fulcra-csv import mood.csv --definition-id $MOOD_UUID ...` and wants to see what landed:

```bash
fulcra-csv export \
  --definition-id $MOOD_UUID \
  --start yesterday \
  --columns start_time,value,note,tag,source_id \
  --date-format local --tz America/New_York
```

Stdout is a CSV with five columns, timestamps in the user's wall-clock TZ, plus `source_id` so they can match rows against the importer's deterministic hashes.

### Pitfalls

- **Export does NOT trigger a re-fetch from the source.** It reads what Fulcra has. If a recent import is still propagating through ingest, you may see fewer rows than you posted — wait a minute and re-run.
- **`--definition-id` requires the underlying `data_type` to match.** The export defaults to `DurationAnnotation`; if the user bootstrapped an instant annotation, pass `--data-type InstantAnnotation` alongside `--definition-id`.
- **Don't promise byte-identical round-trips.** Booleans go to lowercase, lists/dicts JSON-encode to single-line, and the formula-guard `'` prefix is a real cell value. The data round-trips through `coerce_value`, but the CSV bytes differ from the original import file.

---

## Periodic ingestion

CSV imports are inherently one-shot ("import this file"), not incremental. There's no watermark layer (that lives in `fulcra-media` for its API-poll importers).

For agent-driven recurring sync of a CSV that grows over time (e.g. a Pipedream workflow appending rows to a Drive sheet):

1. Download the latest CSV to a temp file.
2. Run `fulcra-csv import <tempfile> ...` — re-imports are idempotent.
3. Skipped rows are silent (the importer's client-side source-id readback dedup). Posted rows are the delta.

The `--dry-run` flag is useful for a cheap "any new rows?" check (it parses and prints the first 5, doesn't ingest).

---

## Fulcra Life API auth

Same as fulcra-media: shells out to `fulcra auth print-access-token` from the [fulcra-api](https://github.com/fulcradynamics/fulcra-api-python) package, or honors `FULCRA_ACCESS_TOKEN=...` for non-interactive contexts.

---

## Common failure patterns

### "No definition_id found" exit

You forgot `--definition-id <uuid>` AND didn't set `--data-type` AND `--annotation-type` is duration. Either bootstrap a def, or pass `--data-type DurationAnnotation` explicitly.

### Events post but don't appear in `fulcra get-records`

Fulcra's ingest-to-query indexing lag (seconds to minutes for bulk imports). Re-run shouldn't help — wait a few minutes and re-query.

### Same content keeps producing duplicate events on each import

You're probably using `--source-id-col` for a per-row id (database row uuid) and expecting per-row uniqueness. The CLI ALWAYS mixes timestamp into the hash. For truly stable per-row identity, ensure the (timestamp, id) pair is stable across imports. If the row's timestamp changes between exports, you'll get distinct source_ids — that's intentional but maybe not what you want.

### "ValueError: unparseable timestamp"

dateparser can't parse the timestamp column. Try `--tz America/New_York` if rows are local-time-no-tz. Worst case, pre-process the CSV to ISO 8601.

---

## Architectural references (for the curious agent)

- `fulcra_csv.events` — `GenericEvent`, `ColumnMap`, `coerce_value`
- `fulcra_csv.parser` — `parse_csv(path, column_map, tz, annotation_type)`
- `fulcra_csv.fulcra` — `FulcraClient` (auth, ingest, dedup readback, per-chunk verification)
- `fulcra_csv.confidence` — `apply_cluster_policy`, `find_low_conf_twins`, `apply_twin_decisions` (used by fulcra-media's Trakt importer, but available standalone)
- `fulcra_csv.cli` — click-based CLI entry

---

## Don't

- **Don't bake `--definition-id` into a script as a constant** — the user might soft-delete + re-bootstrap, changing the UUID. Always read it from a fresh `fulcra-csv bootstrap` or from the user's state.
- **Don't try to dedup at the agent layer** — the importer's `run_import` already reads back existing source_ids per window and skips duplicates before posting. Just re-run the import; already-landed rows are skipped silently.
- **Don't write `data.note` from a column the user didn't intend to expose** — `--note-col` is opt-in.
- **Don't pass `--value-type bool` for "0/1" rows unless that's truly the schema** — bool collapses any non-truthy string to false; for diary-style yes/no events use int (0/1).
- **Don't print the user's access token to stdout** — neither in error envelopes nor in success messages. The CLI scrubs URL params in exception messages, but raw token blobs are sensitive.