Birdclaw

---
name: birdclaw
description: "Search or sync the user's local X archive: posts, supplied URLs, DMs, people, links, and historical analysis."
---

# Birdclaw

Use the latest local snapshot first for X reads, including supplied URLs and recent content. Follow the [X data lookup order](../x-data/SKILL.md) for missing or stale evidence. For a requested or scheduled personal archive sync, read [incremental sync](references/sync.md).

## Data

Read through the installed `birdclaw` CLI or read-only SQLite at `~/.birdclaw/birdclaw.sqlite`. Lookup precedence lives in `x-data`; explicit `--mode bird` avoids Birdclaw's automatic paid fallback during live syncs.

Check basic health/freshness before analysis:

```bash
birdclaw --json db stats
```

```bash
sqlite3 ~/.birdclaw/birdclaw.sqlite "pragma quick_check;"
```

## Picking the Right Approach

Match the depth of read to the task:

- **Single fact / one tweet lookup** → SQL probe is fine.
- **DM identity lookup ("who is the X person?", "blacksmith guy")** → start with `birdclaw whois <query> --context 4 --no-xurl-fallback --json`. This searches local DMs, adds surrounding context, resolves archive numeric profiles from the persistent cache and `bird`, and avoids `xurl` unless you explicitly allow it.
- **DM search with context** → use `birdclaw search dms <query> --context 4 --resolve-profiles --expand-urls --no-xurl-fallback --json` when adjacent messages, profile names, or expanded `t.co` links matter.
- **Year vibe / theme summary** → CLI with `--originals-only --hide-low-quality`, full year, `--limit 20000`. One year at a time.
- **Life summary, biography, "movie script of my life", multi-year arc** → CLI per year across the full archive range, `--originals-only --hide-low-quality`, `--limit 20000` per year. Expect to ingest **50k+ tweets** total. Do NOT shortcut this with a top-N `like_count desc` SQL query — that yields only viral peaks and misses the everyday texture, recurring themes, and emotional tone the task needs.

Top-liked SQL slices are for spot-checking, not for vibe work. A 30-row `order by like_count desc` is the wrong tool for any task that asks for arc, narrative, or "what was X like."

## DM Identity Search

Prefer cached local-first commands before web/API:

```bash
birdclaw whois blacksmith --context 4 --no-xurl-fallback --json
```

```bash
birdclaw search dms "blacksmith" --context 4 --resolve-profiles --expand-urls --no-xurl-fallback --json
```

Caching model:

- profile resolution reads local `profiles`, then `sync_cache`, then `bird user`
- pass `--no-xurl-fallback` by default so enrichment cannot bypass the shared lookup order with a paid API call
- failed profile lookups are cached briefly to avoid repeated live calls
- URL expansion reads `sync_cache` first and mirrors results into persistent `url_expansions`; use `--refresh-url-cache` only when stale links matter
- resolved profiles preserve bio, profile URL, location, verification type, structured URL entities, raw profile JSON, and X affiliation badge metadata when available
- inspect `profileEvidence` in `whois --json` to separate `affiliation`, `bio_handle`, `bio_domain`, `bio_company`, `profile_url`, `profile_bio_url`, `profile_history`, `dm_context`, and `expanded_url` matches

How the richer identity evidence works:

- When the installed `bird --help` advertises `profiles` or `user --profile-only`, those commands can hydrate GraphQL profile URL entities and highlighted-label affiliations without the paid X API. Some builds lack these commands; check help and use the shared lookup order for unsupported profile reads.
- Birdclaw stores profile metadata on `profiles`, active organization/badge edges in `profile_affiliations`, profile-change history in `profile_snapshots`, and extracted bio identity hints in `profile_bio_entities`; backups include all four shards.
- Birdclaw also keeps a derived `identity_search_index` for fast local whois lookups. It is rebuilt from profile/bio/affiliation/history data and should not be treated as source-of-truth evidence.
- When X only gives a highlighted-label badge such as "Vercel" plus an org handle, Birdclaw first stores a deterministic synthetic org id, then resolves the handle through `bird` on a fresh profile hydration and rewrites the edge to the real local organization profile id when available.
- Bio entity extraction is first-class: bios/profile URLs/affiliations yield `@handle`, domain, and company-phrase rows. This is why `whois "blacksmith guy"` can rank someone from `@useblacksmith` and `blacksmith.sh` even if the exact phrase was not in the DM text.
- Profile snapshots are deduplicated by identity fields and affiliations. If a hydrated profile changes from "currently Vercel" to another bio/affiliation, `whois` can surface old matching values as `profile_history`.
- `whois` scores profile bio/name/handle matches, profile URL and bio URL matches, affiliation matches, bio entity matches, profile-history matches, DM context, and expanded `t.co` URLs separately. It ranks current affiliation and bio identity evidence above plain domains, distinguishes ecosystem labels such as "GitHub Star" from staff/company matches, and buckets human output into likely affiliated, ecosystem, profile/link, DM-context, and other matches.
- Use `--current-affiliation <org>` for strict active badge matches, `--affiliation <org>` for active/bio/history affiliation evidence, and `--exclude-domain-only` when a query like "GitHub people" should ignore accounts that only have `github.com` links.
- A cached rerun should show profile resolution from `local`/`sync_cache` and URL expansions from `cache`; use refresh flags only when current profile/bio/link evidence matters.

Use `--expand-urls` when `t.co` links are evidence. It may touch the network on cache miss, but it is not an X API call.

## Link Search

Use the persistent link index when looking for remembered shared tweets, videos, or `t.co` expansions:

```bash
birdclaw links backfill
```

```bash
birdclaw --json search links "the work" --source dm --media video --limit 50
```

Notes:

- `links backfill` indexes tweet/DM URL occurrences and expands missing/error/miss `t.co` rows; use `--refresh-url-cache` to force re-expansion.
- Default backfill indexes `t.co`; add `--all-urls` only when non-shortened links matter.
- `search links` matches short URLs, expanded URLs, linked tweet text/author, and source tweet/DM text.
- Link source-of-truth is `url_expansions` + `link_occurrences`; both are included in Git-friendly backups under `data/links/`.

## Year Analysis

For annual summaries, compare raw counts against summary-quality originals:

```bash
birdclaw --json search tweets --since 2020-01-01 --until 2021-01-01 --limit 20000
```

```bash
birdclaw --json search tweets --since 2020-01-01 --until 2021-01-01 --originals-only --hide-low-quality --limit 20000
```

Use exact date bounds: `YYYY-01-01` inclusive to next-year `YYYY-01-01` exclusive. Report counts and note archive gaps if stats show them.

When summarizing vibe:

- sample across the whole year, not just top-liked posts
- include a few representative paraphrases or short quotes
- separate recurring themes, emotional tone, work topics, jokes, travel/events, and relationship/community signals
- do not overfit one viral post

## Current Filters

`--originals-only` is separate from quality. It excludes authored replies using the current Birdclaw query contract.

`--hide-low-quality` maps to `qualityFilter: summary`. It hides common noise while preserving meaningful short posts:

- pure retweets
- low-like, no-media tiny posts under 16 characters after stripping `https://t.co/` URLs
- low-like short authored replies under 60 characters
- low-like short link captions under 45 characters when they only contain `t.co` links and no media

It should preserve:

- media-only posts
- high-like short posts
- normal link posts with meaningful caption text
- longer replies when replies are intentionally included

For full-year summary work, default to exact bounds:

```bash
birdclaw --json search tweets --since 2020-01-01 --until 2021-01-01 --originals-only --hide-low-quality --limit 20000
```

In the current implementation, "low-like" means `like_count < 50`.

## Designing Better Filters

Before changing thresholds, inspect real included and excluded examples.

Recommended checks:

- count how many tweets each proposed rule removes
- sample by year, not just one month
- keep a reason label per rule while tuning
- verify media-only and high-like posts survive
- verify link-only quote posts are removed only when they are low-signal
- add `--min-likes`, media flags, or debug reason output only when the use case needs it

Useful SQL sketch for rule tuning:

```bash
sqlite3 ~/.birdclaw/birdclaw.sqlite "
select id, created_at, like_count, text
from tweets
where created_at >= '2020-01-01' and created_at < '2021-01-01'
order by random()
limit 50;"
```