Whatsapp Sync

---
name: whatsapp-sync
description: >
  On-demand refresh of WhatsApp text into the Obsidian vault (data layer + dashboard + group
  labels + contact notes). TEXT ONLY — never downloads media. Pulls the recent window the live
  companion bridge exposes, not the full multi-year archive. Semi-auto by design: it drives the
  live main-number bridge and needs the phone nearby. Trigger on "/whatsapp-sync", "pull
  whatsapp".
license: MIT
---

OBJECTIVE: Refresh the WhatsApp slice of the Second Brain — pull the recent text the live bridge exposes, rebuild the SQLite+FTS5 data layer, re-resolve names, re-label active groups, rebuild the dashboard, and refresh the vault notes (MOC + groups + optionally contact notes). Fully deterministic except the group-label step (Sonnet). Idempotent: `build_db.py` is a full rebuild from `raw_train/`, so re-runs never duplicate.

CONTEXT:
- Live bridge = `@oss_maintainer/whatsapp-mcp` (Baileys), main number "Tony PaloAlto ResearchLab" (jid `16213388980@s.whatsapp.net`). Memory [[whatsapp-mcp-integration]] has the full setup + pairing playbook.
- Pipeline home: `$IMPORTS_ROOT/whatsapp/`. Data: `whatsapp_train.db` (SQLite+FTS5) + `raw_train\` (JSON). Vault home: `$OBSIDIAN_VAULT/01-Conversations/WhatsApp/` (`_WhatsApp-MOC.md`, `_WhatsApp-Groups.md`, 9 contact notes). Dashboard: `_Dashboards\WhatsApp-Dashboard.html`.
- SCOPE: **TEXT ONLY.** Never `download_media`; ignore the `hasMedia` flag.
- Model routing: the group-label step is grunt classification → **Sonnet subagent** (per [[model-routing-sonnet-grunt]]). Contact-note summaries (CRM intel, not Anton's authorial voice) → Sonnet draft acceptable under the quality gate; escalate to Opus if weak.

⚠️ HARD SAFETY (corrected model, proven 2026-06-16):
- A 2nd client doing READ-ONLY (list_chats/list_messages) does NOT trigger AUTH_KEY_DUPLICATED — the server degrades it to read-only and coexists. Only WRITE ops (resolve_contacts) need sole-client (else they error harmlessly into read-only). So a read pull can run alongside the registered MCP.
- On Windows `subprocess.terminate()/.kill()` does NOT reliably kill the node child → zombies ACCUMULATE (found 11 once). Any spawn script must `taskkill /F /T /PID <its-own-pid>` in a finally block (see `nightly_pull.py`). The nightly twin does this; it kills ONLY its own child, never the registered server.
- Gentle on the main number ([[telegram-safety]] sibling): read-only, modest pacing; ban-risk was accepted consciously (variant A).

STEPS:
1. PULL-MODE decision. Call `mcp__whatsapp__get_my_profile`.
   - If it returns the profile → bridge is LIVE → use **PULL-LIVE** (step 2a). NEVER spawn `train_pull.py`.
   - If the tool is absent/errors (headless) → use **PULL-SPAWN** (step 2b).
2a. PULL-LIVE (preferred, no double-client risk):
   - `mcp__whatsapp__list_chats {limit:100}` → the chat list (jid, name, isGroup).
   - For EACH chat: `mcp__whatsapp__list_messages {jid, limit:50}` (bump to 100 only if a chat is high-value). Collect {id, from, fromMe, type, text, timestamp, hasMedia}. Pace gently.
   - Assemble `$IMPORTS_ROOT/whatsapp/live_pull.json` = `[{jid,name,isGroup,messages:[...]}, ...]` (Write tool) and run `python ingest_live.py` (→ raw_train/ + train_summary.json, the format build_db eats).
2b. PULL-SPAWN (headless only): pre-flight kill stray node, then `python train_pull.py` (it spawns ONE temporary client, waits ~45s for history sync, writes raw_train/ + train_summary.json), then hard-kill node.
3. BUILD DATA: `python build_db.py` (drops + rebuilds `whatsapp_train.db` from raw_train/ — idempotent; categorizes; named=0/1).
4. NAMES (DMs): if PULL-SPAWN, `python names_fix.py` (resolve_contacts + re-snapshot chats2.txt). If PULL-LIVE, optionally call `mcp__whatsapp__resolve_contacts {resync:true}` then re-list to upgrade numeric DM names. (Group subjects: the server is PATCHED to fetch real group subjects via groupMetadata on `get_chat` — after the next MCP restart `list_messages`/`get_chat` return real names; until then groups are labeled by content in step 6.)
5. DASHBOARD: `python build_dash_export.py` (→ `_Dashboards\WhatsApp-Dashboard.html` + `valuable_chats.json`).
6. GROUP LABELS (active groups, n_mine≥3):
   - `python extract_active_groups.py` (→ `active_groups.json`, compact content samples, 0 tokens).
   - Spawn ONE **Sonnet** subagent (Agent tool, model:'sonnet') to read `active_groups.json` and write `group_labels.json` = `[{jid,label,category,lang,confidence,one_line}]` (categories: work-business|household-community|project-windmill|family-personal|crypto-web3|longevity-health|services-vendors|other). No media, no WhatsApp tools.
   - `python apply_group_labels.py` (writes labels into DB, named=2 = INFERRED) then re-run `python build_dash_export.py` so the dashboard shows the labels.
6.5 GRAPH-LINK (Rail 1 people + Rail 2 concepts — the [[relink-mechanism]] applied to WhatsApp; rich path only):
   - `python link_people.py` (0 tokens): phone-join WhatsApp DM jid (=phone `last10`) → `apple-contacts\contacts.db` → `vault_matches` → CRM/person note → `people_matches.json`. Phone match = T1 (trust); name-only = T2 (DO NOT trust — surname-blind false positives).
   - For T2 candidates spawn a **Sonnet judge** (Agent, model:'sonnet') → `people_verified.json` (conservative: confirm only on surname+role match, else null = WA note stays canonical). Identity-critical: a wrong link corrupts the graph.
   - Concepts: create/confirm any NEW topic-concept (windmill-park, etc.) per concept-creation-rules (DUP-CHECK first — e.g. household already = `concept-bible-household`/`concept-bible-staff-hr`).
   - `python link_apply.py` (idempotent): writes "## 🔗 Graph" into each WA note (verified person/CRM + concepts) + back-links into the 4 rich targets = BIDIRECTIONAL. Run AFTER vault_backup. Verify 0 broken targets.
7. VAULT (BACKUP FIRST — [[vault-backup-rule]]): `python $IMPORTS_ROOT/vault_backup.py`, then:
   - `python build_groups_note.py` (→ `_WhatsApp-Groups.md`).
   - Refresh `_WhatsApp-MOC.md` counts if chat/msg totals changed. (Contact notes: refresh only if a key chat changed materially — keep open-action-items current; that's the high-value part.)
8. REINDEX (RAG): rely on the nightly Brain Reindex @04:00, or `python $IMPORTS_ROOT/brain_embed_update.py --wait-gpu 10` if Anton wants it searchable now.
9. REPORT: chats/msgs pulled, named vs ✎-inferred counts, new/changed open-action-items flagged for Anton, dashboard path. End with a 🧒 In plain words recap (messages TO Anton only).

CONSTRAINTS:
- WINDOWS cp1252: never print Cyrillic to python stdout (crashes) — scripts write UTF-8 files; keep stdout ASCII (counts only). ([[deterministic-script-gotchas]])
- The bridge has NO history pagination (`list_messages` limit≤100, no offset) — this refreshes only the recent window, NOT all years. Full archive = separate phone-backup path (variant B, not built). State this honestly; don't imply completeness.
- Re-applying the server group-subject patch: it lives in node_modules `dist\whatsapp.js` (getChat) — re-apply after any `npm update @oss_maintainer/whatsapp-mcp` (documented in [[whatsapp-mcp-integration]]).

ALWAYS-MODEL (Anton 2026-06-16): graph-linking is NOT optional — it is how we ALWAYS work with WhatsApp. Split by cost:
- **NIGHTLY auto (0 tokens, deterministic): Windows Task "WhatsApp Nightly Sync" @ 03:00 daily** = `nightly_sync.cmd`: `nightly_pull.py` → build_db → apply_group_labels (re-applies persistent `group_labels.json`) → dashboard → groups note → **`link_people.py` → `link_apply.py`** (re-weaves KNOWN verified people from `people_verified.json`, idempotent) → vault_backup. Keeps data + dashboard + known labels + known people-links fresh every night.
- **MANUAL/weekly (LLM): THIS skill** does the judgment-needing parts — Sonnet labels brand-NEW groups, the Sonnet judge confirms identity of NEW people (appends to `people_verified.json`), and contact-note open-items get refreshed. So: new people/groups enter via the rich path; the nightly twin then maintains them forever, free. Single source of truth = the scripts in `_imports\whatsapp\` + `people_verified.json`.

RELATION (do not duplicate):
- First-time setup + pairing + gotchas + the nightly task: memory [[whatsapp-mcp-integration]] (single source of truth). Automation registry: [[automation-inventory]].
- Sibling sync skills: [[health-sync]] / [[faaa-sync]] / [[telegram-reimport]] (same on-demand-refresh architecture). Routine policy: [[evaluate-recurring-into-routine]] — nightly cheap data twin + this richer manual LLM path.

---

<!-- CONTACT-FOOTER -->
## About & contact

Built and battle-tested at **Palo Alto AI Research Lab** — a fleet of Claude Code machines
running 24/7 as a second brain and synthetic cofounder. Every skill here survived real
production use before publication.

- 📦 All 101 skills: https://github.com/tonydzi/second-brain-starter-kit
- 👤 Author: **Anton Dziatkovskii** — Telegram [@tonydzi](https://t.me/tonydzi) · WhatsApp [+1 341 222 9178](https://wa.me/13412229178) · X [@Tony_Stef_](https://x.com/Tony_Stef_)
- 🧪 **Engineers: want to test-drive this setup?** Message me — I hand out free starter seeds to engineers who test and report back. Custom skill requests welcome.