Ref2scene

---
name: ref2scene
description: Rebuild a real place from one or a few reference photos as an editable, camera-matched Blender scene - decompose the photo into procedural and generated elements, write GPT Image 2.5 four-view prompts, check the views the user downloads, prepare Tripo web multi-view uploads, check the returned models, fit the camera from vanishing lines and a scale anchor, and build and verify the scene headless with numeric gates. Use when asked to reconstruct, recreate or match an interior or scene from photos in Blender, to make image-to-3D asset prompts or four-view (front/left/right/back) reference images for Tripo, or to fit a Blender camera to a photo.
---

# ref2scene: reference photos → Blender scene (stage-gated)

Toolkit: this repository (`tools/r2s.py`, `r2s/`). Requires Blender 5.2+ and Python with
numpy, pillow, scipy, opencv-python. Never open the Blender GUI; never call image or 3D generation
APIs unless the user asks (their GPT Image runs in ChatGPT, their Tripo on the website).

## Loop

1. `python tools/r2s.py new <project> --ref photo.jpg [--ref photo2.jpg]`, then work inside the
   project. Run `r2s status <project>` at the start of every session and after every stage.
2. **Inventory (your judgement).** Look at the photo(s). Write `inventory.json`
   (`templates/project/inventory.example.json`): one element per distinct thing, each with a route
   and a reason. Rules: architecture, anything that must line up with walls, and thin repeated
   members (slats, mullions, balusters: class `thin_repetitive`) are `procedural`; free-form
   furniture and plants are `generate`; one design = one asset reused for every instance. For
   generated assets give `class`, `size` (+ `basis`), `instances[].bbox` and `.placement`, and the
   prompt slots: say what "front" is in physical terms, state counts of repeated parts.
3. `r2s prompts` → hand the user `prompts/README.md`: one NEW ChatGPT chat per asset, attach the
   photo and `prompts/<id>_crop.png`, paste `prompts/<id>.txt`, download the four images unrenamed
   into `01_asset_images/<id>/raw/`.
4. `r2s views` → read `reports/views.json`, look at every `contact.png`. FAIL → ask for a
   regeneration with `prompts/<id>_front_first.txt`. Folder with retries → check the ranked
   `candidates.json`, then `--pick-best` or write `views.json` with `"mapping_source": "manual"`.
5. `r2s tripo` → hand the user `prompts/tripo/README.md`. They upload the four files by slot label
   and download into `02_models/<id>/`.
6. `r2s models` → read `reports/models.json`, look at every `turntable.png`. A failed part count
   means: switch the element to `procedural`. A yaw/left-right warning: decide from the turntable,
   then set `"apply_yaw_offset": true` in params if the correction is right.
7. `r2s lines` → look at `04_renders/compare/lines_<ref>.png`; cite real depth (y), vertical (z)
   and across (x) lines by id in `03_project/landmarks.json`; add height anchors (bottom/top pixel
   + an assumed size with its basis) and table-top ellipses (centre, a, b) if any. `r2s camera`:
   RMS ≤ 2 px, look at the overlay, drop landmarks the report names as worst if they are wrong.
8. Write `03_project/params.json` (room, assets, placements with `uv` = floor contact pixel) and
   grow `03_project/scene.py` (template: `scene_kit`). `r2s build --check gray` until every
   placement's box IoU passes, then materials and lights, `--check lit`, `r2s compare`.
9. Deliver `05_output/<name>.blend` (packed), the checkpoint renders, `asset_manifest.json`, and a
   short list of assumptions (scale anchor, hidden geometry) and remaining differences.

## Pitfalls

- "Left" = the object's own left: its front points to the image's LEFT edge (Tripo's slots, GPT Image).
- Never trust a vision judgement over a failed numeric check; never fix a FAIL by loosening its
  threshold without recording why in `docs/VERIFICATION.md`.
- Keep every accepted change in `params.json`/`scene.py`; the reopened `.blend` is what is checked.
- Blender's Python has no PIL/scipy: image work stays in `r2s/*.py`.