Openscientist Hypothesis

---
name: openscientist-hypothesis
description: Use when running or wiring OpenScientist for AI Gene Review gene-function hypotheses, especially blinded comparisons against local bioinformatics analyses.
metadata:
  short-description: AIGR OpenScientist hypothesis workflow
---

# OpenScientist Hypothesis Workflow

Use this skill for AIGR work that runs, stages, or maintains OpenScientist
hypothesis jobs for gene function review.

## Core Rules

- Treat OpenScientist as an independent bioinformatics scientist exploring a
  hypothesis: gene `G` has function `F`.
- For existing annotations, prefer the neutral function-assignment mode:
  `--as-function-hypothesis`.
- Do not feed existing local `*-bioinformatics/RESULTS.md` analyses into
  OpenScientist prompts. Those are holdout evidence for post-run comparison.
- It is acceptable to pass bounded non-holdout review context, especially the
  selected term, original evidence reference, and a small number of relevant
  literature/deep-research references.
- Avoid sending every function for genes with large reviews; select the specific
  hypothesis being tested.

## Running Jobs

- Use the `just` recipes, not raw `deep-research-client`, for AIGR hypothesis
  jobs. The justfile carries project defaults such as OpenScientist iteration
  count.
- Be patient with OpenScientist. Jobs may run quietly for a long time before the
  local report is written.
- Do not interrupt a real OpenScientist run merely because the local wrapper is
  silent or the child process is sleeping.
- Check completed and running jobs at <https://www.openscientist.io/jobs> before
  deciding a job is stuck.
- One-iteration OpenScientist runs are smoke tests only. Real hypothesis jobs
  should use the project default of at least 3 iterations unless the user
  explicitly asks otherwise.

## Timeouts

- Give analyses ample time. Real 3-iteration runs commonly take ~50-70 minutes,
  and structural / fold-discovery jobs can run longer; they are cancelled
  mid-analysis if the timeout is too short (observed: a job cancelled at the
  upstream 3600s default after producing nothing).
- The `just` recipes now inject the maximum OpenScientist job timeout
  (`--param timeout=7200`, i.e. 2h) alongside `--param max_iterations=3`, and
  the wrapper's subprocess wall (`--timeout-seconds`, default 8100s) is kept
  above the job timeout so the child process is not killed first. If you ever
  bypass `just` and call the script directly, pass both: e.g.
  `-- --param max_iterations=3 --param timeout=7200` plus
  `--timeout-seconds 8100`. The two timeouts are independent — raising one
  without the other still cancels the run.
- 7200s is the API ceiling: `OpenScientistParams.timeout` is validated `le=7200`,
  so a larger value fails fast with a pydantic `less_than_equal` error and writes
  nothing. Do not exceed it.
- Do not lower these to "save time"; a too-short timeout wastes the whole run.
- When a job *still* hits the 7200s ceiling (common for human proteins asked a
  multi-faceted question), the fix is **scope, not time**: re-run with a single
  decisive analysis in the hypothesis (e.g. "Foldseek fold assignment" alone,
  not fold + disorder + motif + targeting) and/or `--param max_iterations=2`.
  Observed: C18orf21 7200s timeout -> 2708s when narrowed to Foldseek-only;
  HSPA12A/DNAJC28 likewise completed once narrowed. The paralog of a job that
  *did* finish (e.g. HSPA12B at 5208s) will usually finish on a plain retry —
  variance, not a hard limit.
- Scoping each hypothesis to one computable question up front (rather than a
  broad "characterize this gene") both avoids timeouts and gives cleaner,
  more decisive provenance.

## Failure Modes

- A job can exit 0 yet write no report. Always confirm `openscientist.md` (and
  the `openscientist_artifacts/` dir) actually exist before treating a run as
  done; an empty hypothesis directory means it failed.
- Two distinct failure signatures seen, both leaving an empty output dir:
  - `detail=... timed out after Ns ... has been cancelled` -> raise the timeout
    (see above) and re-run.
  - `detail=... Request ID: req_...` -> a transient OpenScientist-internal LLM
    error (their server-side agent call failed; the id is for their debugging).
    Re-run; if the same gene fails this way repeatedly, treat it as a
    persistent upstream issue, move on, and report it rather than retrying
    indefinitely.
- Remove the empty `GENE-hypotheses/<slug>/` directory before re-running so a
  failed attempt does not masquerade as a completed one.

## After Completion

- Verify the generated report does not include withheld local bioinformatics
  paths or `RESULTS.md` content.
- Compare the OpenScientist conclusion against the held-out local bioinformatics
  result after the run.
- Keep operational lessons in skills or agent instructions, not in user-facing
  project pages.

## Wiring Verdicts Into Reviews

- **A refuted *proposal* must be deleted, not set to `REMOVE`.** Validation
  enforces that annotations absent from GOA carry `action: NEW` (they are AI
  proposals, identifiable by having no `original_reference_id`). If a run refutes
  such a proposal, delete the annotation block *and* any `core_functions` entry
  built on it, and record the refutation in the `description` + an OpenScientist
  `file:` reference. `action: REMOVE` is only for real GOA annotations and will
  fail validation on a not-in-GOA term ("Found N annotations not in GOA").
- **Gate findings that depend on uncacheable references.** When a run's
  conclusion rests on very recent papers not in `publications/` (so their
  `supporting_text` cannot be verbatim-verified) and that you cannot otherwise
  confirm, record it as a clearly-labelled *lead* — keep the conservative
  existing annotation, note "not yet verified / PMIDs not cached", and cite the
  OpenScientist `file:` report — rather than asserting a new annotation. Verify,
  don't trust. (Example: C18orf21 -> RMP24 / RNase MRP, left as a lead pending
  primary-literature caching.)
- **Don't overstate disagreements.** Before wiring "the run overturns the prior
  review", check the primary data: the two analyses often *agree on the facts*
  (e.g. residues present) and differ only on *interpretation* (e.g. a PROSITE
  signature failure). Wire genuine standoffs as `UNDECIDED` with both lines of
  evidence and the decisive experiment, and do not import the run's secondary
  claims (e.g. a named degenerate paralog) unless corroborated by repo data.
- **Verbatim `supporting_text` gotchas:** plain (unquoted) YAML scalars cannot
  contain `": "` — single-quote any quote containing a colon-space (e.g.
  "Foldseek: zero ..."). The pre-edit hook catches these.

## Job Hygiene

- Background jobs are killed on container restart; the harness reports which.
  Check the `GENE-hypotheses/<slug>/` dir — if empty, clean it and re-launch.
- Check whether the API key is already set before assuming it is not:
  `printenv OPENSCIENTIST_API_KEY` (it is exported in some environments, so the
  `just` recipes work directly). Only if it is absent, pass
  `OPENSCIENTIST_API_KEY` inline when calling the script directly, and wire it
  into the environment/setup config for reproducible `just` runs.