Review Function Prediction

---
name: review-function-prediction
description: >
  Review computational protein or gene function predictions and fill PredictionReview
  YAML files using the COR/CNN/LSP/UNC/PLI/NPI/REP biological-validity taxonomy.
  Use for ProtNLM, DeepECTF, BioReason/GO-GPT, InterPro2GO, PANTHER/IBA, CLEAN,
  GloEC, MAPred, ProteinInfer, or other predicted EC/GO annotations, including
  audits of the evidence and reasoning in existing prediction reviews.
---

# Review Function Prediction

Use this skill to review computational function predictions against allowed evidence and write schema-compatible `PredictionReview` YAML.

the predictions are in YAML alongside the main curation review YAMLs:

`genes/<TAXON>/<GENE>/<GENE>-<TOOL>-predictions-review.yaml`

```yaml
id: <UniProt accession>
gene_symbol: <gene symbol>
locus_tag: <locus tag if available>
taxon:
  id: NCBITaxon:<taxon id>
  label: <organism>
status: COMPLETE
description: >-
  Brief summary of the prediction review outcome, based only on the allowed
  evidence for this review mode.
source_documents:
  - <allowed file path used>
predictions:
  - source_method: <do not change>
    source_version: <do not change>
    source_reference_id: <do not change if present>
    predicted_term:
      id: <do not change>
      label: <do not change>
    predicted_term_type: <EC | GO_MF | GO_BP | GO_CC>
    review:
      assessment: <COR | CNN | LSP | UNC | PLI | NPI | REP>
      confidence_score: <0 | 1 | 2>
      error_type: <omit unless applicable>
      summary: >-
        Explain the decision in 3-8 sentences. State what allowed evidence was
        used, whether the prediction is already present in curated annotations,
        whether it is more or less specific than existing annotations, and why
        the chosen category is justified. If using UNC, explicitly say what
        evidence is missing and avoid speculation.
```

## Selected records with no GO/EC predictions

When a selected benchmark target has no emitted GO/EC predictions in the frozen
source, still write its `PredictionReview` sidecar with an explicit
`predictions: []`. Use `description` for the summary evaluation: identify the
method and snapshot, distinguish absent GO/EC output from any emitted name,
location, or function text, and assess whether well-supported functions were
omitted. Cite the frozen output and decisive biological sources through
`source_documents` and `references`, with supporting excerpts in reference
`findings` where applicable. `status: COMPLETE` can describe a completed
assessment of the empty output; it does not imply the full gene review is complete.

Use exact-accession and isoform evidence before calling an omission an obvious
miss. Unresolved substrate specificity or absent catalytic architecture can make
withholding a term reasonable. Do not insert expected terms into `predictions`,
assign a COR/CNN/LSP/UNC/PLI/NPI/REP score to an absent prediction, or infer recall
without a defined expected annotation set. Report zero-output records separately
from emitted-claim assessment counts. A missing sidecar means unrecorded review,
whereas `predictions: []` plus a completed summary records an assessed absence.

If the user requests a cohort of genes **with GO predictions**, enforce that
selection criterion against the actual source. Function text, protein names,
and subcellular-location identifiers do not satisfy it.

## Evidence and reasoning

Separate biological correctness, agreement with existing annotations, and the
quality of the evidence supporting the judgment. A plausible conclusion can have
an inadequate rationale; repairing the rationale need not change the conclusion.

- **Do not use ARBA assertions to bolster correctness.** An ARBA-derived GO term,
  UniProt keyword, or functional description is an association-based prediction,
  not independent biological validation. It may be recorded as annotation
  provenance or as a comparison result. Agreement with ProtNLM does not establish
  correctness. This is not necessarily circular; the problem is treating an
  unreliable predicted assertion as validating evidence. If the assertion leads
  to useful underlying evidence, inspect and cite that evidence instead.
- **Assess AI-assisted sources by their evidential content.** A generated gene
  description or accepted/core-function label adds no support by repeating a
  conclusion; use it to locate the underlying evidence. An OpenScientist
  investigation that integrates sequence, structural, evolutionary, and literature
  evidence is a substantive analytical synthesis and can carry substantial weight
  in adjudication. Cite the report directly in `references` and `supported_by`,
  identify its decisive findings and reasoning, and inspect the underlying sources
  or analysis artifacts for consequential claims where available. Distinguish
  reported computations from independently reproduced results and experimental
  validation. Evaluate limitations and disagreements claim by claim; neither accept
  a bare verdict as proof nor dismiss an integrated investigation because AI
  produced it. Do not count the report and its underlying sources as independent
  replications. A `source_documents` entry alone does not explain evidential weight.
- **Anchor decisive claims with source excerpts.** Include concise, verbatim
  `supporting_text` for sequence features, domain assignments, experiments, and
  analytical findings used in the rationale. Use explicit ellipses between
  noncontiguous excerpts. A report's recommendation is not a substitute for the
  findings used to reach or qualify the assessment.
- **Inspect provenance within database records.** "UniProt says" is insufficient
  when the relevant text or keyword was generated by ARBA or another predictor.
  Distinguish experimental findings, curator-assessed inferences, sequence/domain
  observations, and automated functional assertions. Do not count several
  restatements of the same inference as independent lines of evidence. This does
  not equate PAINT/IBA's curated phylogenetic judgments with ARBA associations;
  assess an IBA through its ancestral assertion and experimental grounding.
- **Family-based inference is legitimate when the transfer is justified.** Identify
  the target's supported family/subfamily placement, cite the characterized
  relatives and their relevant findings, and explain why the property is expected
  to be conserved. State that this is an inference rather than an observation on
  the target. Consider relevant divergence, paralog differences, sequence features,
  and taxonomic context without inventing exceptions or requiring a new experiment
  on every target. Strong, well-grounded transfer can support correctness; a
  generic domain label or an unsupported family stereotype cannot.
- **Match evidence to the exact claim.** Evidence for extracellular localization
  does not by itself establish extracellular-matrix residence, matrix structural
  activity, or matrix organization. Evaluate each prediction's specificity and
  mechanism separately. For the OLFML2A example, a defensible rationale would
  identify verified family/sequence evidence and relevant localization findings in
  characterized relatives, then explain transfer to the target with its limits.
  Quoting an AI description of a "matricellular regulator" supplies none of those
  missing steps. This example does not pre-adjudicate either ECM prediction.
- **Distinguish activity from participation.** Loss of a catalytic domain can
  refute an intrinsic enzyme activity without refuting participation in the
  corresponding biological process through a complex or regulatory role. A
  scaffold need not catalyze the reaction to participate in that process.
- **Separate limited usefulness from biological error.** A broad but true term
  is not incorrect merely because a more informative annotation is available.
  Use `LSP` only when a supported, more specific annotation actually exists;
  functions in different GO aspects are not automatically parent/child terms.
  Annotation overlap alone does not establish training-data contamination, and
  a common predicted term alone does not establish frequency bias.

Use the evidence scope requested for the evaluation. In an unrestricted review,
follow citations to publications and reproducible analyses. In
curated-annotation-only mode, do not silently broaden the evidence scope, and do
not substitute ARBA or AI prose when permitted evidence is insufficient. State
what remains unverified and use `UNC` when the exact prediction cannot be supported
or refuted. Never invent supporting quotes or recast a paraphrase as a quotation.

### Auditing existing prediction reviews

For each prediction, trace the rationale's decisive claims to their underlying
sources. Check for ARBA used as validation, AI descriptions used as authority,
repeated inferences presented as independent support, and family-level evidence
stretched to a more specific function. Keyword searches can find candidates but
cannot adjudicate them; an ARBA mention may simply document provenance.

Replace weak reasoning with a source-grounded explanation where possible. Retain
the assessment if that evidence supports it; use `UNC` if support remains
insufficient, and an incorrect category only when justified by contrary evidence.
Do not automatically downgrade a reasonable family inference or mark a prediction
incorrect just because its original rationale was poor. Synchronize the file's
overall description with its final per-prediction judgments. Preserve the model's
original prediction and source metadata.

Descriptions, summaries, explanations, and other substantive YAML fields must
state the current best assessment and its evidence, not the sequence of revisions.
Do not write "previously", "reclassified", "on re-review", or accounts of changed
minds into those fields. Put the old/new assessments, reasons for changing them,
and audit provenance in the separate append-only history mechanism described in
[`docs/history.md`](../../../docs/history.md), using `just new-history` to scaffold
records. Notes fields may carry review-process context where appropriate.

## Validation

Validate sidecars with `just validate-predictions <file> [<file> ...]`. This checks
the PredictionReview schema, score/category agreement, source paths, publication
titles, and supporting excerpts, fetching missing PMID caches. CI runs the same
check for changed sidecars and publishes `prediction-evidence-validation` results.
The recipe requires excerpts for every supporting-source entry, and local-file
excerpt segments must follow source order without reusing the same passage.
Missing source text cannot be certified as verified. The check tests citation
accuracy; biological relevance and inference quality still require review.

## Assessment Categories

Use these categories for `review.assessment`.

**COR: correct novel prediction.** The predicted function is supported by allowed evidence and is not already present in UniProt/GOA as an equivalent annotation. Be conservative. In curated-annotation-only mode, `COR` requires strong non-literature curated support such as a specific reviewed UniProt functional statement, diagnostic domain/subfamily annotation, or local pathway context. Do not call `COR` merely because the prediction sounds plausible.

**CNN: correct but not novel.** The prediction is biologically supported and the term is already present in UniProt/GOA, or is an obvious synonym/equivalent of an existing curated annotation. Mere overlap with an ARBA or other automated assertion establishes annotation overlap, not correctness. If biological support remains insufficient, use `UNC` and describe the overlap separately.

**LSP: less precise.** The prediction is broadly correct but less specific than an existing curated annotation. Example: predicting generic oxidoreductase activity when the curated annotation specifies the exact substrate or EC reaction.

**UNC: uncertain.** The allowed evidence cannot validate or refute the prediction. This is the default for plausible predictions absent from curated annotations when no allowed evidence establishes them. In curated-annotation-only mode, `UNC` should be common and is not a failure mode.

**PLI: paralog incorrect.** The prediction appears to come from over-propagation within a homologous family or paralog group, and allowed evidence indicates this gene belongs to a different paralog/subfamily/function than the predicted one. Set `error_type: PARALOG_OVERANNOTATION`.

**NPI: nonparalog incorrect.** The prediction is refuted for reasons other than paralog confusion: pathway absent in the organism, predicted process incompatible with taxon or compartment, curated function clearly different, activity belongs to an unrelated gene, or the prediction contradicts allowed evidence. Choose the most specific error type, often `PATHWAY_CONTEXT_IGNORED`, `MULTIPLE_FUNCTIONS`, `CURATION_MISTAKE`, or another schema enum value. Other specific options: `PSEUDOENZYME_OVERANNOTATION` (catalytic activity predicted for a fold-retaining protein that has lost its catalytic residues), `LOCALIZATION_DEFAULT` (cytosol/cytoplasm assumed when no TM/signal features, mislocalizing secreted/organellar/membrane proteins), `TAXON_CONSTRAINT_VIOLATION` (term valid only in a different lineage/kingdom, e.g. animal terms for a plant protein), `WRONG_INPUT_SEQUENCE` (the predictor was fed the wrong protein — a pipeline error, not a model error).

**REP: repetition / frequency bias.** The prediction looks like a high-frequency/default label assigned without supporting sequence or curated evidence, especially common generic enzyme classes such as histidine kinase or PTS transporter. Set `error_type: FREQUENCY_BIAS`.

## Confidence Score

- `2`: concordant with allowed evidence. Use with `COR`, `CNN`, or `LSP`.
- `1`: uncertain. Use with `UNC`.
- `0`: discordant with allowed evidence. Use with `PLI`, `NPI`, or `REP`.

## Judgment Rules

- Prefer `UNC` over confident correctness when the prediction is absent from curated annotations and only weakly suggested by domains.
- Once correctness is supported, prefer `CNN` over `COR` when the prediction is already in GOA/UniProt. Agreement with an unsupported existing assertion is not sufficient for either category.
- Prefer `LSP` over `CNN` when the prediction is correct but materially less specific than the curated annotation.
- Use `PLI` only when the error is specifically homolog/paralog/subfamily confusion.
- Use `NPI` for pathway absence, wrong organismal context, wrong compartment/process, or contradiction by curated function.
- Use `REP` only when the best explanation is a generic high-frequency label rather than a specific biological confusion.
- Do not invent identifiers, publications, or supporting quotes.