Pre Submission Review

---
name: pre-submission-review
description: >-
  Evidence-anchored evaluation of the USER'S OWN manuscript before journal submission.
  Use when the user wants their draft evaluated ("review my draft", "is this ready for
  submission", "act as reviewer 2 on my paper", "check my manuscript before I send it"),
  names a target journal for their own paper, or asks how their study would fare in peer
  review. Handles original research AND research syntheses (systematic/scoping reviews,
  meta-analyses) in language assessment / applied linguistics. Produces a prioritized
  revision plan plus a 0–100 quality estimate derived from the evidence. NOT for reviewing
  other people's manuscripts under journal peer review — that is a different activity with
  different obligations.
---

# pre-submission-review

Evaluates the user's own manuscript before journal submission, through the criteria its eventual reviewers will apply. Two passes: an **evidence pass** that reads the manuscript and records every observation with a page location and, where relevant, the recomputed number — then a **writing pass** that produces a prioritized revision plan and a quality-score estimate from that verified material only. Unlocated criticism is not recorded.

This skill evaluates **the user's own manuscript only.** Reviewing someone else's submission for a journal is a different activity, with confidentiality and publisher-policy obligations this skill does not manage — most publishers prohibit uploading others' manuscripts to AI tools at all.

**Confidentiality of outputs.** The manuscript is unpublished and this skill's outputs quote it: treat the notes and plan as part of the manuscript's confidential record — store them beside it, never in a public repository, and leave them out of anything shared beyond the author team.

## Step 0 — intake

Read the manuscript. Text extraction is not reading: **figures and appendices are where the checkable material lives**, and extraction drops images — prose may come from the text layer, but **anything two-dimensional may not**: tables, figures, diagrams, and appendix grids are read from rendered page images. Extraction linearizes tables — cell-position is destroyed and rows silently vanish at page breaks — so any numeric or absence claim about a table is verified against the rendered page (or word coordinates), never the text stream alone. Use the manuscript's printed page numbers, strip page furniture before any word count (and say which method you used), and say plainly what you could not see. **Run `../../starter-kit/pdf_probe.py` on the PDF before reading**: its verdict (TEXT-RICH / TEXT-SPARSE / IMAGE-ONLY / ENCRYPTED) and its visual-read page list turn the reading-basis declaration into a measured fact and give you the render plan up front — an IMAGE-ONLY scan is read entirely from rendered page images (vision is the OCR; no extra tools).

Then establish three things:

- **Article type** → the evaluation backbone: *original research* → the empirical backbone below plus the **original-research module** in `references/evaluation-criteria.md`; *research synthesis* (systematic/scoping review, meta-analysis, methodological synthesis) → the backbone PLUS the synthesis module below PLUS the **review-article module** in `references/evaluation-criteria.md`; conceptual/position, replication, short report → the backbone minus what doesn't apply. The criteria file carries the section-by-section questions reviewers at the field's journals are actually asked, per article type — evaluate against those, not an invented rubric.
- **Draft stage** → complete manuscript, or a partial draft (gated sections, "[to complete]" blocks, placeholder appendices)? Partial drafts are a normal, valuable case — the cheapest review is the pre-data one. Review what exists; collect the acknowledged placeholders into a **pending gate** block in the triage (they gate submission, they are not findings); and scope the score accordingly (Step 2).
- **Target journal** → the paper's actual judges. **Verify the journal's identity first** when there is any ambiguity — near-namesake and predatory titles exist, and evaluating against the wrong venue's standards misdirects the whole review. Then load the journal's section from `references/journal-reviewer-guides.md` — it records what several major venues *tell their reviewers to look for* (criteria, scope bars, methodological expectations), so the evaluation runs through the eyes the paper will actually meet. If the journal isn't in the file, fetch its reviewer-facing guidance now and append a section. Also **fetch the journal's current instructions for authors** (live — these change too often to cache): word limits, abstract format, structure requirements, anonymization prep, data statements. Compliance is part of readiness. **When the journal's guidance cannot be found** (new journals, some non-Anglophone venues): say so, agree a proxy journal with the user, run compliance against the proxy, and mark every proxy-derived item "confirm with the journal." For **invited papers**, the editors' invitation governs — proxy items become questions for the editors.

**One reminder to surface every time (author-side, the user's to act on):** most journals now require authors to **declare AI use** per their policy — and depending on the target journal's wording, using an AI evaluation like this one may itself belong in that declaration. Check the journal's instructions for authors and include the declaration it asks for; what to declare and how is the author's call.

## Step 1 — evidence pass

Produce `<MS-ID>-notes-<date>.md` — working notes, blunt and telegraphic (date-stamp the filename; re-runs must not overwrite prior reviews). When the manuscript's own project folder is available — own-manuscript reviews usually have one — **check questionable claims against the project's audit trail before raising them**: "the evidence exists but the manuscript doesn't state it" is a different, far cheaper finding than "missing analysis." The manuscript is still judged as its reviewers will read it: folder-blind — the fix is in the manuscript either way.

- **A · What the paper claims.** Two or three sentences: construct, design, headline finding, the claim the reader is asked to accept. If the argument can't be stated fairly, the evaluation isn't ready to start.
- **B · Section-by-section observations.** Walk the manuscript in order; each entry: `[p. N]` — for PDFs use the manuscript's **printed** page numbers (what reviewers and authors cite), not PDF sheet numbers; name unnumbered front-matter pages ("Highlights page"); state the convention once at the top (unpaginated drafts: section anchor + a short verbatim quotation) → what the text says → what's wrong or missing → how serious. Record what's *done well* too — the author needs to know what not to touch. Mark inferences that need the author's knowledge as **observations/questions**, distinct from defects — they're asked, not asserted.
- **C · Verification ledger.** Where the evaluation earns its keep: recompute effect sizes from reported statistics; reconcile every N against every df; check the abstract's numbers against the results tables; check the reliability coefficient is the right one for the design (an unqualified "ICC = .91" will draw reviewer fire — model and type matter). Actual computation, not mental arithmetic. Record each check **PASS / FAIL / CANNOT CHECK** with the numbers — a numeric inconsistency caught now is one a reviewer never sees. Type each entry by the kind of claim it checks — **citation** (the source exists and says what is attributed to it), **numerical** (the value traces to a table, appendix, or dataset in the manuscript), **methodological** (the described procedure resolves to the instrument actually included), or **conclusion** (it follows from verified claims of the other kinds) — so coverage itself is auditable: all four kinds, every manuscript.
- **C′ · Claim–warrant checks** — the layer expert reviewers supply, run mechanically: **cited-source check (concept attribution)** — for **every** literature warrant in the argument-bearing sections (Introduction / literature review, Discussion) and every quotation — not a 3–5 sample: mis-attached claims hide in the routine supporting citations written from memory of what a paper "is about" — open the cited source and verify it actually **develops** the framework, construct, or finding attributed to it — not merely a neighbouring idea (manuscripts routinely reframe prior work in their own vocabulary, and sometimes retrofit a proposed concept onto papers that never focused on it; **the cited authors' own terms are the test — quote them**). A useful tell: when the attributed concept **postdates** the cited paper, retrofitting is the likely explanation. Precision guards: no attribution finding without the source's verbatim wording quoted beside the manuscript's; tag it `[interpretation]` and phrase it as a question ("the source's own framing is X — does it carry the manuscript's concept Y?"); where the call needs field depth, say so and route it to the "needs your expert judgment" block rather than perform confidence. **Gap-claim check** — every "little attention has been paid to X" triggers a targeted literature search for X; report counter-examples (where the project runs a literature radar, point it at the gap sentence). **Construct-relationship check** — if the paper relates two constructs, state their claimed relationship in one sentence; if the text supports incompatible framings (analytical lens vs. causal antecedent vs. overlap), that is a finding. **Temporal-validity check** — compare when data were collected against the events they are asked to explain; weeks-later retrospective accounts cannot carry real-time claims. **Method-rationale consistency** — a format justified for one property (e.g., group interviews chosen for co-construction) must not be analysed as its opposite (individual cognition). **Method–instrument alignment** — every procedure the methods describe must match the instrument that operationalizes it in the manuscript's own materials: the rubric, band descriptors, codebook, interview guide, questionnaire, or scoring prompt in the appendices. Count the categories, match the names, check the versions — a described-but-absent (or present-but-different) instrument is among the most persistent errors in print. **Causal-lexicon sweep** — audit the full causal verb family (influence, effect on, shape, drive, foster, trigger, intensify, lead to) against what the design licenses, not just the flagship term. **Paragraph promise audit** — when a topic sentence promises a taxonomy, check every promised branch is delivered.
- **D · Do-not-raise list.** What *looked* wrong but checks out — the df that is correct, the limitation already stated, the analysis that was in the appendix. This protects the plan's credibility and stops the author burning effort on non-problems. Keep beside it a **residual-tensions register**: conceptual observations you noticed but filed as minor or resolved (a framing wobble, a construct boundary that almost holds) — re-examine these in every later round rather than carrying them silently; the expert-grade findings usually live there.
- **E · Triage** — **Fatal** (the design/data cannot support the claims as framed; fix the framing or the study, not the prose) / **Major** (conclusions outrun evidence, but closable with analysis, reporting, or narrower claims) / **Minor** (clarity, reporting detail, presentation, references). **Tag every finding by its basis**: `[fact]` — recomputed, quoted, or mechanically checked; the finding stands whoever reads it — or `[interpretation]` — construct boundaries, severity of conceptual issues, field-norm calibration ("is this gap claim fair?", "will this venue accept this design?"), what a construct claim means. Interpretation findings never carry a solo severity: they are presented as **questions with a provisional severity, pending the author's read** — this is the layer where the expert outranks any model, and the review must say so rather than perform confidence it doesn't have.

For thorough runs, add **parallel lens passes** — separate readers for construct theory, methodology, and field coverage over the same manuscript. One pass reads with one pair of eyes; the panel the paper will meet has several, and they demonstrably see different things.

**Then stop and show the user the triage list** before writing prose — the cheapest moment to catch a misreading, and where the author's knowledge of their own study overrides yours. Present it in two blocks: the `[fact]` findings (skimmable — they were verified), and a **"needs your expert judgment"** block carrying every `[interpretation]` finding, every construct-boundary question, and any point where legitimate positions conflict (reviewers demonstrably pull opposite directions on the same wording — surface the tension with a recommendation, never silently side). **Tell the user the score arrives with the revision plan, after this gate** — otherwise its absence here reads as a malfunction.

## Synthesis module (loaded when the manuscript is a research synthesis)

On top of the ledger: **search transparency** (databases — several, with rationale; exact strings; date ranges; grey literature; a flow accounting of records) · **selection rigour** (criteria stated before searching; two-level screening; more than one screener with agreement reported) · **extraction & appraisal** (piloted form; double coding with agreement; quality appraisal with an existing tool where applicable) · **registration/protocol** where the venue's culture expects it, plus reported deviations · **meta-analysis specifics** (effect-size metric justified; dependency among effect sizes handled; fixed-vs-random justified; heterogeneity *interpreted*; publication bias; moderators pre-specified) · **the promise audit** — quote the manuscript's own statement of contribution and check the results deliver it item by item (syntheses routinely promise an explanatory account and deliver a frequency table) · **agreement-benchmark literacy** — human–human agreement is itself an estimate with error; phrase "exceeds human agreement" claims accordingly · **the artifact check** — could the paper's own filters have manufactured the gap it reports? Enumerate every exclusion and ask what each removed. Where the target journal ships its own synthesis checklist (several do — see the guides file), run its items as an extra ledger section and tell the author to self-assess against it before submitting.

## Revision rounds (when the manuscript is a revised version under review)

Run the evidence pass on the new version, plus:

- **Resolution audit** — every prior comment (the reviewers' and this skill's) classified **resolved / partial / responded-but-not-resolved**: a response letter's claim is not the fix — check the manuscript delivers it, and check any revision text pasted into the response against what the manuscript actually says.
- **Commitment-delivery check** — every methodological commitment the revision *adds* is verified where the data are presented, not where the commitment is stated.
- **Sync check** — the abstract, highlights, and front matter are where body-fixes go to be forgotten; check them against every body change.
- **Duplication sweep** — revisions move text and leave copies; run a near-duplicate sentence check across the new version.
- **Two-way reference re-check** — added *and removed* citations both create orphans.
- **Contradictory reviewer demands** — where prior comments pull in opposite directions (it happens between entirely competent reviewers), name the tension, recommend one resolution with reasons, and note the alternative: the author must answer both.
- **Re-score** and report the trajectory against the previous round.

## Step 2 — the revision plan

Write `<MS-ID>-revision-plan-<date>.md` from the ledger only. Register: **objective, clear, cordial** — about the manuscript's evidence, checkable by a reader who disagrees, in the tone of a colleague who wants the paper to succeed. Two register rules separate a review from a log: **state findings, never narrate the audit** (what was recomputed and how it was verified lives in the notes; the plan reports what is true of the manuscript) — and **recommend as a courteous colleague** ("consider…", "I would recommend…", remedies as options rather than imperatives): firmness lives in the content, not the delivery — the same tone the field's publishers instruct their reviewers to use (see the guides file). On request, render each must-fix as a **plain-prose paragraph in reviewer-report style** (concern → evidence → why it matters → remedy) — useful for anticipating how a reviewer would word it, and as response-letter preparation. Structure:

- **The verdict paragraph**: what the paper is and genuinely contributes, then the honest read on where it stands — closing with an explicit **readiness verdict** on its own line, tied to the score bands: **READY TO SUBMIT** (after the listed minors) · **NOT YET — MAJOR REVISION FIRST** · **NOT YET — STRUCTURAL WORK FIRST** · **NOT ASSESSABLE FOR READINESS** (partial draft — reviewable-portion score only). The verdict is the sentence the author is actually asking for; the score gives it granularity and a trajectory to re-measure.
- **The score** — a **0–100 quality estimate, derived from the triage, drivers shown**: e.g. "≈72: two major issues (X, Y) hold it out of the mid-80s; resolving both plus the abstract–table mismatch puts it at submit-ready." Bands: **85+** submit after the minors · **70–84** major revision before submitting · **below 70** structural work first · any **fatal** finding caps the score below 60 regardless of polish. **Partial drafts:** score only the reviewable portion and label it so ("≈N for the written sections — not a submission-readiness score"); the readiness score waits for a complete draft — re-score then and compare. **Open interpretation items widen the score into an interval**: state the composition ("verified-fact findings put it at ≈76; two construct-level questions await your judgment — resolving them lands it 72–80"), because a point score pretends a precision the interpretation layer doesn't have. State the honesty clause with the score, every time: *this is a heuristic estimate from one AI reading, not a prediction of editorial outcome; its best use is trajectory — re-score after revising and watch the direction — and it never substitutes for a colleague's read.*
- **Prioritized fixes**, each in the actionable anatomy: the issue with its consequence up front → evidence, page-anchored → why it matters → the concrete fix (often as options). "Report the ICC model and type and the CI" is actionable; "reliability reporting is inadequate" is not. Label **must-fix vs would-strengthen**.
- **The compliance checklist** from the journal's current author instructions (limits, formats, statements, anonymization, the AI-use declaration) — plus a **preprint-citation pass**: check whether the venue states a position on citing preprints (most author guides are silent — their preprint sections usually govern *posting*, not citing; when silent, the integrity charter's rules 4 and 14 apply: label preprints as such, and supersede any whose version of record now exists — the citation-integrity skill's VoR check does this mechanically). Independent of policy, **flag any load-bearing claim resting on a non-peer-reviewed source**: reviewers challenge those whatever the guide says.
- **Standard critical checks, run on every paper**: does the design license the conclusion (descriptive ≠ normative, correlational ≠ causal)? · does the paper deliver what its own framing promised? · could the headline finding be an artifact of the method (a filter, a codebook gap) rather than a property of the world? Papers overclaim in the discussion far more often than they err in the results.
- **Never invent literature.** Recommend engaging only with work verified to exist; if unsure, describe the body of work by topic rather than fabricating an author-year. When a finding leaves a claim without on-point support, you may **offer** to run the literature-radar skill's seed-based discovery on that claim — surfacing real, resolving records as candidates *for the author to read and judge*; candidates to read, never citations to insert.

## Step 3 — verification pass (always)

A **fresh subagent** reads only the manuscript and the draft plan, with no memory of writing it, and checks: every page reference points where claimed · every quoted number matches the paper, recomputed with a real tool · every table-derived claim (counts, empty cells, row presence or absence) re-checked against rendered pages or word coordinates, never the extracted text stream · every criticism describes something the manuscript actually does or fails to do · every recommended reference exists · no two findings contradict each other. Self-checking rereads what you meant, not what you wrote — that's the whole reason for the fresh pass. Log what it caught, fix the plan, then deliver.

## Step 4 — cross-vendor audit (optional; only on the user's explicit request)

A second, **different-vendor** model re-judges the review — the field's own logic applied to us: one rater is one reading, and qualified raters demonstrably diverge, so a second independent rater turns the score and triage from a point estimate into something with a visible spread. Step 3's fresh agent shares the first reviewer's vendor and blind spots; the cross-vendor audit does not.

Run `../../starter-kit/manuscript_audit.py` (standard library, no installs; the report stamps the request format alongside model, effort and prompt version — on the `responses` path it also records that the storage opt-out was sent — the kit asks the provider not to retain the text; it cannot verify that the provider complied): it sends the manuscript text plus the numbered findings to the second model and returns per-finding **concur / dispute / cannot-judge** verdicts, evidence-anchored missed-finding proposals, and an independent score + readiness verdict — one dated report, **propose-only**; the author adjudicates every disagreement (the evidence ledger is the tie-breaker discipline: page-anchored beats plausible).

**The gate that makes this opt-in every single time:** the script transmits the *unpublished manuscript* to another provider. It refuses to run without `--confirm-send`; before suggesting it, remind the user this is their call — their own API key (env var, never through chat), their institution's policy on external AI services, the venue's AI policy, and a provider whose API terms exclude training on submissions. Always start with `--dry-run` (payload + estimated cost; nothing transmitted — real audits measure in cents). Pinning: reuse the project's `review-conventions.md` `## Audit` block where one exists (one pinning block serves both this and the literature-review audit; `manuscript-model:` / `manuscript-effort:` override for this audit specifically). **Tier the model to the task**: volume row-audits run on budget reasoning tiers, but this audit is one high-stakes judgment call — use a **frontier-tier** model at high effort ("correctness matters more than cost"); a full manuscript audit still measures in cents, and in validation the frontier tier returned expert-layer findings (construct circularity, timeline contradictions) that cheaper tiers and the entire human pipeline had not surfaced.

## Output

`<MS-ID>-notes-<date>.md` (working evidence) and `<MS-ID>-revision-plan-<date>.md`, next to the manuscript — plus, when Step 4 ran, `Manuscript Audit — <date>.md`. Close with the short version: the score with its one-line driver, the single issue that matters most, and anything you're unsure about — surfaced, not smoothed over.

## References

- `references/evaluation-criteria.md` — the two article-type criteria modules (**original research** · **research synthesis/review**), built from the section-by-section questions publishers put on their reviewer forms (Elsevier structured peer review) and journal-distributed synthesis checklists (e.g. SMART), stated in this field's terms. Load the module matching the manuscript's article type.
- `references/journal-reviewer-guides.md` — what major venues in the field tell their reviewers to look for (criteria, scope bars, expectations), so the paper can be read through its future judges' eyes; append new journals on demand.

## Provenance

Field-neutralized from a working researcher's tested review practice (two-pass evidence discipline, verification ledger, do-not-raise list, triage gate, fresh-agent verification) and from the public reviewer guidance of the field's major publishers and journals. Subsequently validated and refined in a **blind ground-truth test** against a published article's complete peer-review history — original submission, two revisions, seven reviewer reports across three rounds, and the version of record, with this skill's review frozen before each comparison. Its verdicts tracked the editorial trajectory at every round; its misses became the claim–warrant checks, the revision-rounds mode, and the register rules above; and its day-one mechanical findings included an internal methods–appendix contradiction that survived every human stage into print — which is why the ledger exists. The test materials are private; none appear in this skill or its references. The optional cross-vendor audit follows the same pattern as the literature-review skill's second-model audit — a cross-model verification pattern also proven in the wider open-source research-skill ecosystem (ideas only; no code reuse).