Geniml

---
name: geniml
description: 'Plans and audits local genomic-interval machine learning workflows with Geniml (0.8.4) and Gtars: validates BED files against chromosome sizes and assembly contracts, plans Region2Vec, scEmbed, and BEDspace runs, checks model, tokenizer, and universe.bed compatibility, and assesses consensus universes (CC, CCF, ML, HMM, assess-universe). Bundled scripts only validate or plan; they do not train. Use when checking BED coordinates, contigs, or assembly before analysis. Use when planning Region2Vec or scEmbed training on tokenized Parquet data. Use when verifying that a checkpoint, config.yaml, and universe match. Use when building or assessing a consensus peak universe. Use when reviewing BEDbase or Hugging Face download risks. Not for general BED manipulation or peak calling; use bedtools or similar tools.'
license: MIT
compatibility: Requires Python 3.10+ and uv. Guidance targets geniml 0.8.4 with gtars 0.9.2; ML workflows need the pinned ml extra and compatible native wheels. Bundled planners and inspectors are dependency-free, local-only, and make no network requests.
allowed-tools: Read Write Edit Bash Glob
metadata:
  version: '1.2'
  category: life-sciences
  maintainer: Kalaris Labs
  upstream-version: 0.8.4
  last-reviewed: '2026-07-23'
---

# Geniml

Use Geniml for machine learning and statistical workflows over genomic interval
sets. Treat coordinates, assemblies, token vocabularies, model artifacts, and
sample grouping as explicit contracts. The bundled scripts validate or plan;
they do not import Geniml, contact services, deserialize models, or execute
training.

`Bash` is declared only for explicit, user-approved `uv`, Python, Geniml,
Gtars, Git, and native CLI commands shown in this guide; bundled Python helpers
do not spawn subprocesses. Example paths under `data/`, `refs/`, `work/`, and
`models/` are user-provided project placeholders, not missing bundled files.

## Verified release snapshot

- Latest stable PyPI release on 2026-07-23: `geniml==0.8.4` (2026-01-14).
- PyPI does not declare `Requires-Python`; its classifiers list Python
  3.10-3.14. Prefer Python 3.11 or 3.12 where all native/ML wheels resolve.
- `geniml==0.8.4` accepts `gtars>=0.2.5`; the verified base smoke used current
  `gtars==0.9.2` (2026-06-17, Python >=3.10).
- Extras are `ml` and `test`. The base install omits Torch, Gensim, Scanpy,
  Hugging Face Hub, pyBigWig, and HMM dependencies.
- Upstream documentation contains stale examples. Release source and installed
  `--help` output take precedence where they conflict.

## Install reproducibly

Use a project environment and commit its generated lockfile:

```bash
uv venv --python 3.12
uv pip install "geniml==0.8.4" "gtars==0.9.2"
```

For Region2Vec, scEmbed, evaluation, or universe methods needing ML libraries:

```bash
uv pip install "geniml[ml]==0.8.4" "gtars==0.9.2"
```

For a durable project, prefer:

```bash
uv add "geniml[ml]==0.8.4" "gtars==0.9.2"
uv lock
```

Do not install an unpinned Git branch. Record Python, OS/architecture, the
resolved lockfile, and the PyPI artifact digest. Geniml itself is BSD-2-Clause;
the `MIT` frontmatter value licenses this skill's content.

## Start with the safety gate

Before importing Geniml or running an external binary:

1. Work only with explicit local regular files. Reject URLs, FIFOs, devices,
   and symlinks unless the user deliberately changes that policy.
2. Validate BED structure and the declared assembly against a trusted local
   chromosome-sizes file.
3. Bound file count, bytes, rows, workers, epochs, and output size.
4. Separate train/validation/test by patient, donor, biological replicate, or
   other independent unit—not by BED row or cell alone.
5. Inventory and checksum the universe, tokenizer, model, config, inputs,
   metadata manifest, and native binaries.
6. Obtain explicit approval before any BEDbase or Hugging Face download. Never
   infer approval from a model ID or BEDbase identifier.
7. Keep logs aggregate and bounded. BED filenames, sample IDs, phenotypes,
   labels, barcodes, and genomic intervals may be sensitive.

## Coordinate and assembly contract

BED intervals are normally **0-based, half-open** `[start, end)`: start is
included, end is excluded, and length is `end - start`. Do not mix them with
1-based closed coordinates from VCF/GFF or user-facing genome browsers.

For every corpus and artifact, record:

- assembly and patch/accession where possible (for example GRCh38 versus
  GRCh38.p14), plus the chromosome-sizes checksum;
- contig naming convention (`chr1` versus `1`), alt/random/decoy policy, and
  mitochondrial naming;
- coordinate convention, sorting order, duplicate/overlap policy, and whether
  BED strand is meaningful;
- liftover tool, chain digest, source/target assemblies, unmapped fraction, and
  post-liftover validation.

Reject negative coordinates, `end <= start`, integer overflow, unknown
contigs, ends beyond contig length, malformed columns, mixed assemblies, and
silent contig renaming. Sorting and normalization never repair an assembly
mismatch. BED3 has no strand; when column 6 is present, preserve `+`, `-`, or
`.` unless the assay contract says otherwise.

Run a bounded validation and normalization **plan** before analysis:

```bash
python skills/geniml/scripts/bed_validator.py \
  --input data/peaks.bed \
  --assembly GRCh38 \
  --chrom-sizes refs/GRCh38.chrom.sizes
```

The validator reports proposed actions but never rewrites the BED file.

## Current API map

### Region and tokenizer I/O

Prefer Gtars for new interval/tokenizer code:

```python
from gtars.models import Region, RegionSet
from gtars.tokenizers import Tokenizer

regions = RegionSet("data/peaks.bed")
tokenizer = Tokenizer.from_bed("refs/universe.bed")
encoded = tokenizer(regions)
input_ids = encoded["input_ids"]
```

`RegionSet` and `Tokenizer` also accept remote inputs in some constructors;
this skill permits local paths only unless network access is explicitly
approved. `geniml.io.RegionSet(regions, backed=False)` remains available as a
legacy Python implementation; backed sets are iterable but not indexable.
`geniml.io.Region` uses `stop`, while `gtars.models.Region` uses `end`.

With gtars 0.9.2, seven special tokens are added to a BED vocabulary. Therefore
`len(tokenizer)` is not simply the number of universe rows. Preserve universe
row order and the exact special-token map.

### Region2Vec

The modern class lives at a concrete module path:

```python
from geniml.region2vec.main import Region2VecExModel
from geniml.region2vec.utils import Region2VecDataset
from gtars.tokenizers import Tokenizer

tokenizer = Tokenizer.from_bed("refs/universe.bed")
dataset = Region2VecDataset("work/tokens.parquet", shuffle=True)
model = Region2VecExModel(tokenizer=tokenizer, embedding_dim=100)
model.train(dataset, epochs=10, window_size=5, num_cpus=4, seed=42)
```

The Parquet input must contain one list-valued `tokens` column, one document
per row. See [references/region2vec.md](references/region2vec.md) for export,
encoding, legacy CLI, and evaluation details.

### scEmbed

Import `ScEmbed` from `geniml.scembed.main`. AnnData `.var` must contain
`chr`, `start`, and `end`; rows are cells and nonzero features identify
accessible regions. Pre-tokenize to a Parquet `tokens` column and use the same
Tokenizer for training and inference. See
[references/scembed.md](references/scembed.md).

### BEDspace

BEDspace remains in 0.8.4 and invokes an external StarSpace executable.
StarSpace is archived and upstream Geniml does not pin a compatible revision.
Treat BEDspace as a legacy reproduction path, not the default for new systems.
See [references/bedspace.md](references/bedspace.md) for the exact stable CLI
spelling and an immutable, explicitly unverified build baseline.

### Consensus universes and assessment

The installed 0.8.4 CLI uses:

```text
geniml build-universe {cc,ccf,ml,hmm} ...
geniml assess-universe ...
geniml eval {gdst,npt,ctt,rct,bin-gen} ...
```

CC/CCF/ML/HMM consume precomputed coverage bigWigs. Do not concatenate or
generate coverage until all BED files pass the same assembly contract.
Assessment and embedding metrics are distinct: `assess-universe` measures fit
of a universe to interval collections, while `eval` implements CTT, RCT, GDST,
and NPT for embeddings. See
[references/consensus_peaks.md](references/consensus_peaks.md) and
[references/utilities.md](references/utilities.md).

## Important 0.8.4 migration notes

- The 0.7.0 changelog moved new RegionSet/tokenizer work toward Gtars.
- The 0.4.0 names `TreeTokenizer` and `AnnDataTokenizer` are historical; the
  current Gtars API exposes `Tokenizer`.
- In the 0.8.4 wheel, `geniml.region2vec` and `geniml.scembed` do not re-export
  their modern classes/functions. Use the concrete module paths above.
- `geniml tokenize` and `geniml region2vec` call names no longer exported by
  their package `__init__` files; do not build new workflows around those CLI
  paths without an installed-version smoke test.
- `geniml scembed` parses legacy MatrixMarket options but its command body is a
  no-op in 0.8.4. Use `geniml.scembed.main.ScEmbed`.
- Official pages still show `geniml assess`; the release command is
  `geniml assess-universe`.
- `.gtok` remains present in legacy datasets, but upstream issue #14 proposes
  deprecating many-file `.gtok` workflows. Prefer one bounded Parquet corpus.
- Config key `embedding_size` is accepted only for backward compatibility;
  use `embedding_dim`.

## Model and universe compatibility

A Region2Vec/scEmbed inference bundle is valid only when these agree:

- model `config.yaml` `vocab_size` and `embedding_dim`;
- exact `universe.bed` bytes/order and assembly;
- tokenizer implementation/version and special-token IDs;
- checkpoint tensor shapes and pooling policy;
- Geniml/Gtars versions and any tokenization parameters.

Geniml 0.8.4 defaults to `checkpoint.pt`, `config.yaml`, and `universe.bed`.
Its loader uses `torch.load(..., weights_only=True)`, but `.pt`, Gensim
`.model`, pickle, joblib, and native binaries remain untrusted inputs. Inspect
and checksum artifacts before loading; use an isolated environment and never
load a checkpoint merely to discover its metadata.

```bash
python skills/geniml/scripts/model_artifact_inspector.py \
  --model-dir models/region2vec

python skills/geniml/scripts/tokenizer_compatibility.py \
  --model-dir models/region2vec \
  --universe refs/universe.bed \
  --assembly GRCh38
```

`Region2VecExModel(model_path="org/repo")`, `ScEmbed(model_path="org/repo")`,
and Gtars `Tokenizer.from_pretrained(...)` can download from Hugging Face.
Local `from_pretrained("models/local")` loads a local bundle. Pin Hub revision
and expected hashes when a user approves download; then work offline from the
verified cache.

## BEDbase downloads and caches

`BBClient.load_bed`, `load_bedset`, and token-cache operations may contact
`https://api.bedbase.org`. The default cache is
`$BBCLIENT_CACHE` or `~/.bbcache`; `BEDBASE_API` changes the endpoint. Do not
read unrelated environment variables. Set an explicit project cache, estimate
size, approve identifiers/endpoints, and verify returned checksums before use.

Local inspection commands are safer:

```text
geniml bbclient seek ID --cache-folder /absolute/project/cache
geniml bbclient inspect-bedfiles --cache-folder /absolute/project/cache
geniml bbclient inspect-bedsets --cache-folder /absolute/project/cache
```

The `cache-bed`, `cache-bedset`, and `cache-tokens` subcommands may use the
network. Do not run them implicitly or include sensitive local BED files in an
upload/cache workflow.

## Local audit and planning CLIs

All scripts are standard-library-only and default to redacted JSON:

```bash
# Audit manifest paths, checksums, assemblies, and patient/donor leakage
python skills/geniml/scripts/corpus_auditor.py \
  --manifest data/manifest.tsv --assembly-column assembly \
  --group-column patient_id --split-column split

# Plan tokenizer/model compatibility checks
python skills/geniml/scripts/tokenizer_compatibility.py \
  --model-dir models/r2v --universe refs/universe.bed --assembly GRCh38

# Plan consensus construction; does not execute Geniml or coverage tools
python skills/geniml/scripts/consensus_plan.py \
  --manifest data/manifest.tsv --chrom-sizes refs/GRCh38.chrom.sizes \
  --assembly GRCh38 --method cc --output-dir work/consensus

# Plan an embedding run; does not import ML libraries
python skills/geniml/scripts/embedding_plan.py \
  --mode region2vec --data work/tokens.parquet \
  --universe refs/universe.bed --output-dir work/r2v \
  --assembly GRCh38
```

Use `--help` for resource limits and explicit path-disclosure controls.

## References

- [Region2Vec](references/region2vec.md): modern API, artifacts, CLI drift,
  training, encoding, and evaluation.
- [scEmbed](references/scembed.md): AnnData/token preparation, training,
  inference, annotation, privacy, and leakage.
- [BEDspace](references/bedspace.md): metadata schema, exact legacy CLI,
  StarSpace status, artifacts, and retrieval.
- [Consensus peaks](references/consensus_peaks.md): coverage prerequisites,
  CC/CCF/ML/HMM, assessment, and assembly safeguards.
- [Utilities](references/utilities.md): I/O, Gtars tokenizers, BBClient,
  evaluation, model safety, migration, and dated sources.

Source snapshot and primary-paper links are dated in
[references/utilities.md](references/utilities.md). Re-check release metadata
and installed signatures before changing the pinned versions.

## Agent operating procedure

1. **Check the environment.** Confirm tool versions, the reference genome/annotation build and the input formats (FASTQ, BAM, VCF, h5ad).
2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.
3. **Run a small version first.** Run the pipeline on a small subset (one sample, one chromosome, a few thousand cells) first.
4. **Execute the full task** using the instructions and references above.
5. **Validate the result.** Check QC metrics, sample identities, genome build consistency and batch effects before interpreting results.
6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.

| If this happens | Do this |
|---|---|
| Genome builds or identifiers do not match between inputs | Stop and harmonize (liftover, ID mapping) before continuing. |
| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |
| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |

**Integrity rules**

- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.
- Do not interpret biological significance beyond what the statistics support; report multiple-testing correction.
- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.
- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.

## Related skills

- `gtars`: Inspects and plans work with Gtars, the Rust/Python/CLI toolkit for genomic intervals: BED RegionSet set algebra (reduce, setdiff, intersec…
- `genomic-coordinates`: Convert genomic intervals between coordinate conventions, normalise and compare variant representations, and detect assembly or contig-nami…