Jionlp

---
name: jionlp
description: "Chinese NLP preprocessing, parsing, augmentation, and
  language-analysis router for JioNLP."
metadata:
  disco-role: operating
disable-model-invocation: true
license: Apache 2.0
---

# JioNLP

Use this skill when the task is about JioNLP's Chinese text preprocessing, rule extraction, parsing, augmentation, annotation helpers, or bundled language-analysis utilities.

## Quick start
- Install: `pip install jionlp`
- Import: `import jionlp as jio`
- Sanity check: `print(jio.__version__)`
- Search APIs interactively: `jio.help()` or `jio_help`
- For a silent smoke check, run `scripts/smoke_core.py`

Importing `jionlp` prints a banner and loads `jiojio`'s C helpers when available. That is expected.

## Route map

| User intent | Use this sub-skill |
| --- | --- |
| Clean raw text, strip URLs/emails/phones, normalize full-width chars, clean HTML, split sentences, read/write line files, or search docstrings | `sub-skills/text-cleaning-and-extraction/` |
| Parse or normalize time, money, location, phone, ID, vehicle plate, pinyin, radicals, idioms, or lunar/solar dates | `sub-skills/parsing-and-normalization/` |
| Augment text with back translation, character swaps, homophone substitution, random add/delete, or entity replacement | `sub-skills/text-augmentation/` |
| Convert NER/CWS/POS tags, build lexicon NER, split annotated datasets, or batch long NER inputs | `sub-skills/annotation-and-dataset-tools/` |
| Load packaged dictionaries, extract keyphrases or summaries, score sentiment, find new words, use BPE, load the LLM test set, or work with MELLM | `sub-skills/dictionaries-and-language-analysis/` |

## How to route quickly
- If the request starts with a noisy paragraph and asks to remove or detect obvious patterns, start with `text-cleaning-and-extraction`.
- If the request asks what a time string, money string, address, phone number, ID card, or plate means, start with `parsing-and-normalization`.
- If the request mentions augmentation, back translation, or replacing entities, start with `text-augmentation`.
- If the request mentions BIO/BIOES tags, entity offsets, CWS/POS conversion, or dataset splits, start with `annotation-and-dataset-tools`.
- If the request mentions loaders, keyphrase, summary, sentiment, new words, BPE, LLM test data, or MELLM, start with `dictionaries-and-language-analysis`.

## Practical notes
- `jio.help()` is interactive. Use `scripts/search_api_docs.py` if you want a noninteractive keyword search.
- `jio.bpe.byte_level_bpe` lives under the `bpe` submodule, not as a root attribute.
- Keyphrase, summary, new-word, NER time extraction, and text-classification helpers are exposed through submodules such as `jio.keyphrase`, `jio.summary`, `jio.new_word`, `jio.ner`, and `jio.text_classification`.
- A NumPy 1.x release older than 1.24 is required for the CWS/POS converters because this repository uses `np.unicode`.
- `phone_location()` is safer on text that contains a leading non-digit boundary before the phone number.
- `llm_test_dataset_loader()` expects version strings such as `'1.0'` or `'1.1'`.

## Read these first when needed
- `references/cli-reference.md` for `jio_help` and the bundled search wrapper.
- `references/troubleshooting.md` for import, NumPy, loader, and external-data issues.
- `scripts/smoke_core.py` for a fast import and sample-API smoke check.