Bertscore

---
name: bertscore
description: Compute the BERTScore metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute BERTScore, or asks how to score with BERTScore.
metadata:
  skill_kind: metric
  source_lib: torchmetrics
  import_path: torchmetrics.text.BERTScore
  source: library_introspection
---

# bertscore

> Metric `BERTScore` from `torchmetrics` (torchmetrics.text.BERTScore)

## When to invoke this skill

The user has predictions + ground truth and asks to evaluate with BERTScore, or
mentions `torchmetrics.text.BERTScore` directly, or wants the standard torchmetrics implementation.

## Reference signature

```python
from torchmetrics.text import BERTScore

# BERTScore(model_name_or_path: Optional[str] = None, num_layers: Optional[int] = None, all_layers: bool = False, model: Optional[torch.nn.modules.module.Module] = None, user_tokenizer: Optional[Any] = None, user_forward_fn: Optional[Callable[[torch.nn.modules.module.Module, dict[str, torch.Tensor]], torch.Tensor]] = None, verbose: bool = False, idf: bool = False, device: Union[str, torch.device, NoneType] = None, max_length: int = 512, batch_size: int = 64, num_threads: int = 0, return_hash: bool = False, lang: str = 'en', rescale_with_baseline: bool = False, baseline_path: Optional[str] = None, baseline_url: Optional[str] = None, truncation: bool = False, **kwargs: Any) -> None
```

## Library docstring

```
`Bert_score Evaluating Text Generation`_ for measuring text similarity.

BERT leverages the pre-trained contextual embeddings from BERT and matches words in candidate and reference
sentences by cosine similarity. It has been shown to correlate with human judgment on sentence-level and
system-level evaluation. Moreover, BERTScore computes precision, recall, and F1 measure, which can be useful for
evaluating different language generation tasks. This implementation follows the original implementation from
`BERT_score`_.

As input to ``forward`` and ``update`` the metric accepts the following input:

- ``preds``: Predicted sentence(s). Can be one of:

    * A single predicted sentence as a string (``str``)
    * A sequence of predicted sentences (``Sequence[str]``)

- ``target``: Target/reference sentence(s). Can be one of:

    * A single reference sentence as a string (``str``)
    * A sequence of reference sentences (``Sequence[str]``)
    * A sequence of sequences of reference sentences for multi-reference evaluation (``Sequence[Sequence[str]]``)

As output of ``forward`` and ``compute`` the metric returns the following output:

- ``score`` (:class:`~Dict`): A dictionary containing the keys ``precision``, ``recall`` and ``f1`` with
  corresponding values

Args:
    preds (Union[str, Sequence[str]]): A single predicted sentence or a sequence of predicted sentences.
    target (Union[str, Sequence[str], Sequence[Sequence[str]]]): A single target sentence, a sequence of target
        sentences, or a sequence of sequences of target sentences for multiple references per prediction.
    model_type: A name or a model path used to load ``transformers`` pretrained model.
    num_layers: A layer of representation to use.
    all_layers:
        An indication of whether the representation from all model's layers should be used.
        If ``all_layers=True``, the argument ``num_layers`` is ignored.
    model:  A user's own model. Must be of `torch.nn.Module` instance.
    user_tokenizer:
        A user's own tokenizer used with the own model. This must be an instance with the ``__call__`` method.
        This method must take an iterable of sentences (`List[str]`) and mus
```

## Quick recipe

```python
import torchmetrics.text as _m
score = _m.BERTScore(y_true, y_pred)
```

## Don'ts

- Don't reimplement when the library version handles edge cases (NaN, ties, empty inputs) better than a hand-rolled formula.
- Always check the library version's argument order — sklearn is `(y_true, y_pred)` while torchmetrics is `(preds, target)`.