Clipscore

---
name: clipscore
description: Compute the CLIPScore metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute CLIPScore, or asks how to score with CLIPScore.
metadata:
  skill_kind: metric
  source_lib: torchmetrics
  import_path: torchmetrics.multimodal.CLIPScore
  source: library_introspection
---

# clipscore

> Metric `CLIPScore` from `torchmetrics` (torchmetrics.multimodal.CLIPScore)

## When to invoke this skill

The user has predictions + ground truth and asks to evaluate with CLIPScore, or
mentions `torchmetrics.multimodal.CLIPScore` directly, or wants the standard torchmetrics implementation.

## Reference signature

```python
from torchmetrics.multimodal import CLIPScore

# CLIPScore(model_name_or_path: Union[Literal['openai/clip-vit-base-patch16', 'openai/clip-vit-base-patch32', 'openai/clip-vit-large-patch14-336', 'openai/clip-vit-large-patch14', 'jinaai/jina-clip-v2', 'zer0int/LongCLIP-L-Diffusers', 'zer0int/LongCLIP-GmP-ViT-L-14'], Callable[[], tuple[None, None]]] = 'openai/clip-vit-large-patch14', **kwargs: Any) -> None
```

## Library docstring

```
Calculates `CLIP Score`_ which is a text-to-image similarity metric.

CLIP Score is a reference free metric that can be used to evaluate the correlation between a generated caption for
an image and the actual content of the image, as well as the similarity between texts or images. It has been found
to be highly correlated with human judgement. The metric is defined as:

.. math::
    \text{CLIPScore(I, C)} = max(100 * cos(E_I, E_C), 0)

which corresponds to the cosine similarity between visual `CLIP`_ embedding :math:`E_i` for an image :math:`i` and
textual CLIP embedding :math:`E_C` for an caption :math:`C`. The score is bound between 0 and 100 and the closer
to 100 the better.

Additionally, the CLIP Score can be calculated for the same modalities:

.. math::
    \text{CLIPScore(I_1, I_2)} = max(100 * cos(E_{I_1}, E_{I_2}), 0)

where :math:`E_{I_1}` and :math:`E_{I_2}` are the visual embeddings for images :math:`I_1` and :math:`I_2`.

.. math::
    \text{CLIPScore(T_1, T_2)} = max(100 * cos(E_{T_1}, E_{T_2}), 0)

where :math:`E_{T_1}` and :math:`E_{T_2}` are the textual embeddings for texts :math:`T_1` and :math:`T_2`.

.. caution::
    Metric is not scriptable

.. note::
    The default CLIP and processor used in this implementation has a maximum sequence length of 77 for text
    inputs. If you need to process longer captions, you can use the `zer0int/LongCLIP-L-Diffusers` model which
    has a maximum sequence length of 248.

As input to ``forward`` and ``update`` the metric accepts the following input

- source: Source input.

    This can be:

    - Images: ``Tensor`` or list of ``Tensor``

        If a single tensor, it should have shape ``(N, C, H, W)``.
        If a list of tensors, each tensor should have shape ``(C, H, W)``.
        ``C`` is the number of channels, ``H`` and ``W`` are the height and width of the image.

    - Text: ``str`` or list of ``str``

        Either a single caption or a list of captions.

- target: Target input.

    This can be:

    - Images: ``Tensor`` or list of ``Tensor``

        If a single tensor, it should have shape ``(N, C, H, W)``.
        If a list of tensors, each tensor should have shape ``(C, H, W)``.
       
```

## Quick recipe

```python
import torchmetrics.multimodal as _m
score = _m.CLIPScore(y_true, y_pred)
```

## Don'ts

- Don't reimplement when the library version handles edge cases (NaN, ties, empty inputs) better than a hand-rolled formula.
- Always check the library version's argument order — sklearn is `(y_true, y_pred)` while torchmetrics is `(preds, target)`.