Kldivergence

---
name: kldivergence
description: Compute the KLDivergence metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute KLDivergence, or asks how to score with KLDivergence.
metadata:
  skill_kind: metric
  source_lib: torchmetrics
  import_path: torchmetrics.KLDivergence
  source: library_introspection
---

# kldivergence

> Metric `KLDivergence` from `torchmetrics` (torchmetrics.KLDivergence)

## When to invoke this skill

The user has predictions + ground truth and asks to evaluate with KLDivergence, or
mentions `torchmetrics.KLDivergence` directly, or wants the standard torchmetrics implementation.

## Reference signature

```python
from torchmetrics import KLDivergence

# KLDivergence(log_prob: bool = False, reduction: Literal['mean', 'sum', 'none', None] = 'mean', **kwargs: Any) -> None
```

## Library docstring

```
Compute the `KL divergence`_.

.. math::
    D_{KL}(P||Q) = \sum_{x\in\mathcal{X}} P(x) \log\frac{P(x)}{Q{x}}

Where :math:`P` and :math:`Q` are probability distributions where :math:`P` usually represents a distribution
over data and :math:`Q` is often a prior or approximation of :math:`P`. It should be noted that the KL divergence
is a non-symmetrical metric i.e. :math:`D_{KL}(P||Q) \neq D_{KL}(Q||P)`.

As input to ``forward`` and ``update`` the metric accepts the following input:

- ``p`` (:class:`~torch.Tensor`): a data distribution with shape ``(N, d)``
- ``q`` (:class:`~torch.Tensor`): prior or approximate distribution with shape ``(N, d)``

As output of ``forward`` and ``compute`` the metric returns the following output:

- ``kl_divergence`` (:class:`~torch.Tensor`): A tensor with the KL divergence

Args:
    log_prob: bool indicating if input is log-probabilities or probabilities. If given as probabilities,
        will normalize to make sure the distributes sum to 1.
    reduction:
        Determines how to reduce over the ``N``/batch dimension:

        - ``'mean'`` [default]: Averages score across samples
        - ``'sum'``: Sum score across samples
        - ``'none'`` or ``None``: Returns score per sample

    kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info.

Raises:
    TypeError:
        If ``log_prob`` is not an ``bool``.
    ValueError:
        If ``reduction`` is not one of ``'mean'``, ``'sum'``, ``'none'`` or ``None``.

.. attention::
    Half precision is only support on GPU for this metric.

Example:
    >>> from torch import tensor
    >>> from torchmetrics.regression import KLDivergence
    >>> p = tensor([[0.36, 0.48, 0.16]])
    >>> q = tensor([[1/3, 1/3, 1/3]])
    >>> kl_divergence = KLDivergence()
    >>> kl_divergence(p, q)
    tensor(0.0853)
```

## Quick recipe

```python
import torchmetrics as _m
score = _m.KLDivergence(y_true, y_pred)
```

## Don'ts

- Don't reimplement when the library version handles edge cases (NaN, ties, empty inputs) better than a hand-rolled formula.
- Always check the library version's argument order — sklearn is `(y_true, y_pred)` while torchmetrics is `(preds, target)`.