Bregman divergence

A Bregman divergence, or Bregman distance, $B_{F}$ on a space $\mathcal{X}\subseteq\mathbb{R}^{d}$ is defined for a strictly convex and differentiable function $F:\mathcal{X}\to\mathbb{R}$ as

B_{F}(p,q)=F(p)-F(q)-{\langle p-q,\nabla F(q)\rangle},

(1)

where

{\langle p,q\rangle}=p^{T}q

denotes the inner product, and

\nabla F(x)=[\frac{\partial F}{\partial x_{1}},\cdots,\frac{\partial F}{% \partial x_{d}}]^{T}

the partial derivatives.

Choosing $F(x)=\sum_{i=1}^{d}x_{i}^{2}$ yields the squared Euclidean distance $B_{x^{2}}(p,q)=||p-q||^{2}$ , and choosing $F(x)=\sum_{i=1}^{d}x_{i}\log x_{i}$ yields the relative entropy, called the Kullback-Leibler divergence.

Title	Bregman divergence
Canonical name	BregmanDivergence
Date of creation	2013-03-22 19:11:38
Last modified on	2013-03-22 19:11:38
Owner	FrankTokyo (25936)
Last modified by	FrankTokyo (25936)
Numerical id	6
Author	FrankTokyo (25936)
Entry type	Definition
Classification	msc 51K05
Synonym	Bregman distance