# Bregman divergence

A Bregman divergence, or Bregman distance, $B_{F}$ on a space $\mathcal{X}\subseteq\mathbb{R}^{d}$ is defined for a strictly convex and differentiable function $F:\mathcal{X}\to\mathbb{R}$ as

 $B_{F}(p,q)=F(p)-F(q)-{\langle p-q,\nabla F(q)\rangle},$ (1)

where

 ${\langle p,q\rangle}=p^{T}q$

denotes the inner product, and

 $\nabla F(x)=[\frac{\partial F}{\partial x_{1}},\cdots,\frac{\partial F}{% \partial x_{d}}]^{T}$

Choosing $F(x)=\sum_{i=1}^{d}x_{i}^{2}$ yields the squared Euclidean distance $B_{x^{2}}(p,q)=||p-q||^{2}$, and choosing $F(x)=\sum_{i=1}^{d}x_{i}\log x_{i}$ yields the relative entropy, called the Kullback-Leibler divergence.

Title Bregman divergence BregmanDivergence 2013-03-22 19:11:38 2013-03-22 19:11:38 FrankTokyo (25936) FrankTokyo (25936) 6 FrankTokyo (25936) Definition msc 51K05 Bregman distance