# Bregman divergence

A *Bregman divergence*, or *Bregman distance*, ${B}_{F}$ on a space $\mathcal{X}\subseteq {\mathbb{R}}^{d}$ is defined for a strictly convex and differentiable function $F:\mathcal{X}\to \mathbb{R}$ as

$${B}_{F}(p,q)=F(p)-F(q)-\u27e8p-q,\nabla F(q)\u27e9,$$ | (1) |

where

$$\u27e8p,q\u27e9={p}^{T}q$$ |

denotes the inner product, and

$$\nabla F(x)={[\frac{\partial F}{\partial {x}_{1}},\mathrm{\cdots},\frac{\partial F}{\partial {x}_{d}}]}^{T}$$ |

the partial derivatives^{}.

Choosing $F(x)={\sum}_{i=1}^{d}{x}_{i}^{2}$ yields the squared Euclidean distance ${B}_{{x}^{2}}(p,q)={||p-q||}^{2}$, and choosing $F(x)={\sum}_{i=1}^{d}{x}_{i}\mathrm{log}{x}_{i}$ yields the relative entropy^{}, called the Kullback-Leibler divergence.

Title | Bregman divergence |
---|---|

Canonical name | BregmanDivergence |

Date of creation | 2013-03-22 19:11:38 |

Last modified on | 2013-03-22 19:11:38 |

Owner | FrankTokyo (25936) |

Last modified by | FrankTokyo (25936) |

Numerical id | 6 |

Author | FrankTokyo (25936) |

Entry type | Definition |

Classification | msc 51K05 |

Synonym | Bregman distance |