entropy encoding EntropyEncoding 2002-03-08 15:13:55 2002-03-08 15:13:55 Definition % this is the default PlanetMath preamble. as your knowledge % of TeX increases, you will probably want to edit this, but % it should be fine as is for beginners. % almost certainly you want these \usepackage{amssymb} \usepackage{amsmath} \usepackage{amsfonts} % used for TeXing text within eps files %\usepackage{psfrag} % need this for including graphics (\includegraphics) %\usepackage{graphicx} % for neatly defining theorems and propositions %\usepackage{amsthm} % making logically defined graphics %\usepackage{xypic} % there are many more packages, add them here as you need them % define commands here An \emph{entropy encoding} is a coding scheme that involves assigning codes to symbols so as to match code lengths with the probabilities of the symbols. Typically, entropy encoders are used to compress data by replacing symbols represented by equal-length codes with symbols represented by codes proportional to the negative logarithm of the probability. Therefore, the most common symbols use the shortest codes. According to Shannon's theorem, the optimal code length for a symbol is $$-\log_b P$$ where $b$ is the number of symbols used to make output codes and $P$ is the probability of the input symbol. Two of the most common entropy encoding techniques are Huffman encoding and arithmetic encoding.