entropy encoding

An entropy encoding is a coding scheme that involves assigning codes to symbols so as to match code lengths with the probabilities of the symbols. Typically, entropy encoders are used to compress data by replacing symbols represented by equal-length codes with symbols represented by codes proportional to the negative logarithm of the probability. Therefore, the most common symbols use the shortest codes.

According to Shannon’s theorem, the optimal code length for a symbol is


where b is the number of symbols used to make output codes and P is the probability of the input symbol.

Two of the most common entropy encoding techniques are Huffman encoding and arithmetic encoding.

Title entropy encoding
Canonical name EntropyEncoding
Date of creation 2013-03-22 12:32:22
Last modified on 2013-03-22 12:32:22
Owner vampyr (22)
Last modified by vampyr (22)
Numerical id 4
Author vampyr (22)
Entry type Definition
Classification msc 68P30
Classification msc 94A24
Synonym entropy encoder
Synonym entropy coding
Related topic HuffmanCoding