| Version 11 |
Version 10 |
| {\bf Definition} |
{\bf Definition} |
|
|
| The {\it Hartley function} is a \PMlinkescapetext{measure} of uncertainty, introduced by Hartley in 1928. If we pick a sample from a finite set $A$ uniformly at random, the \PMlinkescapetext{information} revealed after we know the \PMlinkescapetext{outcome} is given by the Hartley function |
The {\it Hartley function} is a \PMlinkescapetext{measure} of uncertainty, introduced by Hartley in 1928. If we pick a sample from a finite set $A$ uniformly at random, the \PMlinkescapetext{information} revealed after we know the \PMlinkescapetext{outcome} is given by the Hartley function |
| \[ |
\[ |
| H(A) := \log_b(|A|). |
H(A) := \log_b(|A|). |
| \] |
\] |
| If the base of the logarithm is 2, then the uncertainty is |
If the base of the logarithm is 2, then the uncertainty is |
| measured in bits. If it is the natural logarithm, then the \PMlinkescapetext{unit} is nats. It is also known as the Hartley entropy. |
measured in bits. If it is the natural logarithm, then the \PMlinkescapetext{unit} is nats. It is also known as the Hartley entropy. |
|
|
| \bigskip |
\bigskip |
|
|
| {\bf Remark:} |
{\bf Remark:} |
|
|
| The Hartley function is a special case of Shannon's entropy. Each element in the sample space $A$ is associated with probability $p=1/|A|$. For an element $\omega\in A$, the Hartley \PMlinkescapetext{information} of the event $\{\omega\}$ is $-\log(p)=\log(|A|)$, which is constant over $\omega\in A$. The average \PMlinkescapetext{information} over the whole sample space is thus also equal to $\log(|A|)$. |
The Hartley function is a special case of Shannon's entropy. Each element in the sample space $A$ is associated with probability $p=1/|A|$. For an element $\omega\in A$, the Hartley \PMlinkescapetext{information} of the event $\{\omega\}$ is $-\log(p)=\log(|A|)$, which is constant over $\omega\in A$. The average \PMlinkescapetext{information} over the whole sample space is thus also equal to $\log(|A|)$. |
|
|
| \bigskip |
\bigskip |
|
|
| {\bf Characterization} |
{\bf Characterization} |
|
|
| The Hartley function only depends on the number of elements in a |
The Hartley function only depends on the number of elements in a |
| set, and hence can be viewed as a function on natural numbers. |
set, and hence can be viewed as a function on natural numbers. |
| R\'enyi showed that the Hartley function in base 2 is the only |
R\'enyi showed that the Hartley function in base 2 is the only |
| function mapping natural numbers to real numbers that |
function mapping natural numbers to real numbers that |
| \PMlinkescapetext{satisfies} |
\PMlinkescapetext{satisfies} |
| \begin{enumerate} |
\begin{enumerate} |
| \item $H(mn) = H(m)+H(n)$ \ \ \ (\PMlinkescapetext{additivity}), |
\item $H(mn) = H(m)+H(n)$ \ \ \ (\PMlinkescapetext{additivity}), |
|
|
| \item $H(m) \leq H(m+1)$ \ \ \ (monotonicity), and |
\item $H(m) \leq H(m+1)$ \ \ \ (monotonicity), and |
|
|
| \item $H(2)=1$ \ \ \ (normalization). |
\item $H(2)=1$ \ \ \ (normalization). |
| \end{enumerate} |
\end{enumerate} |
|
|
| Condition 1 says that the uncertainty of the Cartesian product of |
Condition 1 says that the uncertainty of the Cartesian product of |
| two finite sets $A$ and $B$ is the sum of uncertainties of $A$ and |
two finite sets $A$ and $B$ is the sum of uncertainties of $A$ and |
|
$B$. Condition 2 says that a larger set has larger uncertainty.
|
$B$. Condition 2 says that larger set has larger uncertainty.
|
