Communications in Number Theory and Physics

Volume 15 (2021)

Number 2

Entropy modulo a prime

Pages: 279 – 314

DOI: https://dx.doi.org/10.4310/CNTP.2021.v15.n2.a2

Author

Tom Leinster (School of Mathematics, University of Edinburgh, Scotland, United Kingdom)

Abstract

Building on work of Kontsevich, we introduce a definition of the entropy of a finite probability distribution in which the ‘probabilities’ are integers modulo a prime $p$. The entropy, too, is an integer $\operatorname{mod} p$. Entropy $\operatorname{mod} p$ is shown to be uniquely characterized by a functional equation identical to the one that characterizes ordinary Shannon entropy. We also establish a sense in which certain real entropies have residues $\operatorname{mod} p$, connecting the concepts of entropy over $\mathbb{R}$ and over $\mathbb{Z} / p \mathbb{Z}$. Finally, entropy $\operatorname{mod} p$ is expressed as a polynomial which is shown to satisfy several identities, linking into work of Cathelineau, Elbaz–Vincent and Gangl on polylogarithms.

Keywords

entropy, $p$-derivation, information loss, modular arithmetic, fundamental equation of information theory, Faddeev’s theorem

2010 Mathematics Subject Classification

Primary 94A17. Secondary 11A07, 11A99, 11T06, 13N15.

The author was supported by a Leverhulme Trust Research Fellowship.

Received 28 March 2019

Accepted 27 November 2020

Published 18 June 2021