Information Theory for Modeling and Inference

Amos Golan, American University
External Professor, Santa Fe Institute


In this talk I will discuss the use of information theory for modeling and inference of complex systems and problems across disciplines. Simply stated, the available information is usually too complex, insufficient, and imperfect to deliver a unique model or solution for most systems and problems. Problems with multiple solutions are called under-determined, or partially identified. Information Theory within a constrained optimization setup provides a way to deal with such complex problems under deep uncertainty and insufficient information. It provides us with a way to sort and rank solutions and then choose the one that satisfies our desired properties. It also provides us with a different way of thinking about solving (complex) problems and a way to nest models in terms of the information and decision criteria they use. It also provides new insights into basic modeling and allows us to solve inference problems that cannot be solved with conventional methods without imposing additional structure or heroic assumptions. Though Information-Theoretic inference provides us with a general framework for modeling and inference (I call it info-metrics), the exact specification is problem-specific. In this talk I will briefly summarize the basic idea via a number of graphical representations of the theory and will then provide a few examples.
Key Words: Complex Systems, Complex Data, Constrained Optimization, Decision Function, Deep Uncertainty, Entropy, Inference, Information Theory, Modeling

Amos Golan (BA, MS: Hebrew University of Jerusalem; PhD: UC Berkeley) is a professor of economics and directs the interdisciplinary Info-Metrics Institute at American University. He is also an External Professor at the Santa Fe Institute and was a Senior Associate at Pembroke College, Oxford. His research is primarily in the interdisciplinary field of info-metrics – the science of modeling, reasoning, and drawing inferences under conditions of noisy and insufficient information. He has published in economics, econometrics, statistics, mathematics, physics, visualization and philosophy journals. His most recent book is ‘Foundations of Info-Metrics: Modeling, Inference, and Imperfect Information,’ OUP (2018): Golan is a Fellow of the American Association for the Advancement of Science (AAAS).