recommanded mathmatical references
Probability theory
-
Probability theory and examples Renowned as the ‘Bible’ of probability theory, this book offers a comprehensive understanding of basic probability concepts, using measure theory language. It encompasses various fundamental concepts such as random variables, sample space, stochastic processes, martingales, expectation, the Law of Large Numbers, Central Limit Theorem, Gaussian processes, Markov processes, and more.
-
High dimensional probability theory This book is an invaluable asset in learning theory, particularly covering aspects like generative capability, sample complexity, statistical convergence rate, etc. It presents many elementary inequalities in probability theory, such as Markov inequality, Chebyshev’s Inequality, Gaussian Concentration Inequality, and concepts like Sub-Gaussian distribution, log-concave distribution.
Functional analysis
- Functional Analysis (Math 920) Lecture Notes
- Lecture Notes on Functional Analysis: With Applications to Linear Partial Differential Equation These lecture notes offer an easy-to-follow introduction to functional analysis. They cover foundational inequalities widely used in probability theory, measure concentration, and information theory, such as the Minkowski inequality. The notes also introduce key concepts in optimization, like dual space, the Riesz representation theorem, and weak topology (weak convergence and so on).
Information theory
-
Element of information theory This classic reference book in the field of information theory explains essential concepts like entropy, mutual information, KL-divergence, etc., and their interpretations in information theory. It’s a valuable resource for machine learning researchers.
-
Concentration of Measure Inequalities in Information Theory, Communications and Coding (Second Edition) An easily digestible monograph, it details basic measure concentration inequalities, optimal transportation inequalities, information inequalities and their relations, including famous inequalities such as Talagrand’s inequality, HWI inequalities, logarithmic Sobolev inequalities.