Probability theory

  • Probability theory and examples Renowned as the ‘Bible’ of probability theory, this book offers a comprehensive understanding of basic probability concepts, using measure theory language. It encompasses various fundamental concepts such as random variables, sample space, stochastic processes, martingales, expectation, the Law of Large Numbers, Central Limit Theorem, Gaussian processes, Markov processes, and more.

  • High dimensional probability theory This book is an invaluable asset in learning theory, particularly covering aspects like generative capability, sample complexity, statistical convergence rate, etc. It presents many elementary inequalities in probability theory, such as Markov inequality, Chebyshev’s Inequality, Gaussian Concentration Inequality, and concepts like Sub-Gaussian distribution, log-concave distribution.

Functional analysis

Information theory

  • Element of information theory This classic reference book in the field of information theory explains essential concepts like entropy, mutual information, KL-divergence, etc., and their interpretations in information theory. It’s a valuable resource for machine learning researchers.

  • Concentration of Measure Inequalities in Information Theory, Communications and Coding (Second Edition) An easily digestible monograph, it details basic measure concentration inequalities, optimal transportation inequalities, information inequalities and their relations, including famous inequalities such as Talagrand’s inequality, HWI inequalities, logarithmic Sobolev inequalities.