Statistical Mechanics

From scaling laws to emergent capabilities, modern AI systems, in particular large language models, display various behaviours that nobody understands. Statistical mechanics in physics offers a leading tool to analyse and explain these observed phenomena, and thereby answering some of the extremely crucial questions in modern AI theory. Some references include this review article} and this book. The student will review the basics of the commonly used methods, delve into the modern literature, and together with the advisor agree on a particular problem to tackle, analyse the problem theoretically, and run small experiments.