Learning Bayes from books and online classes

In 2012 I wrote a couple of posts on how to learn statistics without going to grad school. Re-reading it now, it still seems like pretty good advice, although it’s a bit too machine learning and Coursera heavy for my current tastes. One annoying gap at the time was the lack of online resources for learning Bayesian statistics. This is no longer the case, and so here are my top three resources for learning Bayes.

Richard McElreath from the Max Planck Institute for Evolutionary Anthropology recently published the second edition of Statistical Rethinking. In the book, he builds up to inference from probability and first principles and assumes only a basic background in math. I don’t love the obscure chapter names (makes it hard to figure out what’s inside) but this is the kind of book I wish I had when I was learning statistics. The example code had been ported to lots of languages including Stan, PyMC3, Julia, and more. Richard is currently teaching a class called “Statistical Rethinking: A Bayesian Course” with all the materials including lecture videos available on GitHub. For updated videos, check out his YouTube channel.

Aki Vehtari from Aalto University in Finland released his popular Bayesian Data Analysis course online — you can now take it at your own pace. This course uses the 3rd edition of the Bayesian Data Analysis book, available for free in PDF form. This is probably the most comprehensive Bayesian course on the Internet today — his demos in R and Python, lecture notes, and videos are all excellent. I highly recommend it.

For those of us who learned statistics the wrong way or who want to see the comparison to frequentist methods, see Ben Lambert’s “A Student’s Guide to Bayesian Statistics.” His corresponding YouTube lectures are excellent and I refer to them often.

Although not explicitly focused on Bayesian Inference, Regression and Other Stories by Andrew Gelman, Jenifer Hill, and Aki Vehtari is a great book on how to build up and evaluate common regression models while using Bayesian software (rstanarm package). The book covers Causal Inference, which is an unusual and welcome addition to an applied regression book. The book does not cover hierarchical models which will be covered in the not-yet-released “Applied Regression and Multilevel Models.” All the code examples are available on Aki’s website. Aki also has a list of his favorite statistics books.

Finally, I would be remiss not to mention my favorite probability book called “Introduction to Probability” by Joe Blitzstein. The book is available for free in PDF form. Joe has a corresponding class on the EdX platform and his lecture series on YouTube kept me on my spin bike for many morning hours. Another great contribution from team Joe (compiled by William Chen and Joe Blitzstein, with contributions from Sebastian Chiu, Yuan Jiang, Yuqi Hou, and Jessy Hwang) is the probability cheat sheet, currently in its second edition.

What are your favorite Bayesian resources on the Internet? Let us know in the comments.