From Cross Validated (stats.stackexchange.com, a web site which I think deserves to be better known): What is the best introductory Bayesian statistics textbook?
Some of the recommendations from this thread that I’ve seen before:
- John Kruschke,Doing Bayesian Data Analysis with R and BUGS. This book could be improved by having kittens on the cover instead of puppies.
- At a somewhat more advanced level is Gelman, Carlin, Stern, and Rubin, Bayesian Data Analysis. I own a copy of this book but have not had the time to sit down and work through it carefully.
- Not entirely Bayesian, but entirely fun, is David MacKay’s Information Theory, Inference, and Learning Algorithms. (You can download it for free from the author’s web page; this is happily true of a lot of books published by Cambridge.)
God plays dice 