Max Welling's "Intelligence Per Kilowatt-Hour"

Complement of this blog post.

The talk starts from high-level physics analogy but in fact it's about energy-efficient machine learning in the end. In this blog post I am more interested in physics part.

Helmholtz: Free Energy=  Energy - Entropy 

What is the "ability to perform physical work"?

What is the "level of organization, information of  a system"? He didn't explain. 

It From Bit 

Entropic Forces and Gravity
Entropy is degree of ignorance
Bayes
Rissanen: Minimum Description Length Principle
Hinton: Variational Methods
Large Scale Approximate Bayesian Learning: MCMC versus Variational Bayes