Deep Neural Networks Benefit from Teaching Themselves the Fundamentals of Physics… and The Nature of Order?

Published by Ellen Holty on

MIT Technology Review posted a really interesting article based on a paper published in Arxiv last week about how deep neural networks learning to identify photos became more efficient as they learned the basic principles of physics. This is Lin and Tegmark’s “deliberately contrived and over-simplified” illustration:


Illustration from the original paper showing “Causal hierarchy examples relevant to physics (left) and image classification (right).” You’ll have to read the paper to make complete sense of it!

It turns out that while they were pixel-crunching terabytes of cat and dog imagery, the machines derived some basic laws of physics that made their later work much faster. A couple of these patterns include symmetry, locality (the relationship between items close together is stronger than items far apart), and the structural hierarchy of physical reality (subatomic particles assemble into atoms, molecules, into cats and dogs).
These first principles- and patterns- of physics can help us think better as well, especially when we integrate them into the set of mental models we already use.  Consciously expanding and refining our set of mental models puts us in company with Warren Buffett and Charlie Munger, whose models help them distill tremendous value from enormous data sets.


A handy table that hints at fascinating opportunities for further study.

It occurred to me that these machine-learned first principles of physics have some commonality with the first principles or patterns of beauty that Christopher Alexander identified in his amazing work The Nature of Order. For example, locality is reminiscent of his patterns, Strong Centers and Local Symmetry, while structural hierarchy is reminiscent of the Echo Similarity and Levels of Scale Proportion unfolding in the process of morphogenesis.


Source: p2p foundation

If Alexander’s fifteen patterns can be used to understand what humans find alive and beautiful (just as first principles of physics can help explain the nature of the universe), then it might be interesting to look at what patterns or first principles neural networks find in large sets of sentiment data (what we find alive and beautiful).  If a deep neural network applied to imagery of physical structures derives first principles of physics, could a similar experiment deduce some first principles of aesthetic judgment (like Alexander’s theorized fifteen patterns) from an analysis of images and related sentiment data?

So many interesting questions.