I love the Jerry Seinfeld bit about professional sports teams. He points out that none of the players representing a city are actually from that city. In fact, other than the name on the shirt, there is nothing about one team that is necessarily more representative of a city than the other team. Thus, we are essentially “rooting for laundry.”

Seinfeld, in only a way he can, skillfully consolidates disparate information associated with an absurd truth. In fact, that is what most humor does. That is why we, throwing our heads back in laughter, say, “that is so true!”

We long to be surprised by the novel and the beautiful. When we master a new skill, learn a hard-taught truth, or look back on beauty formed in our character by the trials of life, we either shout “eureka!” or we shake our heads, sigh, and say “that was hard, but I wouldn’t trade it for anything.” The value of discovery is not merely its truth; the value is in the costly process we undergo to obtain the truth.

Compression: Not Just a Term for Booty Shorts

This week I read a paper about an idea in the field of deep learning called “compression.” Jürgen Schmidhuber, a computer scientist from Munich, wrote a wonderful paper on the subject as it applies to both human and machine learning. In short, the paper’s premise is that compression is the development of simplifying explanations using a series of observed truths and patterns in a data set.

Compression occurred in Seinfeld’s joke. There exists in the universe a data set of truths related to professional sports:

• Leagues are organized by geography. Each team represents a city.

• People from each city become fans of the team in their city, but have no real influence over the organization, strategy or performance of each team.

• The players who play for each city are usually not from each city. Often, they originate from halfway around the world.

• Regardless of this random collocation of people, the fans and players assume a unified position which is memorialized by uniforms.

• If a player does poorly, the fans ridicule him. If a player does well, the fans adore him. If he leaves to join another randomly selected geographically represented group, they ridicule him once again.

None of these truths are secrets, but the sum of the parts make the whole unintelligible until compressed into a concise notion: “we are rooting for laundry.” Our brains love things like this. We get to essentially delete each bullet point above (called a “regularity” by Dr. Schmidhuber), because its truth, without losing any of its unique applicability, has been compressed into a simpler, smaller regularity.

Schmidhuber astutely points out that artists and scientists have the same motives:

“All of them try to create new but non-random, non-arbitrary data with surprising, previously unknown regularities. For example, many physicists invent experiments to create data governed by previously unknown laws allowing to further compress the data. On the other hand, many artists combine well-known objects in a subjectively novel way such that the observer’s subjective description of the result is shorter than the sum of the lengths of the descriptions of the parts, due to some previously unnoticed regularity shared by the parts.”

Newton’s Law of Gravity is similar. Newton observed that apples seem to fall from trees the same way every time. He went on to compress the data set including every possible apple, every possible tree, and every possible fall into “gravity.” The sum of the data set of “every apple ever falling” is too large to deal with for anyone’s brain, and yet any 5th grade science student knows about gravity.

Dr. Schmidhuber goes on to submit that compression occurs in an algorithmic framework:

1. Storage: We store as much data as possible for future use. It goes in uncategorized and is often forgotten. This is a pretty interesting notion given that we live in the age of rapid data creation through sensors and computers. See Bullet #2 in my Takeaways from Mary Meeker’s Internet Trends, Part 2.

2. Improve subjective compressibility: The obvious form of cognitive improvement is compression - deriving insights from data. The subtle form of improvement is the notion of improving the compressing machinery itself. To improve compression, we must… compress. In Chapter 2 of The Shallows: What the Internet is Doing to Our Brains, Nicholas Carr writes in detail about how, since Freud, the overwhelming view in psychology is that our brains change. Now we know that our physical brains, not just our “minds” are plastic. Carr gives two examples: the first, a study by Dr. Edward Taub on a group of violinists whose right brains were considerably larger than normal due to their left hands fingering the instrument. The second, a study in the 1990s which showed that London cab drivers have larger posterior hippocampuses, the area that stores and manipulates spacial awareness. For more on the subject of our plastic brains, I recommend chapter 2 of the book.

3. Develop intrinsic rewards for compression progress. We are created to be curious: our own bodies teach us to crave learning. When we learn new things, our brains reward us with dopamine, the feel good neurotransmitter. There are external rewards as well, such as praise, money, or both, but there is a clear distinction between intrinsic and external. Ultimately, creating an intrinsic reward system seems to be the key to creating a truly human-like learning system.

4. Maximize the intrinsic rewards: Simply put, the greater the reward, the greater the reinforcement of the behavior. He ultimately describes this as the key to “unsupervised learning.”

Furthermore, our brain affinity to challenge that is neither too easy nor too hard. Picture a spectrum, where one end is labeled “Obvious” and the other “Impossible to Understand.” Our brains hate both. If something is Obvious, it is already compressed and no intrinsic reward lies there. If something is Impossible to Understand, it is too hard, and again, no reward lies there. In the middle, where something is not yet compressed but is "compressible”? That’s the good stuff.

Compression and Meaning

Good art opens our eyes to regularities in the world in a way that initially surprises us. That feeling you get when you see something beautiful is your brain rewarding you for a learned compression of data. Beautiful art, that song lyric that “gets you,” a new bit of knowledge observed through science, a pithy Winston Churchill quote, a novel, a sublime waterfall, all consolidate something you know to be true about the universe.

Perhaps this is our very purpose: that we might learn something about absolute truth and beauty if we earnestly observe and compress the world around us. If I fail to see beauty in a piece of art, it is possibly because the art is too obvious or too complex. Alternatively, I may need to add more data and do more compressing. Perhaps this is at the root of all kinds of self awareness.

Robots and Compression

All of this begs the question: what about AI? I do not know. Can an intrinsic reward system like the one in our minds be replicated? If guys like Schmidhuber think we have little knowledge about the human brain, it is pretty difficult to make a suggestion one way or another. Perhaps, for all the fear we have over artificial intelligence, it will help us find more beauty and truth than we ever imagined.