We don’t know a lot about dreams. A recent paper by Erik Hoel The Overfitted Brain: Dreams evolved ot assist generalization at Tufts Universtity suggests that one possible reason for weird dreams is your brain is trying to avoid “overfitting”. If you don’t want to read the paper itself (long but easy read), you can read an article about it at Discover Magazine. The concept of overfitting is borrowed from deep neural networks where the network is trying to “learn” from a dataset, but focuses so tightly on that dataset that it can’t generalize to similar but not identical datasets. As Ryvar on Metafilter commented, “too often the training data had people wearing purple socks performing the activity we want recognized, so the resulting network over-emphasizes purple socks”. The solution in computer science is to include some noise or corrupted data into the dataset or randomly drop data out of the dataset. Hoel hypothesizes that maybe weird dreams are your brains attempt to do something similar.
Taken on its own, it is an interesting idea, but those of us who are not medical specialists or computer scientists can’t really do much with it. What I want to talk about today is the type of negative responses I see, looking at the conversations. Can we get a better level of discourse?
Specifically, I’m drawing from the conversations on on Hacker News here and Metafilter here.
Conversation Type 1
In this conversation, a computer scientist states that the author, a research assistant professor at Tufts, with a PhD neuroscience, is unaware that overfitting is a necessary step for achieving state of the art performance with deep neural networks (DNNs), citing several other papers. I have a couple of thoughts on this conversation.
First, I’m not even sure it is relevant. Everyone agrees that brains are not the same as DNNs. Even assuming that DNNs need to overfit (and then continue to train and come out the other side) doesn’t mean that brains took the same evolutionary path. A is like B, but A is not B. As an observer looking at some of the arguments, the whole area of neuroscience, cognitive science and deep learning seem to me to get themselves wrapped in knots over whether they are talking in metaphors, similes, analogies or explicit this is that.
Second, the person says “Many state-of-the-art deep learning models today are trained until they achieve ~100% accuracy on the training data, and then we continue to train them because once they go past this “interpolation threshold” they continue learning to generalize better to unseen data. This is known as the “double descent” phenomenon.” and then cites several papers. Ok. I read two of those papers and the abstract for a third. Those papers specifically talk about continuing to train the models by adding different data, not the same data. (One paper also pointed out that having too much data basically drowned the DNN and it got worse results.) That is essentially what Hoel hypothesizes that the brain is doing - adding more data, some of it nonsensical. Sure Hoel doesn’t mention double descent, but it doesn’t actually impact the idea. Actually, one of the papers on double descent concluded that we really don’t understand it. The commentator can feel good about the fact that he cited some other studies not mentioned in the paper. I read them, I learned something about DNNs. OK. Not sure it says anything about what the brain might be doing, particularly given the rather violent disagreement between scientists on whether the brain is sufficiently “like” a DNN that you can draw any conclusions.
Conversation Type 2
“OMG the paper has typos. Must be wrong.” Yeah, the proofreading state of the world depresses me as well. No one proof reads any more. Can we now talk about the substance?
Conversation Type 3
“Not a new idea.” Yup. Actually just a further development on earlier thoughts which is pretty typical of learning. He cited 117 research papers at the end, so I don’t think he disagrees. I agree that Hoel’s language (but more particularly the popular science press) played up the novelty of the hypothesis (and it is just a hypothesis) as a discovery.
Conversation Type 4
“Its not falsifiable and therefore is just more BS being published.” Actually the author (and several commentators in the thread) pointed out several experiments that might be done to show whether this hypothesis was true of false. Obviously this conversation was started by someone who didn’t actually read the paper, but felt they could comment after reading the headline.
Conversation Type 5
“Come back when people can actually report dreams consistently and you can connect your theory to actual data.” Yeah, that is the problem in going from hypothesis to theory. Lots more work is needed. I found the idea interesting, but I can understand a lot of other people won’t be interested until it has been further developed.
Conversation Type 6
“Who does this person think they are? Such hubris! My qualifications demonstrate that the author doesn’t know what they are talking about.” OK. Can you explain disagreement or is incoherent rage your only mode of discussion?
As usual, feel free to disagree using this contact link. My world view is a hypothesis, not a belief.
