A Master Plan for Intelligent Brains.

This is not an everything goes forum, but rather a place to ask questions and request help for developing your ideas.

A Master Plan for Intelligent Brains.

Postby hyksos on August 12th, 2018, 10:01 am 

I have had this intuition about how brains work for over 10 years. The spark that lit the fire of this idea for me was the (2005) book Wider Than the Sky , authored by Gerald Edelman.

Prior to reading that book, I already knew the basic outline of how a mammalian brain operates to produce intelligence from these nagging facts :

1. Action potentials in biological neurons operate in the millisecond range. Thus they are literally millions of times slower than transistors in integrated circuits.

2. The brain makes up for the slowness of neurons by attacking computation with massive parallelism. The parallelism is absolute and fine-grained over billions of neurons all operating simultaneously in the head.

3. Desktop computers are stupid as rocks, because they are operating on a Von Neumann architecture. They are dumb machines that collect instructions from RAM and execute them in sequence.

4. The cortex is organized in a hexagonal grid of columns, called "Cortical columns." The evolution of the human brain is an attempt to squeeze as many cortical columns into our heads as possible. It has done this by making the outer layer of our brains very folded.

To appreciate the importance of the cortical column, I refer you to this lecture.

So a master plan for an intelligent brain would address and lay-to-rest the following questions :

  • Why is more cortical columns better?
  • What is a computational model for massive fine-grained parallelism?

I present now the answer. CC = "Cortical Column"

Every CC is a coupled harmonic oscillator, that builds axonal connections between neurons in a random way. When excited by an input signal, the CC will set up a slightly long-lived pattern of activation after the input stimulus has been removed. The CC's activity after input stimulation will not die out immediately , and simultaneously not go on forever producing random noise. It will, however occupy a dynamics regime that reconciles both attributes.. a sort of mixture of dying out quickly and going for a little while producing sensible patterns. For the purposes of this article, it is best to visualize them producing completely random noise.

When a mammal experiences new stimuli in full view (feelings, seeing, hearing , etc) the entire stimulus is present, and that stimuli activates all ("all") the CCs in its cortex at the same time with the early portion of the stimuli. Because there are many CCs in its cortext, it is statistically likely that at least one of the CCs will (by accident) happen to resonate with the end-portion of the stimulus. If such resonance takes place, the connections to that CC are strengthened. CCs whose post-stimulus pattern formation are out-of-synch with the stimuli will experience weakening of their connections going out of the CC to the rest of the brain.

The cortex is then acting as a "reservoir" of coupled harmonic oscillators, where the oscillator that just happens to predict a correct output pattern, is rewarded by strengthening its connections to the rest of the brain. Those CCs who do not predict correctly are punished by having their connections weakened against the rest of the far-away CCs in the brain. Over time, the cortex will establish connections amongst CCs who correctly predict observed sequences in the real world, and weakening and dampening the "noise" produced by those failed CCs who did not synch up with the stimulus observed in the past.

The eventuality of all this learning is that the brain becomes a well-honed prediction machine. Given only a portion of a stimulus, the brain will automatically reconstruct the missing parts, because the mere activation of the "early" CCs of a stimulus will naturally activate those "later" CCs because their connections were previously strengthened.

With this theory, we can explain why the human cortex is an attempt to place as many CCs into the head as possible. The more random CCs that are available for stimulation, the more likely it is statistically that there will "just happen" to be a complex oscillator that perfectly matches the observed stimulus very closely. This is the same type of mathematics that describes why it is better to play more than 1 lottery ticket to win the lottery. It would be even better if you could purchase 20 tickets instead 2. Your odds of hitting the lottery numbers is even better if you play 1 million tickets. The human cortex is packing as many harmonic oscillators into the human head in order to raise the probability of a match between stimuli and oscillator in the developing/learning child.

Over the course of early development, a child's brain is selecting those CCs that are effective at predicting observational sequences in the world, and killing off those CCs which are ineffective. In other words, psychological development and learning in early life is a form of Neuronal Group Selection. https://www.goodreads.com/book/show/1120069.Neural_Darwinism

AI and NNs
While I have had this nagging intuition about brain function for 10 years now, I have never written about it or tried to communicate my ideas to others. Today I have suddenly come out with it in this article. Why now?
The reason I became motivated to suddenly write this article is because recent advances in Artificial Intelligence appear to be confirming my intuitions. I feel a boost of boldness about this idea, since it is coincidentally dovetailing with certain types of artificial neural networks.

I am not referring to Deep Learning here. In popular science media, Deep Learning has become a hype factory. Rather, the part of artificial neural networks that concerns this article is something called Reservoir Computing.



In Echo State Networks and Liquid State Machines, there is a single reservoir of randomly-connected nonlinear units. Thus in every training case, there is a single reservoir used. For our purposes, we imagine that every CC in the mammalian brain is its own, separate reservoir.

Follow up research would attempt to uncover more real-world stats regarding the structure of cortical columns (CCs) and how many total CCs are found in the brains of various mammals.
User avatar
Active Member
Posts: 1379
Joined: 28 Nov 2014
BraininvatDragonFly liked this post

Re: A Master Plan for Intelligent Brains.

Postby Braininvat on August 12th, 2018, 12:40 pm 

Good stuff. Edelman is a good starting point, though I've never been persuaded by his insistence that consciousness must be biological, and only arises from neural Darwinism. But it's possible that AI, to succeed, must go in the direction of cortical column architecture with some kind of plasticity at the hardware level. (but a nagging voice inside keeps telling me, hey, airplanes were not designed like bird wings, yet they still can fly...) Your harmonic oscillator theory is creative and fascinating, though I would appreciate a fuller account of what a harmonic oscillator does, in this context. I think, if the theory is borne out by experiments, it could really help understanding of how memory works.
User avatar
Forum Administrator
Posts: 6693
Joined: 21 Jan 2014
Location: Black Hills

Re: A Master Plan for Intelligent Brains.

Postby hyksos on August 13th, 2018, 1:25 am 

Your harmonic oscillator theory is creative and fascinating, though I would appreciate a fuller account of what a harmonic oscillator does, in this context.

A randomly-connected neural network can be stimulated once with a pattern, and if its connectivity is sparse enough (a kind of goldilocks measure) then patterns will continue to emerge in the network long after the stimulus has disappeared. The signals will "bounce around" in the network, producing a sequence of complex activation patterns. The network will either 1. die out. 2. Cycle endlessly in an attractor basin. 3. Wander aimlessly in a chaotic dynamic.

This "bouncing around" was actually called an echo by early researchers. It was an "echo of the input" : hence the name Echo State Networks. http://www.scholarpedia.org/article/Echo_state_network

The human genome consists of 30,000 genes. But the human brain contains 1.5 x1014 synapses. It is preposterous that our genome could specify the microcircuitry of our entire brain. We must conclude that our genome only tells us how to grow neuronal columns, and then give a rough statistically-guided method for growing axons and dendrites between them.

The cortical column is (generally speaking) a randomly-connected group of neurons. It is not totally random, full stop. Rather our genes will coordinate in a rough way the total number of neurons in a column and the total number of neighboring cells which they connect to and listen for. But all columns are unique in their details after that. The micro-circuitry in the developing columns of a human fetus are , for all intents, random. It might at first seem that random connectivity among neurons is "useless" for effective thinking , responding, and acting.

But this intuition is destroyed by the fact that the resulting child will later have 100 million of these columns. By statistical probability, some of these columns will accidentally align their cycling activations with sequences invoked by stimuli from the real world. If they do by accident, then those "effective" predictors are rewarded. If they fail to do so, the ineffective ones are punished. "Punishment" at the cellular levels means decreased excitation strengths, lowered connectivity at the synapse, or even active inhibition by inhibitor cells. "Reward" is, of course, strengthened synaptic connections to far-away co-firing columns.

Eventually only a portion of a stimulus need be present (for example, an obscured image) and the cycles of activation of the column will still re-construct the missing parts naturally as they continue to produce sequences. This reconstruction is automatic, and might refer to it as the brain "predicting" something. In another scenario, a re-activated set of columns would be called "remembering".

In artificial intelligence research, a single network with a single reservoir is used to reconstruct a narrow task. Imagine instead a large set of independent reservoirs operating in parallel. (Say 6 million of them or so). That large set of groups begins to look like a mammalian cortex.

Since each reservoir is random, we begin to see why having more and more of them is advantageous. We might gain a plausible answer as to why the brains of higher primates show an attempt to pack as many of these things inside the skull as possible. Having more of these columns "ups the odds" that at least a few of them will match a complex sequence of inputs derived from perceptions and sensations. Furthermore, it increases the fidelity of the correspondence between the brain's produced signal and the actual stimuli.

To convince one's self that this must be true, make the numbers larger and larger. Instead of 1.0 x 108 cortical columns in the human head, each with random connectivity, imagine instead 1.0 x 1018 , or even 1.0 x 1025

100,000,000 (human brain)
1,000,000,000,000,000,000 ( = 1018 )
10,000,000,000,000,000,000,000,000 ( = 1025 )

Now the odds are nearly certain that there will be an accidental match between a complex sequential stimuli, and the activations of a particular column. Without any high-level mediator, those columns which are of the highest predictive fidelity will have their connections strengthened, and the vast ocean of de-synched and "wrong" columns would be suppressed, leaving only the correct predictor super-activated in the silence. I should note: the strengthening of connections are connections between columns, not the connections inside the column itself. The suppression would apply to entire columns, not to individual cells in those suppressed columns.

(For comparisons sake -- a brain with 10^25 cortical columns would be roughly the size of Lake Superior.)

User avatar
Active Member
Posts: 1379
Joined: 28 Nov 2014

Return to Personal Theories

Who is online

Users browsing this forum: No registered users and 3 guests