Brain Model in a Nutshell

"Growing" Brain Tissue

A key insight is to take into account the fractal nature of how the brain grows (like any other tissue) from a single stem cell. Model structure should reflect the way cells grow and connect rather than relying on mathematical structures like matrices. Connections look random but have an underlying fractal structure. On the other hand, the model needs to reflect the well known principle that "cells that fire together wire together". The fractal structure "explains" this rule in a physical way (cortical columns) but the brain can wire itself up in a way that might be seen as 4 dimensional. Connections are formed due to "closeness" in space as well as some kind of time function. 

The model should assume some level of continuous growth. In the brain, new neurons are being created and others are dying all the time. It's not known how these new neurons find their place in the structures of the brain.

Similarly, we know that the brain is, to some extent, "plastic". It allows large-scale re-wiring, probably between columns rather than within them.

Pre-frontal cortex and the "idea"

The prefrontal cortex may work just like the specialized areas of the brain, but feed back on itself to create recursive, fractal "layers" on the fly, at approximately "brain wave" frequency. We know that this process can feed back into the specialized cortex to re-use these circuits to present "ghost" images (like visual memory). Perhaps this is what we mean by an "idea" and account for how these ideas seem to flow in the experience we call consciousness. The rate that these images are formed (brain wave frequency) is interestingly similar to the frame rate of a movie or the perceptible range of audio frequency.

What would be the model of "stimulation"?

Using the example of the retina, a cell would consider itself to be "stimulated" as a function of stimulation in nearby (connected) cells. Connection would follow the fractal patter, with daughter cells of the same stem cell being considered "close" as well as those physically in contact (taking into account direct cell-to-cell communication present in all tissues). Direct stimulation would take place only at one "layer". Would that be layer zero or layer K? Or should we consider the idea of stimulation to define layer zero?

Cell logic structure

Signalling between cells must include the concept of suppression: a signal from a to b may reduce the probability that b becomes "excited'. In general, the "method" in the cell that determines to signal the event "I am excited should involve the range of simple logical constructs (and, or, nor, not) along with dynamic versions of these functions (sum over time). In short, triggering will result from exceeding a threshold described in a set of discreet non-linear functions. Inputs to these functions would be internal variables reflecting  stored "awareness" of events in connected cells. Perhaps we can assume that "time" resets to zero when the cell "fires" or at least assume the cell is in one of two discreet states: firing and not firing.

The set of equations determining whether the cell "fires" or not can be thought of as the model for the cell's "output" axons. The "input" (right hand side of the equations) represent a model of the cells dendrites (inputs). The actual set of equations (the way all these variables are related) can be thought of as the model for the neuron's internal "logic".

Some provision should be made for "noise". In other words, "firing" should not be deterministic, but a result of increasing or decreasing probability.

We may save ourselves the trouble of modelling a "not firing" event by allowing the corresponding "receiver" internal variable to "decay", requiring occasional refresh ("still firing").

The State of the Cell

With each "tick of the clock", the cell proceeds from one state to the next, applying the state model to produce a new state (list of internal variables). At the end of the "tick", it signals the "firing" event or not, then "decays" the state variables according to some internal rule. It then "goes to sleep" unless "woken up" at the next tick or notified of the need to change a state variable due to the external event of the corresponding neuron having fired. Of course, this can be generalized simply by treating the "tick" of the clock as itself an "event".

This is not particularly original concept. It's a simple-seeming idea but nasty to "debug". The event-driven design concepts are drawn from the "tool box" used by the computer's BIOS. For performance reasons, programming should at least provide for an ultimate version that binds closely to the actual computer BIOS, using the actual "tick" of the computer's clock. Obviously, we can't imagine triggering millions of cell events at each computer "tick" (which happens in the nanosecond order of magnitude). However, our model should provide for the possibility that the lists of listeners is partitioned between CPU's and that the cells can refresh themselves independently, possibly using or sharing different CPU's. At the lowest level, then, the model should not assume that everything happens on one CPU, nor that the event list is shared between all CPU's. All state data should be stored within the cell or the event list.

Cell "logic" should be kept simple to provide for direct implementationi in silicon. This would permit mass production of cells for a given "tissue type".

Signalling and events

"Firing" of a cell as an event picked up by "listeners". Ultimately, some cells will "fire" as a result of direct, outside stimulation.The synapse (connecting neuron A to B) would be modelled by the the list of cells Bi listening for firing events from cells Ai. Physical implementation constraints would be proportional to the capacity to store these lists. Dynamic performance would be determined by the refresh rate of the lists (how quickly "events" can be picked up to fire a new set of events). Ideally, the refresh rate should be fast enough so that there are "long" periods where "few" events take place. Cycling between "few" and "lots" of events should mirror brain wave frequency (order of 10 to 100 Hz).

Random "wiring"

How do we model the possibility of virtually any neuron connecting with any other if they "fire together"? This would imply dynamically adding A to the list of "listeners" to the "B fired" event. The initial list of "listeners" should be created as a result of fractal growth from a daugher cell. After that, listeners could be added or dropped as a result of a different rule ("Experiece"). Perhaps we could feed the stream of event states into a (hopfully fast) external machine that would detect correlation using old fashioned statistical methods, then feed back a stream of "suggested" new connections (listener pairs A, B). This would, in fact, simulate "learning".

Qualitative and Quantitative Measures

Even if not implemented in silicon, dynamic properties of the model could be determined and compared to what is known about the brain, such as the way cells signal, the number of cells, the size and structure of cortical columns. "Order of magnitude" estimates could give us an idea as to whether this type of model can tell us anything about how the brain works. Assuming that the brain is more complex that this model, perhaps we can bracket brain complexity, performance and the types of brain functions that can be implemented in silicon.

How can "memory" be Modelled?

I assume that "memory" is implemented in tissues similar to what is discussed above. In particular, there internal state is persistent. Their output, however, is assumed to feed back to produce "ghost" perception. At first glance, this behaviour could be simulate simply by a longer decay of internal state variables, simulating a persistent neuron-to-neuron connection (axon to dendrite). Again, such connections would be richer within daughter cells. We can't assume that the column structure applies since brain structures such as the hippocampus (known to be involved in memory) are not cortex tissues. At first blush, the "wiring" may be simulated by a fractal structure applying to one big "blob" of "tissue" called "memory".

Overall Structure

All of the above hints at a structure of similar tissues arising as variations on a theme, with every region differing mainly in the persistence of internal state variables, fractal "wiring" and the degree of looping (recursive) "wiring". Frontal cortex is "loopy" with state decaying rapidly unless refreshed. Sensory cortex is much less "loopy" but fast decaying. Connection is highly structured along fractal lines. Memory is extensively connected to everything with slow internal decay rate for internal state.

Henry Markham, Ray Kurzweil and the Artificial Brain

A Popular Fairy Tale

Comparing human brains to computers has become something of an industry lately. Like nuclear fusion, an emergent silicon mind seems to be always just around he next corner. The poster boy for this idea is Ray Kurzweil. If you want to watch a movie imagining precisely this vision, take a look at Transcendence. Transcendence is a tolerably good movie but brain capture fails in the movie for truly stupid reasons (plot spoiler) because it apparently fails to capture the "soul" and not due to any fundamental difficulty involved in creating a mind in a machine.

The monstrously expensive and spectacular failure of the Blue Brain project was conceived by Henry Markham. You can view his version of this fairy tale on TED. [1]

So How Big a Computer Do We Need?

The 19 million volumes in the US Library of Congress represent about 10 trillion characters - 10,000,000,000,000 characters. To make the analogy even approximately apt, we need to imagine each character in each book being a tiny, super-powerful computer with an operating system of millions of lines of code. As long as each of these tiny computers is 1,000 times faster than any computer we could ever build, and as long as the computers can communicate with each other at better than internet speed, we are getting into sight of the computational power of the human brain.

The Super Computer Between Your Ears

Let's take a look at how your brain "computes".

The neuron's closest analogue in a computer is the CPU chip (central processing unit), not the memory. Your computer may have terabytes of memory on board but that's almost irrelevant. Bits in memory are "dead". They only become useful when run through the CPU for processing. So we are talking about 100 billion CPU's -- one each per neuron. That's more than the number of CPU's on the planet at the moment. Per brain.

But this isn't quite right either. The neuron has complex behavior based on its genetic "programming", inputs, outputs and a bath of mostly unknown enzymes. Like every cell in the body, the neuron is as complex as a jumbo jet. You could probably model it with reasonable success with a complex program of some sort. So we basically have 100 billion PC's. That's full-scale computers consisting of on or more CPU's, a big chunk of memory (Terabytes),  and a few million lines of code. Each.

Synapse Simulation

But wait!

Active, dynamic "thinking" in the brain is not controlled the neuron. It's about how one neuron is influences another. Each connection is mediated by a synapse. There are about 100 trillion of these in your head. Each synapse connects an "upstream" neuron U with a "downstream" neuron D. Whether or not the connection U,D exists is a dynamic property of the brain. Connections are being formed and broken all the time. What's more, the strength of the connection varies due to processes like "thinking" and "experience". For example, the speed of connection depends on the existence of the myelin sheath around the axon - something that's built up or torn down depending on the dynamic "usefulness" of the connection. These connections are not "dumb" wires - each connection needs to be simulated, probably with a rather simple program, but there are trillions of them.

To make matters more interesting, the synapse is not just a dumb connector. Whether or not it will transmit a signal from U to D depends of a lot of things including the concentration and gradient of dozens of neurotransmitters in the synaptic cleft, the number of receptors for each type of neurotransmitter molecule (ready to pick up a signal from U to D) and the properties of the transmitting "upstream" part of the synapse (axial terminator). All these things are dynamic, changing thousands of times per second. Each factor depends on the others in complex ways.

Simulating all this for a synapse may be feasible, but you'd need 100 trillion powerful PC's to do it. It's hard to imagine how you would connect all these computers together but fortunately for our thought experiment, the connections are not that fast by electronic standards. The number of connections is mind boggling but we don't need to worry about the speed of the connections.

Simulating Synaptic Receptors

Optimistic authors tend to assume that such PC's would have no trouble simulating a synapse in real time (comparing the switching speed of computers to the signalling speed of neurons)  but the speed comparison needs to look at the speed of the chemical processes at the synapse, thousands of which take place in the nanosecond range simultaneously. The problem is that chemical reactions take place on a the "pica" scale, thousands of times smaller and thousands of times faster than silicon logic. We need 100 trillion PC's a thousand times faster than than any computer can be. And then there is the elephant in the room: it is by no means obvious that protein reactions can be simulated. It's a work in progress (to put it mildly). Definitely a day's work on a super computer to simulate just one reaction.

Programming and the Problem of Dynamic Non-Linear Systems

And then we need to program all this! Programming turns out to be not just hard but impossible. It's easy to imagine that the same program might work for 100 trillion synapses, but it could take decades of research to figure out how to approximately model just one synapse. At best, such research would give us a system of a few hundred dynamic non-linear equations. "Solving" such a system to predict or model behavior is known to be impossible. Things like that start to get hard with just three simple equations. Systems of equations that model change through time are called "dynamic". If the variables involved depend on each other in a non-trivial way (rates of change for example), the system is "non-linear". Almost always unsolvable, in the sense that you can't find value of all the variables that satisfy the equations.

And Then There's the Issue of Measurement

Measuring the current state of any particular synapse in a human head is also impossible for Quantum Mechanical reasons, so you have the additional problem of deciding the initial conditions for a few hundred parameters in each of 100 trillion synapses. Finding the initial conditions involves solving a set of dynamic non-linear equations which is impossible.

But Let's Not Give Up Entirely

There is no way to do this, but perhaps we can imagine a start ...

Memory elements (bits) in computers are "dead" and need to be picked up by the CPU to be processed into new memory elements (bits). Like cars that spend 99% of their time parked, almost all the "bits" in the computer sit around waiting to be funnelled through the CPU.

On the other hand, in modern computers, random access memory is refreshed thousands of times per second (it's read out and written back automatically). We could imagine a system that made the write-back a function of more than just the bit being written back. That would be referred to as a massively parallel architecture, orders of magnitude more powerful than today's fastest super computers. But still not beyond he realm of imagination. That would make the memory a "thinking" machine, constantly "unpacking" ideas at thousands of times per second. For this to work, you'd need a way to represent "ideas" in a form that could be quickly and efficiently "unpacked" into new "ideas". I can imagine a structure involving hundreds of thousands of "concepts" (English words for example) where the memory function is to write back concept B if concept A is active and B is "strongly" related to A. That would make "B" active and concepts related to "B" would be activated on the next cycle. In this picture, "A is related to B" is our synapse.

The modern Graphics Processing Unit (GPU) is the kind of parallel processor we need but we need one a few billion times bigger (same size, more capacity), thousands of times faster and with the ability to program itself on the fly. Then all we need to do is figure out what an "idea" is ...

But Maybe We Can Design a Better Brain

The one thing that keeps the "strong AI" idea alive is that a human-designed brain may turn out to be many thousands of times more efficient than its meat counterpart. We shouldn't need to simulate the actual brain to produce the "mind". If this is so, we need to understand the architecture of the mind itself (independent of how it "runs" on meat). To put it kindly, this process is in its infancy. Most AI research uses brute force to solve practical problems and has no interest in how the human mind actually works. (Hofstader hardly attends AI conferences any more).

For now, we are stuck with the meat computer between our ears which apparently zips along much faster than 100 trillion super computers. If you ask me, the singularity is not as close as Ray Kurzweil imagines.

The Blue Brain Project

The Blue Brain project is an attempt to model a tiny brain based on actual data on the brain structure of a rat. It is not even an attempt to model the entire rat's brain - just a patch of its tiny neocortex.

Artist's conception-Fractal Cortical Column

The project shows the sweeping simplifications and assumptions that are required, along with the vast computer facilities to model even these assumptions. The project investigates an intermediary structure, larger than the neuron and smaller than brain modules such as the visual cortex. This is the "cortical column". There is no general definition of what, exactly, this "column" is, but the outer surfaces of the brain (neocortex) seem to have a consistent structure of vertically associated neurons in "vertical" columns (stacked inwards from the surface). The "wiring" within the column and between columns is not random (widely considered to be a fractal structure - each column being daughters of a single stem cell). It has long been recognized that the cortical column is promising both as a subject of study (how does it work, what does it do?) and a subject of computer modelling if you happen to own a super computer. 

The Blue Brain Project seems to be the ultimate evidence that Hofstader's line of investigation (at the "meme" level) is sadly far from the mainstream, even though his model of the mind is, so far, the most successful. "Blue Brain" is working on the reductionist assumption that understanding of the mind will "emerge" from a sufficiently detailed understanding of the "fundamental" aspects of the brain, just like Quantum Mechanics is supposed to be a "Theory of Everything". The hopelessness of this idea is illustrated by the goals of the project, which attempt to model a tiny patch of cortical columns based on the brain structure of a rat.

Literature on the cortical column seems to focus on brain processes at the very origin of perception (such as the visual cortex). The assumption that the same processes are involved in (for example) formation of new concepts is so far not justified by the research.

[1] Recent reports indicate that the Blue Brain project is not going well. Markram pitches the project is here as a TED project. He is stunningly naive about how his "top down" idea of how the brain works. He parades the reductionist assumptions of how this can be "unpacked" by simulating it all on the computer. His talk illustrates a popular technique of "brain talk". Markram talks as if "columns" are well defined, let alone understood, just as others talk about "neurons" as some kind of "explanation" of thought. He mentions "10 million synapses", somehow ignorant of he fact that there are 10 trillion. He lies about "having the math" to describe neurons with a "handful" of equations. He lies by creating the impression that having such equations amounts to solving them. He lies about attempting a "real time" simulation. His bottom line: "It's not impossible to build a human brain and we will do it in 10 years".