The Hawkins Frame - 1

In the long run, Hawkins abandons his initial description of "Frame" and (to my mind) wanders into simple Object Oriented Design and Mathematical constructs that lie behind Quantum Mechanics. I'm not sure gets this, but he starts with a frame-based description of consciousness, which is a fine example of putting this phenomenon on the test bench. Many philosophers claim this is not possible. Daniel Dennet is happy to allow what we say about our experience to be evidence of what that experience is. The problem with Dennet is that he was writing decades before Hawkins used the powerful frame language to speak of what it "feels like" to be in the world. Armed with Hawkins' way of speaking, it may be a good idea to loop back and read Dennet again.

With apologies to Hawkins, this is what I think a frame is:

• It is a frame of reference like Cartesian space
• It contains objects, each of which has a location in the space - for example, (x,y,z) coordinates.
• In object-oriented terms, coordinates are attributes of the object within the space. Other attributes include orientation (such as an (x,y,z) vector).
• Although it is not entirely clear, it seems that these objects also have a "velocity" vector - they may be moving through the reference frame.
• The observer occupies a position in the framework and has a complex set of objects that can be used to interrogate objects in the space. In a sense, the observer experiences this set of objects as "self". An example would be fingers exploring his beloved coffee cup. We will call these objects "sensors".
• A key element of Hawkins' account is the interaction between sensors and objects in the space to determine (or guess) the attributes of the objects. This interrogation results in an updated version of the frame itself - as modeled by the brain.
• All this is experienced dynamically - there is time in Hawkin's frame. When we speak of our experience, we call it "now", but "now" is always changing. This is the flow of time. We can be grateful for any model that makes an attempt to explain this most important aspect of the experience.
• Unlike an abstract mathematical "object", a frame is an experience. When we describe it as a mathematical object, we rely on analogy. This is the entire point of the Hawkins "frame". It is claimed to be the fundamental unit of experience, not the actual "God's Eye" view of the world, nor an abstract description of the world "as it is". It is a mental model, presumably implemented with neurons and synapses. As far as I know, this is the only attempt to formalize experience itself.

Some elements here remind me of Einstein's insight - particularly placing the observer in space and making observations relative to the frame of reference. In both cases, there is no "God's eye" view of the space, only the observations of the observer and the conclusions the observer reaches, such as the position and velocity of objects in the frame of reference. If you are familiar with Einstein's view of the world, Hawkins' "frames" are easy to picture.

Along the same line, Einstein was interested in the ability to interrogate objects to determine their properties. It turns out that it is not possible to learn certain pairs of properties exactly, say a and b. Precision in the measurement of a loses precision in b. Worst yet, counterintuitive, certain objects' properties don't exist until they are measured. But that is a different subject.

The point is that Einstein's mental picture of the Universe is a frame. A very powerful one. What it lacks is a sense of "now". Time is just another dimension, like the x,y,z of space.

Hawkins is shy about generalizing this model - sticking to our familiar 3-dimensional world. He could have escaped this a little by referring to the attributes of an object, which can be more or less unlimited. It would have been clearer (to me at least) if his model included the interrogation of the object, returning another frame since the object itself has all the attributes of a reference frame. Certain sensors are limited to certain queries - "touch" returns information that "look at" does not.

Anyone familiar with Object-Oriented design would wish to add (at least) object states which add some complexity to the result coming back from a query sensor. Other things like the concept of "interface" would be helpful to isolate the "experience" of the object from any secret and unobservable properties that may or may not produce the experience. The whole of OOD* seems applicable, and for good reason: OOD* has been developed precisely to describe (at least) the real world.

That said, there are aspects of the world where OOD doesn't seem very helpful, such as chemistry, cosmology, logic, mathematics, and Quantum Mechanics. Hawkins runs into these limitations since he claims his theory covers all we can know. His theory is only a re-invention or application of OOD - a special case. OOD covers a lot more of what we can know. It underlies the design of virtually all modern computer systems, including the Internet. But we must turn to other ways of "knowing" when leaving the familiar 3-dimensional world described in Hawkins' examples.

To be fair, the areas of knowledge where the Hawkins frames start to break down are precisely those that many find difficult to understand. We are also left with the biggest and most urgent things we need to understand: human-to-human interactions.

--------

* OOD = Object-Oriented Design

Frames All The Way Down

I have been rather dismissive of Hawkins' "Thousand Brains" idea but I can't help thinking he's on to something, specifically:

• Once you broaden your understanding of what a "frame*" is, it is quite plausible that people think in frames. The most powerful computer system we have built so far, the Internet, runs on a relatively simple "frame language", understood by over a billion "hosts" including the one you are reading this on.
• Hawkins provides a plausible description of human experience in terms of "frames" somehow being the unit of experience.
• It is plausible that the neocortex has some kind of structure that implements frames, including the ability to create frames "on the fly" and use them as a unit of memory and reasoning.
• Thousand Brain is not too dissimilar from the theory presented in Surfaces and Essences, which presents analogy as the "fuel of thought". "Surfaces" is really talking about language, which is a brain-to-brain communication device.  Theories of language, such as Chomsky's use frames to develop a theory of how language works.

Hawkins gets tripped up trying to describe complex frames, especially the ones that don't admit to a spatial interpretation. Object-Oriented Design has a rich language that applies directly. Starting at the simplest example, Hawkin's coffee cup, we can speak of objects as "frames". Object-Oriented concepts are extremely powerful in representing the real world - one reason these concepts underlie the Internet and some of its fundamental tools, such as Java.

Hawkins gets into trouble when he starts to speak as a neuroscientist, which he is not. He doesn't get published in Scientific Journals. In spite of his extensive knowledge of the subject, his ideas about how, exactly, columns in the neocortex create frames are sketchy, to say the least. But even here, he is pointing to a structure that is replicated all over the neocortex (the famous thousand brains). If we assume that we think in frames we may be forgiven to think that some structure in the brain is involved. Due to the enormous parallelism and recursiveness in experience, we would expect a large number of similar structures, not some kind of "organ".

Hawkins is particularly interested in the predictive nature of consciousness. I think he could be challenged on this. The brain is obviously filling in what it "expects" to be there but I am not quite sure we need the "priming" idea. It seems the brain "sees" what it expects to see in a "frame". What needs to be explained is how effortlessly it does this and how we are alerted to aspects of the real frame that are out of place. We know that this mechanism often fails, as in the phenomenon of "Change Blindness". In fact, it seems that change blindness may be the Achilles Heel for this aspect of Hawkins' theory. On the other hand, answering this question could turn out to provide strong extra evidence for the theory. 

Many applications in the Internet are at least attempting to be "predictive" - filling in details before the query is completed. "Type Ahead" is something we all experience. Hawkins seems to think that this is something "AI" can't do.

My thinking on this subject rides off in several directions.

• How it is that certain "virtual" environments, such as Second Life, feel so "real". Hawkins' ideas about felt reality are applicable.
• How do frames work in non-spacial domains, such as language, physics, and mathematics?
• Is the Universe actually built of frames or does it just look that way?
• Hawkins' book is very light on visual examples that would show the nested and recursive nature of the frames he is talking about. I'd like to provide some examples and perhaps shed some doubt on the idea that a "frame" is manifested in a column of the neocortex. 
• I'd like to present an alternative interpretation of the neurological "facts" in Hawkin's book. Specifically, I think a "frame" corresponds to a network of active neurons in the brain. There is an interesting visual example in Second Life, where a "frame" (a specific location in the world) "rezzes" a bit at a time, providing a way to think of how a frame could resolve itself almost instantly even though it is full of nested frames.

=================

* I use "frame" to refer to Hawkins' "reference frames"

See also

Object Oriented Design

The Languate of Information Science

Computers as Models of Mind - The OOD Meme

Computers model our minds in two ways. We are tempted to think of our own mental processes in terms of what we know about computers. Conversely, we attempt to implement our mental processes using computers.

Over the last 50 years, these two processes have converged, providing us with a more robust analogy for human thinking while constantly improving our ability to implement human thought processes in silicon.

The best model of "mind" is based on memes [1]. These are the "atoms" of the mind. For the purpose of this note, let me focus on two types of meme: category and behaviour.

Computer programmers don't think about electronics or silicon [2]. They think about "objects" which are instances of "classes" (OOD-Object Oriented Design). Objects inherit from their classes the ability to "know" certain things and behave in certain ways (methods), especially the ability to communicate with other objects. A good computer program will implement classes that mirror robust categories. For example, a computer model (collection of classes) that is "about" a university will have a class that implements "student" which is a special case of "person". A course is taught by an instructor (another special case of person) and attended by a collection of students. You can speak of the "instructor of a student", which is a way for the class of "person" to refer to itself, in the way that memes can form "strange loops". The relationship between "person", "student" and "instructor" illustrates the ability of Object Oriented design to use analogy (According to Hofstader, the very engine of the mind), even though these analogies are being formed, for the present, entirely by the human designer.

"Perception" is modelled in computer systems within the "Object Oriented" paradigm. For example, keys on the keyboard all inherit the ability to detect being pressed and all send their information up to the "keyboard" object in the same way, notifying the operating system that "knows" which program is expecting input from the keyboard. Even a "self driving" car is just another object.

One kind of object that is implemented in modern computer languages is the "Lambda Expression", which is a string (sequence of characters) that represents a program. Such expression can be built by the computer program at "run time"  itself and executed. This is one particularly interesting example of self-reference, since lambda expressions can implement whole new classes.

Applications can fail when they fail to map the "real world" into their set of classes. Older programs were a tangle of algorithms (a rats nest of instructions), that implemented narrow business processes, like "pay this invoice". If they failed to model objects in the real world (implementing classes like vendor, bank, bank account ...) they would require constant revision and be vulnerable to small changes in the "outside world". New applications that work in the same "world" should be easy to implement since statements about the "real world" naturally map to the object-oriented model of that world.

Memes are "fuzzy" categories in the human mind. The "classes" of computer science are special cases and generally not "fuzzy". Human beings are expected to supply unambiguous objects as "input". However, in practice, computer programmers can go a long way to removing "fuzziness" and wind up implementing classes that usefully map to memes. The most obvious meme like this is the "class" of itself, along with all the related memes that make up Object Oriented Design.

We can now produce classes such as "all the human faces in this picture" or even "All the human faces that resemble Joe Smith". When the computer fails to properly classify its inputs, this is usually thought of as an error (a face that is not Joe Smith, a collection of pixels that is not a face ...), but it's possible to think of this failure as a sign of "fuzziness" - a step toward implementing something more like the meme of the human mind. We can get a vague sense of how powerful (and creepy) this gets when a surveillance system files away pictures in a "person" file (its model of a person) as a collection of "possible pictures of Joe Smith", or "possible pictures of Joe Smith driving his car on Elm Street at 12:43 PM on Saturday). Thousands of law enforcement officers now spend their days helping their computer systems resolve the "fuzziness" remaining in classes created by their computer systems on CCTV and arial surveillance pictures. One may think of this as extending the perception of the human operator or extending the ability of the computer to "think".

The gap between "meme theory" and object oriented design should not continue to close, but we should not under-estimate the vast extent of the gap. Challenges exist on both sides:

• Theory of mind needs to formalize the meme idea to speak intelligibly and systematically about mapping memes to Object Oriented classes. 
• It is reasonable to expect that an OOD model of the brain would give us an idea of how it works "in principle". I think it would also provide some insight about the possibility in principle of  "strong AI" (creation of a machine mind). Without some completely new structures in computer memory involving dynamic feedback, we seem to be stuck. Even with this hardware, we will be left with neurons being "black boxes", translating input into output with mysterious inner logic. 
• It is impossible to "read out" the state of the brain. In particular, this would involve measuring the gradient of concentration of dozens or hundreds of neurotransmitters at each of trillions of synapses, along with the rate of change of these gradients. This is implies solving trillions of monster dynamic systems of equations, known to be unsolvable in principle, even if all the gradients could be measured precisely, which Quantum Mechanics assures you can't be done. Even so, we can say some useful things about the behavior of such systems. For example, we can get some clues about how to build a "neural net", how big it needs to be and how complex it needs to be to start learning anything useful.
• Computer Science needs to find methods (and applications for them) that require the computer to create and implement new useful classes, along with the ability to learn - to refine the definition of these classes.
• Computer Science has long abandoned the usefulness of thinking in terms of the actual implementation platform (silicon, switches, bits and bytes). A similar revolution needs to happen in mind science, to focus on how the mind works with no undue emphasis on the way it is implemented in the brain.

[1] The "best" model leans heavily on Hofstader's theories (or perhaps we should call them "observations"), plus a whiff of Zen, which insists on a purely subjective view of the Universe. This leads me to think of memes as "implementations" of "real world" classes - effectively an automatic process of Object Oriented design constantly going on in my head. It should be noted that Hofstadter's interest in how the human mind works has become a bit of a backwater in AI research, mainly due to the pressure for quick results by brute force.

[2] I have worked with computers since 1967 and been part of the revolution that took us from "machine language" through "Relational Database Design" to "Object Oriented Design". The current lingua franca of the internet is Java, a thoroughly object-oriented language that would have been inconceivable back in 1967. The multi-layered structure of modern computer applications is closely analogous to he layered structure of the brain--for example the visual cortex. This is no coincidence, since a computer application is, after all, just a collection of memes.