Virtual Worlds

 

"Virtual" used to mean something like "exactly" but now it means something like this.

Essential Meaning of virtual

1

: very close to being something without actually being it

The country is ruled by a virtual dictator. [=by someone who is not officially a dictator but who is like a dictator in every important way]

Her victory is a virtual certainty. [=she almost certainly will win]

The species is nearing virtual extinction. [=it is almost extinct]

2

: existing or occurring on computers or on the Internet

a virtual library

virtual shopping

Creeping into the science of computer systems, "virtual" became a powerful concept as early as the late 1960s. Programs were assigned "virtual memory" that "looked" like real memory but could be moved around or swapped out to disk according to the needs of the operating system. The idea grew from there, underpinning the architecture of "mainframe" computers and when I last worked for a living, I worked on "virtual" PCs - PCs simulated on other PCs. As a result, I have found the whole idea of "virtualization" quite natural for almost 50 years.

The Matrix is populated by "virtual" people - the "real" ones asleep in pods (sorry, spoiler). This idea caught on almost seriously for many who wondered if reality itself is a "virtual world". You are free to play around with this concept. I don't take it seriously.

In both Second Life and Oculus VR, we are presented with virtual worlds. In both cases (and similar technologies), the "world" is created internally as a structure that can be viewed by what can be thought of as a separate "app" altogether. In a PC or Mac, the "work" of presenting a 3D structure on a 2D surface (or 2 surfaces in the case of VR goggles) is greatly assisted by modern graphics cards - perhaps the most powerful but unappreciated component of the modern computer.

If your "world" is to be shared by multiple virtual people (avatars), there needs to be a central store of shared information - a "server" - that tracks and shares all the relevant goings-on in the shared virtual environment. This, in turn, means that relatively fast communication needs to exist between the personal "world" (in headset or computer) and the shared model. This is never perfect and it results in the bane of all virtual worlds: lag. This means that not all users are experiencing exactly the same, consistent world.

I am tempted to provide some diagrams here. Trust me, such diagrams would make things even more confusing. I am working on a picture for you, but this is what the picture should show (skip this if you like):

• The vendor provides one or more services in the cloud, including a "world", like Second Life;
• These services run on a server "farm" - mostly on Amazon Web Services;
• The "world" is broken into many different "rooms", each simulating a shared experience, each running on its own processor thread. In some cases, a single physical CPU may host multiple threads.
• At the client end, a model software communicates with the server "room" and builds a model room out of the data provided by the server
• A viewer provides a user interface that "shows" the model 
• The user's brain experiences the model through the interface as more or less "immersive"..

In multiplayer games, all this can be simplified by making the avatars "multiple choice" rather than custom and making the game space fixed. The result can be very lively action with relatively low bandwidth and a relatively low-power personal computer.

At the client end, the "room" can be quite large but fixed - the same room for all players. For example, a World of Warcraft room may require many gigabytes of client hardware. This requirement rules out stand-alone wireless headsets such as the Oculus Quest - at least for the moment.

Old "virtual" tricks can be used to load world data "on-demand". For example, settings in Second Life allow you only to see things close to you or to omit detail (such as shadows). All this needs to be updated "on the fly", resulting in the ban of all Virtual Reality applications: lag. Even worse, lag may result in multiple players in the same room seeing different things.

All this falls far short of the ideal "virtual" experience, such as the holodeck on the Star Ship Enterprise.

• Visual input can be one or two 2D screens. 3D screens (goggles) can produce the illusion of depth but the architecture of the system is not fundamentally different;
• Auditory input, including sound effects and talk between characters, is a much-valued feature;
• Haptic input, such as a little bump on the paddles, makes a huge difference.

I pick up this thread with consideration of Hawkins' "Thousand Brains" hypothesis and how it provides a useful definition of "felt reality". What does it take to make a "Virtual" world feel real?

.

Evolution of Mind - Evolution of Machine

I'm wary of the term "evolution" - especially when it is used in a vaguely Darwinian sense to sprinkle scientific fairy dust over wild speculation. I think it makes sense to use "evolution" in a fairly narrow sense, in which we can see that some object, "thing" or situation exhibits the following qualities:

• It's in some "state" of current interest. The analysis of the "state" has some non-trivial explanatory power over the thing in question even when it is considered as a current, static object.
• The "thing" has obviously, necessarily undergone change over time - it wasn't always the same as it is now. It must have been in some other state, usually a simpler state, at some time in the past.
• There is strong evidence that the thing has undergone versions, where one version incorporates design features of previous versions.
• It is legitimate to ask what changes have taken place over time to get from the "simpler" state in the past to the current state.
• Some kind of specific process should be proposed, even if somewhat tentatively. For example, Darwin started with the observation that species are related, asked what we mean by "species" and only gradually came to theorize that species must have emerged over time.  He could only speculate about the mechanism (He liked, but did not use, what we call "survival of the fittest". He didn't know about genetics or DNA). 
• The specific timeline of change will inevitably be speculative. It's nice to establish milestones and approximate dates but this must regarded as secondary to analysis of the change itself. Darwin was famously off by orders of magnitude with his time line but this turned out to be irrelevant to his core ideas.

With those provisos, there are a few questions in the "evolution" of humans that are interesting:

• Speech
• Language
• Writing
• Money
• Social Structure (small bands to cities, craft to corporation, tribe to Nation State)
• Religion
• Nation States
• War

It is useful to speak of the evolution of computation or perhaps the wider and broader term "information processing" rather than the "evolution" of the computer. The other aspect of this evolution is characterized by "information storage". Clay tablets and fingers have their place in this history. It doesn't begin with Babbage or Turing. I like to avoid terms such as "information processing", which would be perfectly acceptable except that virtually all readers will conjure up an image of modern computers when this phrase is used. It is, in fact, still fashionable to call the Computer Department at a University the "Information Technology" department, even though they don't teach about clay tablets or even money. The common thread seems to be progress in externalizing or delegating the task of computing and remembering to machines that grow in complexity and power over the years. Questions can be asked about the progress of mechanical information processing and they are woven in to the above questions about human society:

• Writing
• Counting
• Money
• Computation
• Stored programs ("software as data")
• Networks
• Virtualization
• Class libraries (divorce of design from platform)
• Social networks - close integration of human and computer "societies"

An important thread of this evolution is the way that we gradually shift attention to aspects of reality that can be computed and stored while pushing other aspects of reality aside. This is perhaps the fundamental way that human evolution is linked to the evolution of computation. Wittgenstein's famous saying pops into mind: What can't be said must be passed over in silence.

Several common themes emerge when discussing the parallel evolution of humans and machines:

• Information processing and the evolving nature of what counts as "information"
• Compression
• Virtualization (platform independence)
• Information embedded in language
• Communication
• Formalization and standardization 
• Efficiency

We gain special insight into these common themes by comparing and contrasting their history in the two streams of evolution. In the case of computation, many of the "landmarks" are very clear and have firm dates. Others, such as the appearance of money, have a more complex and debatable history but there is clearly a time when money did not exist and a time when it did. 

In some cases, important innovations appear in one stream and they are not obvious in the other, at least at first glance. For example, "virtualization" is a very clear and specific engineering concept in the computational field. What, if anything, can it mean to speak of a virtual "mind"? One clue is in the fact that philosophers have always felt free to describe the inner workings of the mind without reference to how the brain "works". When they do make reference, their ideas are remarkably sketchy and dated. Tellingly, their ideas about how the physical mind works are not much affected by the specifics. The ideas are "fairy dust". This leaves us free to propose that the "hardware" doesn't matter at all, leadinging in the extreme case to ideas such as Kursweil's - "uploading the mind to silicon" seem perfectly reasonable. This is an argument firmly based on ignorance, like the idea that humans were created out of nothing by God, which made sense until the facts were known.

The evolution of "society" in computation also exhibits some fascination and non-obvious parallels. One precondition of "society" is standardization of components and protocols. Such standards in human history have lead to sweeping changes in human society, including capitalism, cultural genocide and waves of economic crisis when policy is about "jobs" rather than human well being. Standardization of weapons, protocols and military "jobs" has played a key role in the evolution of modern warfare to the point where a single instruction can now spell the destruction of civilization through a "fan out" through standard protocols to standardized weapons and human beings turned into standard components of the military "machine". In the case of the military "economy" (legalized mass murder), there are many milestones on the road to standardization, along with a rich set of parallels between the kind of society humans can form compared to the kind of structure a computation can take.

Society can be usefully regarded as a machine than as collections of living things. In this sense, our two streams have been steadily converging over recorded history. The evolution of this aspect of society is worth discussion in its own right, as I have done in "The Programmable Ape". By regarding machines and humans as co-evolving along very similar paths, it becomes possible for me to describe the human machine (the "dragon") in more precise and familiar terms. This can possibly overcome the common perception of what a machine is. 

So what is a "machine"?

• Machine "language" is tiny, formal and inflexible. 
• Machines are formed of components which exist for no other purpose than to serve the purpose of the machine.
• Very few machines are "unique". They are themselves "cogs" in a larger system that demands precise standardization of components. 

In contrast, in a human ...

• Language is vast, informal, open-ended and general
• Humans are famously difficult to organize into coherent groups with coherent purpose. In practice, this involves stripping them of what makes them human in the first place.
• Humans are unique. This uniqueness is built in by the way DNA works and the inherent limitations of their nervous system. 

Over time, these differences become blurred:

• Humans become "standardized" into "jobs" in order to fit into the larger economic machine. Unique human qualities are suppressed or even punished.
• Human "machine languages", especially the language of economics, become more and more rich and complex. To many, it starts to sound like talk about the "real world" and not "machine language".
• Machines are more and more frequently referred to in anthropomorphic terms, such as what "Wall Street fears" or what "Germany wants".
• Individuals are increasingly "programmed" by tremendous pressures of living in a man-made world of things and information. Thinking "outside the box" is difficult if all of language - including your inner voice - speaks only about boxes.

You can get a lot of fun out of taking these analogies seriously at face value. There is usually a kernel of truth - a way of seeing that machines and men are really the same in important ways. For example, if someone is talking about "efficiency", they are talking about a machine. Think of the situation they describe in terms of machines, computation and the history of such things. When they talk about "profit", think about computer games. When they talk about what is "best" for a corporation, take them at their word and imagine the corporation to be alive and part of human history. Ask how such an abstract machine can possibly acquire a desire to survive and thrive.

Virtualization and Understanding Consciousness

Virtualization is a huge topic, worthy of a dozen posts, but let me sketch it out here in the hope of making some details clear later.

ARCHITECTURE

I'm no fan of using computers as models of "mind". Our minds are not computers, nor do I expect that the AI dream of making a mind out of computer chips will happen any time soon. However, it is clear that both computers and minds process "information" in some broad sense of the term and we should not be surprised to find that designers of computers and "mother nature", the "designer" of our minds, have face similar problems and come up with similar solutions.

COMPUTER ARCHITECTURE

If you know a bit about computers, you know a bit about "virtualization" in computer design. The Central Processing Unit (CPU) of a computer manipulates a very small amount of information very quickly. This information is swapped in and out of the CPU from "on board" memory - Random Access Memory or RAM. RAM has always been relatively expensive, so computers have been swapping RAM in and out of memory to some version of the "hard disk" since the 1960's. So there is a kind of "bucket brigade" or hierarchy of storage, that makes huge amounts of information accessible to the CPU in a fairly painless way. A lot of this swapping takes place more or less invisibly or "under the hood" and does not concern the computer programmer.

THE CLOUD

Most recently, with the advent of languages like JAVA and object-oriented design (which cares not where the object "lives") location of the information (data or program) has become irrelevant as chunks are swapped in and out of the "cloud" and between "hosts" with all the "plumbing" functioning automatically. We wind up with our CPU's manipulating information from all over the world, selected automatically from a vast amount of information made available on hosts all over the planet.

That's virtualization.

VIRTUAL MINDS

Something similar goes on in our minds.

What we call "working memory" is very similar to what we call "attention" or "consciousness". It is what it feels like when a portion of our brain activates a very small subset of what the brain "knows". This process is powerful and mysterious - quite unlike what goes on in a computer - but it works in a similar way to address similar problems that we see in computer design. For example, we can "pay attention" to about half a dozen "things" at once. We can switch things in and out of attention a few times per second, producing the continuing experience we call "consciousness" - the flow of "things" through our CPU of working memory. What's more a "thing" can be as simple as pressing the "E" key on a keyboard of the idea of representative democracy. Our minds are perfectly capable of flipping around "things" very quickly as we type away furiously on the keyboard thinking and writing about political philosophy while trying to remember how to spell "philosophy". Almost magically, "ideas" that we have packed away in our brains pop in and out by means of their connections to ideas held in attention a fraction of a second earlier.

LANGUAGE AS THE GREAT VIRTUALIZER

Language is a great virtualizer.  A word like "Canada" can be used to reference a vast amount of information stored away in the brain awaiting unpacking, resorting and repacking. What's more, such a word can be transmitted via speech to another brain, whereupon it proceeds to unpack "meaning" in the second brain. The equivalent of the "cloud" is what we call "culture". Those plugged in to "culture" have access to a vast ocean of information shared among members of that culture. Most of us belong to more than one culture - for example there is an international culture of Scientists and Intellectuals for whom language differences are a mere inconvenience to sharing of ideas.

Without getting too spooky or mystical, I think it's not entirely wrong to see the cultural equivalent of the "cloud" as a group mind - something that is alive and purposeful and not a mere information store.

BEES

Consider the bees ...

An individual bee is an incredible product of evolutionary engineering. Its ability for controlled flight would be the envy of a veteran helicopter pilot. With a microscopic brain, it is capable of navigation and able to communicate the location of promising flower beds to its hive mates.

But it doesn't know it's a bee. 

Everything the bee does is in the service of the hive. It lives and dies to ensure that the hive will survive through the winter. The hive as a whole seems to "know what it's doing", yet the hive has no mind of its own. Hive "behavour" and hive "knowledge" somehow emerge from the collective, individual behavour of bees. I won't go wandering off into abstract speculation to speculate on how this happens. My point is simply that it does happen. The hive mind is distributed among the minds of tens of thousands of bees, each with a lifetime measured in months at most, each with a remarkable ability to sense the world and act in it, but each totally "unaware" of what it is doing.

It is also important to note that the "hive mind" has been shaped (and is being shaped) over millions of years in a wonderful dance with the flowers, who "use" the bees to reproduce sexually and thereby maximize the chance of evolutionary variation. And of course, a rose doesn't know it's a rose.

DO WE KNOW WE ARE HUMAN?

So we are tempted to ask, do we know we are human beings? Is there a level of reality in human history that is forever beyond our ability to understand, just as a bee will never understand the hive it dies for and a rose doesn't know it smells sweet? This is hardly an original question. Religions pretend to answer it by somehow uncovering the secret thoughts of "God", along with the heavenly "big picture". I don't think I'm alone in thinking these efforts result in little more than childish fairy tales, much smaller than the phenomena they attempt to "explain".

NOT SURE WHAT IT IS, BUT HOW DOES IT WORK?

I think we can step back from telling ourselves mystical stories and learn a lot simply by looking at the architecture. There is a hierarchy of information processing that goes all the way from individual "attention" to cultural knowledge. Indeed, since cultural knowledge relies at least in part on experience of the real world, we can regard physical reality as our ultimate "cloud" storage, just as the story of the bees is incomplete without accounting for the role of the flowers. However, like bees, we are restricted by this architecture from experiencing any of it except through a tiny window of "working memory" - less than a dozen "chunks" at a time. Language itself is our "operating system", providing reasonably efficient ways to "download" chunks from our "cloud", process them and store them back in the form of new "mash ups" of old ideas. Human language plays a similar role in our minds as Java plays in the computer you are looking at now. Ideas are popping into your head through words and it makes no difference where these ideas came from. The only requirement is that both the author (me) and the reader (you) are both human - running the basic operating system, capable of turning ideas into words and vice versa.

Of course, this insight applies far beyond the realm of language and "words", but that's a topic for another essay ...