Equal Time

Dear CyberStage,

I read your piece about post-symbolic communication ("Post-Symbolic Mind Control", Issue 2.1, Fall 1995), and I'm absolutely astounded that you could have it, and me, so wrong.

You are in a position to do damage to ideas and people by publishing pieces such as this. Read any of my essays, listen to any my lectures, and you'll discover that I am utterly committed to human autonomy and dignity. I have ceaselessly attacked the likes of Delgado, and sometimes it has been a lonely job. There are, sadly, far too few people in the computer science field who are fully, unambiguously committed to humanism.

If you want to understand my ideas, you can read a variety of essays on my web pages (http://www.well.com/user/jaron/). You might start with Agents of Alienation, a controversial and very widely read essay which is a critique of the idea that computers can ever represent people. Here's a quote:

In another context, discussing Virtual Reality, I came up with the slogan "Information is Alienated Experience". This phrase came to me partially in response to the imperialist tendency of theorists of politics, art, and computer design to pretend that ideas or words can represent people. The discipline of science is to only respect falsifiable theories. When you create a boundary for yourself or others by believing in a theory of what you or they are, you create a conundrum of scientific method in which you can never know what might have been, and therefore have no opportunity to test the theory. Part of the beauty of the American idea of government is in its self- limiting charter. For instance, the phrase "the pursuit of happiness" in an instant identifies an indefinable territory beyond the reach of law, or even language, that constitutes a critical part of "freedom". Let's limit the charter of computers and the infobahn in the same way.

That sounds pretty different from your assessment of my goals. Here's a quote from your article:

Lanier hopes to translate the human experience into digital binary code much the way that the human genome project hopes to translate the human genome into digital binary code.

The idea of post-symbolic communication is essentially the opposite of mind control.

You have done bad work here.

Sincerely,
Jaron Lanier

So began a dialogue about post-symbolic communication and virtual reality with the man who is widely credited as founding the VR industry. Jaron Lanier -- best known in pop-culture as the man who coined the term, "virtual reality" -- created the very first hardware and software with VPL Inc., the company he founded in San Fransisco, California. He is responsible for taking first-person computer simulation outside the walls of NASA and bringing it into the hands of regular people and artists. After being fired by his own board of directors in 1993, Lanier has since relocated to New York City where he now teaches, talks about the direction of virtual reality, and plays music.

Post-symbolic communication is one of Lanier's basic tenets, that eventually VR systems may become advanced to the point where entire new forms of communication can evolve out of virtual environments, whereby one can create meaning instead of simply referring to it, which is what we do with literal language today.

The article that created the heat, Post-Symbolic Mind Control, was written by F. Scott Taylor and published in our Fall 1995 issue. It argued, among other things, that feedback loops embedded in post- symbolic communication VR systems enabled the possibility that if fallen into the wrong hands these systems could lead to a new age of mind-control experiments similar to those performed by Jose Delgado in the 1970s, a man who would later argue for the technological control of "socially-undesirable persons."

Lanier said we had it all wrong, that the very principals of post- symbolic communication which was the basis of Taylor's points "completely inverted" what he meant by it.

And so, in the true-Canadian style of equal time, we invited Lanier tell us where post-symbolic communication came from, how he envisions it being used, and by whom. While we were at it, we also found out what he was up to these days, and queried his feelings on the "New VR" currently being marketed by video game giants and Hollywood.

The starting point, really, in my mind [comes from] the need to think about what kind of mythic relationship people can have with technology, and when I talk about mythic I mean the good kind of myth -- the Joseph Campbell kind of myth -- as opposed to meaning a lie. I think there's been one mythic relationship which has been in effect for some hundreds of years, which is that technology is what makes us more powerful, especially relative to nature. I've come to believe that that kind of myth has overstayed its welcome. We're [on some] sort of on auto-pilot [now] where we keep trying to become more and more powerful with technology.

There's a little book called Finite and Infinite Games. It suggests that there are two ways that people can do things. A finite game is like a game of baseball in that it has to come to an end, but the overall field of baseball is infinite, it keeps on cycling forever. So one of my starting points is how we can find a relationship to technology that's of an infinite game variety instead of a finite game. I think the goal to attain more powerful is a finite one for two reasons: one, if we just keep on trying to be more powerful we'll blow ourselves up with one technology or another; two, there's this problem of philosophy, where when we start to think of ourselves as technology -- when we think of everything about life and its meaning as being as just technology -- [and] we end up in this absurd universe where nothing is happening and nothing has any meaning. I think that that's the origin of this sort of bland and nerdy quality that the aesthetics around technology tend to have. The idea of post-symbolic communication is one attempt to try to find such an infinite game. I don't think it's the only one, nor do I think it's the best one, but I think it's one I've been able to articulate. I'm convinced that our survival -- and certainly our happiness -- depends on infinite games, whether [it's] this particular one or other ones.

[Post-symbolic communication] is something that wouldn't be able to happen for at least another hundred years -- and possibly even longer than that. The notion is that in that time there would be a generation of children who would have a chance to grow up with interesting resources, one of which would be cheap and plentiful virtual reality equipment, so that they would be able be experience and create simulations and share them at a distance. I'm pretty confident of that happening, I think that's sort of an inevitable occurrence.

We're seeing the start of that happen already.

Just barely. I would imagine something considerably better. The next resource they would have would be new programming languages or authoring tools, or whatever you want to call them that allows them to make up the content of virtual worlds very quickly and easily. Let's suppose those two things happen. This generation of children could grow up with a truly extraordinary experience. They would have the ability to make up the content of shared worlds as a form of communication. What post-symbolic communication means is that, if you grow up with this capability and you're fluent in it and you can make up the stuff inside virtual reality as quickly as you might compose a sentence, then you have the possibility for a kind of communication where you directly make up the content of a shared world instead of using symbols to refer to it. It's like cutting out the middle-man.

Why would this be important? One reason is that it would fit into the psychology of childhood, and therefore into the psychology of life in general, in an interesting way. I think that children find themselves trapped between two different worlds: the internal world of the imagination, where they're infinitely powerful and, in most cases, have a natural tendency to be extremely creative; the other is the shared, practical world, where you find food, parents and survival, and that shared practical world is the only one where you find yourself not completely alone. Ultimately it tends to be inflexible in comparison, it's not one that rewards your creativity as readily.

So what you're talking about is a tool for children to help them bridge those kinds of worlds.

Yeah, exactly. It would represent a whole new stream of communication, it wouldn't replace language, certainly, but it would be something that happens along side of it.

What is the earliest age of a child you would put into VR using post-symbolic communication?

Six years.

Do you think it would be dangerous to start a child in VR who is too young to know what physical reality is yet?

Not dangerous, but still I think it's better to develop a facility with physical reality first -- that's the only way to make VR exciting, by way of contrast anyway.

How do you think this would contribute to shaping children's perceptions of the world?

Well, if I could answer that question the whole idea would be worthless because it would be too predictable. What I think is very interesting about that question is that right now our primary way of communicating ideas is with words, and words are not perfect. Words are very crude instruments and I think having this alternate type of communication -- where you directly make up stuff instead of referring to it -- could help us break out of the prison of words, it could help us find forms of expression that are a little more continuous and little more representative of just how fine- grained the world really is, and that to me would be very wonderful, very exciting.

When I first heard the term "post-symbolic communication", even though I hadn't actually done any research into its meaning at the time, the phrase seemed to imply to me a kind of communication which bypassed the intermediary of language.

Absolutely --

-- Almost a kind of communication where it was just my mind to yours, meaning communicating meaning. But now that you're talking about it my understanding has changed slightly, that we're still developing tools but it's just different tools than a literal language.

Right. It's not the same thing as psychic communication or something, it still involved manipulating the world in order to communicate like language, but this done through creating the stuff of experience instead of using symbols. The meaning of psychic communication, if it exists or if it could exist, is yet another topic (laughs).

Let's talk about your music for a little bit, because you are obviously an avid musician. Have you ever had anything recorded?

There's a record on Polygram, it's called Instruments of Change.

What music or groups are you into these days?

Too many to really say, but some of them are Paul Hillier and his Theater of Voices, Jaipongan and related styles of music from Java, The St. Paul Chamber Orchestra, and Joni Mitchell's last record.

What is the connection between your music and what you talk about in post-symbolic communication?

Music is our most prominent non-symbolic form of communication, and there are some theorists who try to frame music in symbolic terms and say that this major chord means this or that, but I think even the people who write that stuff ultimately realise they're winking. I mean nobody really believes that stuff. So in that sense it really is an exploration of the edge of meaning of that part of ourselves that we know but we can't explain. There's also the physicality of it, that the way I imagine post-symbolic communication -- as well as the fun use of virtual reality -- is that it's very very physical. Playing music is a very physical artform, you do it with your body at least as much as your mind, and probably more so, and that's important to me.

Another connection has to do with this sense of mysteriousness, which is part of the complicated part, I guess. One of things that I think is a problem with computers and science and technology in general is that we think of them as explaining things, or somehow reducing the amount of mystery in the world. I think what makes people very good musicians or good scientists is being comfortable with that sense of mystery, and I think that one of the problems with computer culture has been rejection of mystery, an assumption that people and computers must be the same. There's this assumption that you can explain things and that lead to a very simplified world in which you limit yourself in order to pretend that you've explained yourself.

Is this where you see an artist has a real role to play in the development of technology?

Yeah, absolutely, in fact I think artists are probably going to be more important in the next fifty or hundred years than they ever have been in the history of culture. See, this whole business with computers is very much a double-edged sword. On the one hand they are immeasurably important because more of our culture will happen through them, so the way somebody programs a network will be more important than whatever laws we write about what should happen. The future of privacy will be determined by a programmer much more than by a legislator. Anything that effects what actually ends up getting solidified into our operating systems of the future is immensely important. What can effect those things? Artists. Essentially, if you see computers as only things that solve problems, you'll end up with a very narrow model, and when people run their societies through computers based on these narrow models something immeasurable will have been lost, but we will never know what. The only people who can push computers [past narrow limits] are artists. For instance, if it wasn't for virtual reality, people may think that it's okay for computers in the future to be based on a discreet call-and-response model, where the computer sits still while a person does something, and vice-versa.

There's an irony in this for artists, though. Of all the artforms to work in, I think, high technology art is the one in which the art pieces don't last. It's a lot like sand-painting in the wind, because of the way the platforms change. If you make an art piece in virtual reality, for instance, the chance of anybody being able to see the same art piece in five years -- or certainly in ten years -- is almost non-existent. They're very, very temporary. The legacy of these art pieces is not so much in the pieces themselves but in the way they influence this emerging computer culture which is partly held in the programs and partly held in the myths that people have in their heads about how they should be used.

Let's talk about the direction of VR a little bit. It's interesting to see where VR has gone now that corporate and commercial interests are beginning to play a role in things like video games. How do you feel about what VR is becoming?

Well, obviously I think any sensitive person would be displeased with a lot of things that have gone on. I'm particularly unhappy with many of the video game virtual reality products. To me, there are two different things that go hand in hand. One is the sort of crushingly stupid set of cliches of shooting aliens and of being pursued paranoid. The origin of that aesthetic is economic. In coin-operated machines, there were really only two ways to keep quarters coming in: one was to have a timer, and the other was to have some sort of paranoid scenario in which the player would be killed. People preferred the paranoid scenario because it gave them some chance at being powerful, whereas having a timer shut them off made them feel as if they were subservient to a time-clock. I would hope that we're going to outgrow that very soon because I think that the coin-operated mode of experiencing interactivity is a very temporary phase.

But then there's another side to it too, which has to do with imagination, [and this is also the source of my optimism]. When I go into virtual reality, the first thing I think about is, "What can I do that I can't do in the physical world that's really dramatic?" Among the things that really excite is transforming my body so that I would become a giant octopus, for instance, or having a world that's a four- dimensional world instead of a three-dimensional world.

Without exception, in all of these games that I've seen, the only interactivity that happens is navigation and target-practice. Now, if you imagine scenarios that have more interactivity, I think it's very hard to reconcile a more open world with as limited a scenario as "kill or be killed." But in order to get away from target practice and navigation, we have to force ourselves to be imaginative. I have been extremely disappointed in the last few SIGGRAPH shows where virtual reality worlds are shown. Essentially, all the interactivity in these worlds have been limited to navigation and target practice. I had proposed last year that there be an outright ban on it.

How did they react?

Well, I was saying it facetiously. Actually, though, I can do it more seriously here. So, I propose a ban on navigation and target practice (laughs). If we simply don't do those things for a few years, it will force us to do other things. I mean, there's nothing wrong with navigation, certainly, it's one thing you would do with the space, but it's hardly the only thing and I think we've done enough of it now.

Do you think that we're a bit wiser from having become aware of what television has done to our culture in coming into these new technologies?

I'm not sure that we're wiser, but I think the setup is different in VR. There's such a wide disparity between the expense of an individual television set and a television studio that there was a very weird separation between audience and creators from the start, it's only now beginning to change with computer tools, but not enough. With virtual reality, any system at all is powerful enough to be a development system, so there is from the very start a difference. Products from companies like Nintendo and Sega are artificially closed, kids can't make their own games, and to me that's the single worst thing about them. That is exactly the thing that has to change. I don't know when that point is, it could be in five or ten years, but when that economic transition happens, I think we'll see a very positive corresponding change of using these machines. Closed machines will always have unhealthy software, I think it just goes with the territory.

So, now that we're at this point where Hollywood has grasped the VR concept, and now there entire movies centred around virtual reality situations, is there any sense for you that "your" baby has grown up and now it has become its own creature, or are you still have a very intimate connection every time you see something to do with VR?

(Pause) I'm not sure how to answer that. I still cry a little inside when I see these stupid violent Hollywood VR movies, but I think I cry a little inside when I see any of them, whether or not they're VR-themed. I don't think of it that way, I see myself as trying to influence the metaphors and trying to help out in any small way I can by talking about it. I've sort of decided that Hollywood is hopeless. Hollywood is yet another example of a closed-system, just like Sega and Nintendo, and closed-systems are only rarely capable of making good-quality culture, and when it happens it's essentially by mistake. I think good culture comes from open systems, and that's what we really have to be committed to.

Tell me about what your activity has been post-VPL. Have you started any new companies, etc.?

I haven't been doing the mogul thing too much. I've done a little bit of stuff with software applying virtual reality to surgery, but I think my mogul days are pretty much over. I like being an unemployed musician much more. I talk a lot, I speak quite a bit. I certainly am performing and writing music, and I'm involved with the computer science department at Columbia University, and also a very wonderful art and technology program at New York University called "ITP". I've made some films, and I work with choreographers a lot, and I've been working on some experimental architecture, and some writing.

Have you published any books yet?

Not yet, they're coming though. I've been doing quite a mix of things, but I don't think I'm going to do the industrialist number any more, it didn't suit me well. It was an odd fit.

You seem like a happier artist now.

Yeah, absolutely.