From New Mexico and West Texas where Jaron Lanier grew up, he moved to California in 1981 with the intention of carving out, as he says, “some sort of hippie-like lifestyle in Santa Cruz, playing flute in the mall.” But it never worked out. Instead, at the age of 25, Lanier finds himself running his own company called Visual Programming Languages, and developing products most of us never dream could exist. He initially entered the computer world by programming the sound portion of video games. Eventually, he developed whole video games while working at Atari. His most successful video game was Moon Dust, which made the Omni top ten in 1983. Currently, his work as a programmer centers on the development of a language Lanier believes could revolutionize the computer industry. He resides in Palo Alto, California.
After some searching, I found Jaron Lanier’s house on a small, rather obscure dirt road off El Camino Way, a small circular road branching off from the well-traveled El Camino Real in Palo Alto. I parked my car behind several others in the driveway and wandered up a short path to the small cottage-style house, painted white with blue trim. Overgrown bushes graced the house and gave a comfortable, unkempt air, a lived-in look. I paused on the porch at the front door, wondering what I would find inside, and then I knocked, expecting the unexpected.
Hundreds of old musical instruments, many of them exotic, filled the front room where I waited. Strewn on top of the coffee table were flutes and recorders of various sizes, most made from bamboo. A variety of lute instruments, including mandolins, hung on the walls. Tacked up on the wall across from me was a batik fabric. It was not centered nor especially straight. In one corner stood an upright piano with the box opened so that the strings were exposed. Against the adjoining wall stood a Yamaha synthesizer, and next to it was a Macintosh computer. These were the most modern instruments in the room. Directly across from where I sat was a bookshelf that was filled with books on a wide range of topics from Buddhism to SmallTalk.
Jaron bounded into the room wearing sandals and a short-sleeved royal blue shirt with the collar unbuttoned and the shirttail outside his pants. He was a big, heavy-set young fellow with curly light-brown hair, a beard, and big alert hazel eyes. He smiled widely and greeted me with an excited, animated voice, and then sat down in a chair next to the futon sofa where I sat. After we talked a while, I got the distinct impression Jaron was a free-thinking, unpredictable sort of guy who was always full of ideas and whimsy. When I remarked, gesturing to the instruments, about Jaron’s obvious love of music, he quipped that he was a member of the musical-instrument-of-the-week club.
Throughout the interview, Jaron seemed to instill the computer with a mystery and an offbeat future that most people within the industry had not envisioned. He posed the question, “What if the computer affected your reality and the way you perceive things?”
INTERVIEWER: What are you doing with programming languages now?
LANIER: Well, basically, I’m working on a programming language that’s much easier to use.
INTERVIEWER: Easier because it uses symbols and graphics?
LANIER: It needs text, too. It’s not exclusively graphics. With a regular language, you tell the computer what to do and it does it. On the surface, that sounds perfectly reasonable. But in order to write instructions (programs) for the computer, you have to simulate in your head an enormous, elaborate structure. Anytime there’s a flaw in this great mental simulation, it turns into a bug in the program. It’s hard for people to simulate that enormous structure in their heads. Now, what I am doing is building very visual, concrete models of what goes on inside the computer. In this way, you can see the program while you’re creating it. You can mold it directly and alter it when you want. You will no longer have to simulate the program in your head.
INTERVIEWER: Where did you get the inspiration to create this particular programming language?
LANIER: When I was doing video games, I realized programs could be a lot of things. They could be forms of expression, teaching tools–many things. And I thought that ordinary people should be able to make them, that hackers shouldn’t have the exclusive ability to write programs. People should be able to speak and breathe programs just like they talk now. Making little worlds inside the computer should be as easy as saying hello to your friends in the morning. I really believe we’re going to get to that point, and that it will be a very profound type of communication.
INTERVIEWER: Do you mean people will be able to communicate through programs in the future?
LANIER: Sure. Imagine we’re cave people, and someone comes along and somehow communicates to us that there’s this thing called language that we can speak. And you ask him, “What’s that for?” We’re in a similar situation today. Now we use symbols, called words, that when spoken invoke meanings in our minds. But what is more interesting to me is that you can actually build full models of concepts instead of just giving them names. For example, we can say “solar system” and we can say “planets go around,” and we can describe it. But with a computer you can actually build one, an actual simulation of the concept you’re talking about. I think this capacity to make models, as opposed to just giving concepts names, will be the most worthwhile contribution computers will make for humanity. It will eventually allow people to really communicate ideas they can barely communicate now.
INTERVIEWER: What kind of progress are you making?
LANIER: Oh, great. The biggest problem we’ve had is a hardware problem. One of the major problems with the computer world right now is that when you start working with a particular machine, by the time you finish the project, the machine isn’t around anymore. That’s set us back a bit. The full-blown, professional version of what we’re working on won’t reach the world for a couple of years yet. But when it does, it will really change the way people think about programming. Everyone will program. I’m serious. You’ll like programming because it will be fun.
INTERVIEWER: Are you aiming at the mass audience? Are you trying to replace the programming languages that are in use today?
LANIER: None of those are reaching the mass audience, despite Turbo Pascal selling half a million or whatever. That’s hardly a mass audience. So, yes, I am aiming at the mass audience that programming languages have never reached before. But I’m also aiming at those people already using programming languages. One of the characteristics of my language is that it has a rather chameleon-like form to it. I can’t explain too much about it technically. It can mimic a traditional language, or it can take on the appearance of C, for example, for people who are used to that language. It will be very graceful. People will be able to adapt to it gradually without feeling a sudden change.
INTERVIEWER: So why can’t you explain very much now about the language technically?
LANIER: Because there’s a company involved and, you know, the usual big black wall of trade secrets.
INTERVIEWER: Your company is Visual Programming Languages, VPL. How did you go about starting a company?
LANIER: Oh, well, when you get right down to it, it really just happened. I was doing this work on languages, and various people were very supportive of me. I came to a point where I ran out of money and they said, “Let’s invest in it.” And, “ta-da,” there was the company.
INTERVIEWER: What originally brought you to programming?
LANIER: I once saw someone using a word processor and I thought, “What an amazing thing. If you could make one of those for music, wouldn’t it be great!” I was a composer at the time. That was five or six years ago. I thought it was a great idea, but I never built one. I put it off to build this more general, more powerful tool. With what we are working on, it will be much easier to build a word processor for music, or a hundred other tools.
INTERVIEWER: Did you study programming in school?
LANIER: No. I didn’t think it made sense. I was more interested in the computer conceptually. So I began to track down many of the people who invented computers. They’re all still alive and they’re even accessible. You can just call them up. I learned what they thought about computers in the old days. Nobody really sat down in the beginning and thought about what a computer would be like. Initially, people thought of them in terms of different metaphors, mostly in mathematical terms. The impression I got was that the whole process of developing computers was kind of random. That’s not to belittle the people who invented them, because they did a wonderful, wonderful thing. But it’s just not possible to see the future. Today, we’re using computers to do tasks they were never originally designed to do. What I mean is, the idea of what a programming language is doesn’t really relate very well to using them to process words. Programming languages were invented by people who thought mathematically, and word processing was invented by those who thought about business and offices; two separate worlds. Different people look at the same idea in different ways.
INTERVIEWER: Do you mean the original idea went through changes in the process of becoming reality?
LANIER: See, what I’m doing goes way back to the fifties. I’m going back and taking a fork in the road that everyone passed by. Everyone in programming today is talking about different ways of telling the computer what to do. My programming language doesn’t do that. With mine, you actually look at what the program is doing and you mess with it until it’s right. It’s really a different process.
For example, you have a recipe, which a person follows to make a cake or something. That’s what current programming is like. On the other hand, there’s tuning your car’s engine. You watch the thing running and see what it does and change it until it works the way you want it to. My programming language is more like the latter.
INTERVIEWER: Are you saying there’s not much creativity in the way people program today?
LANIER: No, no. The way people program is great. There are a lot of great programmers around. I’m saying that the languages they use are awkward. They’re being impaired. And in particular, a lot of people who should be making interesting programs aren’t, because the languages are too difficult for them to even consider it. With my programming language, people who are trying to convey ideas in any field–history, philosophy, politics, psychology, and certainly in the sciences and mathematics–will be able to create their own programs.
INTERVIEWER: Which programmers do you especially admire?
LANIER: Oh, gosh, there are a lot of brilliant people. Just this morning I was talking to Dan Ingels, who is one of the original SmallTalk crew. He’s a very inspiring person. And some of the generation who invented computers are just really wonderful, like Doug Ingelbart. He came up with the mouse and windows, among other things; the whole world that Xerox and the Macintosh were based on. He’s still around. He lives in Menlo Park. There’s Marvin Minsky. He kind of invented artificial intelligence. He’s an inspiring guy.
INTERVIEWER: What do you think of artificial intelligence?
LANIER: I think the term is an odd one. Calling it artificial intelligence is pretty strange. People are associating consciousness with behavior, that is, the displayed ability to accomplish certain tasks. I don’t see them being connected at all. It seems clear that you can get computers to behave in any way you can program them to behave. Some programs might be so complicated that you want to call them intelligent, but it’s kind of a meaningless term to me. What they are doing at MIT is really quite interesting: teaching a program to recognize a certain picture. But all the expert systems are bogus. There’s no content whatsoever in a lot of the commercial products that are called AI. Artificial intelligence means different things to different people.
INTERVIEWER: Do you see your language being used commercially?
LANIER: Oh, sure. It will have a profound effect commercially. It will make developing software much faster, and therefore cheaper. It will also totally change the way people choose software and the way they design software. Right now, because the technology is so new, you can sell software based on the sole attribute that it works. In the future, software standards will be much higher. People will say, “Of course it works. I can make it work.” And they will ask, “Does it make me feel good? Does it help me think in the way I want to? Does it, in some way, match the way I write?” I think that computer programming will soon be judged a little more on its quality and aesthetic content, and a little less on just the basis of whether it works or not.
INTERVIEWER: Will your visual programming language look beautiful? What exactly will it look like?
LANIER: What is aesthetics in programming all about? Well, it depends on the program. One very important kind of program is an editor, which helps you make text. A graphics program like MacPaint helps you make pictures. And there are programs to help you make music. Now, if you observe these programs at work, you will notice that they all have assumptions built in about what you are going to do with them. With word processors it isn’t so bad because, with text, you are basically just putting words in the right order. But with pictures or music you really start seeing the limitations. MacPaint, for example, won’t let you turn images; the ability just isn’t there. In addition, it’s not so much that the specific ability isn’t there, but that the ways the program presents ideas doesn’t match the way you think. Some people may not think in terms of the specific lines or line quality and other terms of form and tone as they are presented. Instead of there being just one right word processor or graphics program, people will soon be looking for a program that suits the way they work. Does this program match the way I think? Does it flow with me when I want to do something with it? That’s one kind of aesthetics.
Another kind of aesthetics can be found in the way a program expresses something to you. For instance, take the solar-system simulation I mentioned earlier. With educational software like that, the issues are different. How well does the program explain the subject? How eloquent is it? Does it let you interact with it in such a way that you really get insights that are different than you would get from just watching a movie of the same thing? There will be more kinds of aesthetics in the future. It so hard to predict. When movies started, people didn’t have any idea what they would be like. Computers are just starting now and we don’t have the slightest idea about their future.
INTERVIEWER: As you develop a new programming language, do you plan the whole thing out beforehand or figure it out as you go along?
LANIER: I know a little bit about it before I really start. I’ve been doing partial language versions on smaller machines and working my way up to the complete version.
INTERVIEWER: You have quite a collection of musical instruments. Which is your favorite?
LANIER: I don’t know. It changes every week. I belong to the instrument of-the-week club. They mail me this funny instrument from some different part of the world every week. Actually, musical instruments have a lot to do with computers. They’re one of the best examples of user interfaces in the world. They’re very inspiring to study.
INTERVIEWER: How can the computer be used in music?
LANIER: As an example, I have a program that makes an editor for canons. A canon is like a round, where people sing or play the same melody, but the melodies start at different times and then mingle together. With this pro- gram, you enter one note at one place and the program automatically enters all the voices. You can immediately hear how the canon comes together. Usually, canons are very hard to write, but with this program, they can be much easier to write.
INTERVIEWER: Do you think more and more people are going to be composing music with the help of computers?
LANIER: Maybe. I hope so. Music is already pretty easy to make when you come right down to it. It’s really a question of motivation. For the serious composer, the computer is just great. Now, the composers have to copy all the different parts of the music for all the different musicians, so the computer’s wonderful for them. With popular music now, you basically just listen to it, but I think more and more music will become interactive. You will actually interact with the music, with other people, and with dance. I think we’ll see a lot of that soon.
INTERVIEWER: Do you ever apply your background in composing music to developing your programming language?
LANIER: A lot of people who are into computers and math are also into music. Music is similar to programming languages in that it has a fairly elaborate kind of notation, music notation. But even more, it’s the musical instruments themselves that are a lot like what I am trying to do. Because, with my language, you interact with a program while it’s running instead of specifying in advance what it should do and then hoping it will run right. It ‘s more like playing a musical instrument rather than looking at a piece of music.
INTERVIEWER: Do you consider programming an art, or a science, or a skill, or a trade, or . . .?
LANIER: Well, computers don’t have any quality in themselves. They’re absolutely empty things, tabula rasa. Since they are such empty minds, it depends entirely on the person involved, more so than in any other field of human endeavor. That’s why I’m designing my language to be able to take on so many different forms; it will have to do that to meet the needs of different people. I treat programming more as an art than anything else. I was talking with Peter Deutsch on a television program last weekend. He said programming was a craft. Then there are some people who think of it as mathematics. It just depends on the person.
INTERVIEWER: Do you ever get tired of working with computers and programs all the time?
LANIER: Oh, sure, especially the way they are today. Computers are very frustrating machines to work with, and programming can drive you crazy. Yes, definitely.
INTERVIEWER: Do you work regular hours?
LANIER: You know the answer to that one; they’re very irregular. Especially these days with the company. I tend to work in the middle of the night just so I can get things done.
INTERVIEWER: Are you working on other things besides your language?
LANIER: Well, it’s all just the language. There are specific parts of the language that have applications in different areas. For instance, there’s one subset of what we’re doing that applies to medicine; we’re doing some projects for the local hospital.
Developing my language is frustrating in a way. I’d like to spend twenty-four hours a day working with the language instead of having to share my time with running a company. But the programming work itself is great. It’s really fun. Any time there’s an improvement, it’s very tangible; it’s right there on the screen. That kind of progress is very gratifying.
INTERVIEWER: When you were involved with games, what kind of games did you do?
LANIER: I did music for various people’s games. There are some Electronic Arts programs out with my music. My most successful game is called Moon Dust. It made the Omni top ten in 1983 and supported me for a year. It’s a very abstract and experimental game, which made its success all the more delightful. It has music and little spaceships that fly around leaving fading trails that sparkle. You fly the spaceships and influence the music, trying to play it but not hit any wrong notes. There was a scoring system that nobody paid any attention to. I think people just liked playing with the pretty music and the pretty visuals.
INTERVIEWER: What do you think about the future of computers?
LANIER: Computers right now are ridiculous. They hardly do anything for people. In the near future, it’s going to be a boring struggle for the marketplace. Everybody will be concentrating on these big companies that still have not computerized. But there are software and hardware developments that, in the next few years, will totally turn the computer world around. They’ll catch everyone by surprise. So it will become very interesting.
The whole structure of the computer world is based on the assumption that computers and programs are difficult to make. And that’s the reason program vendors like Lotus can become so monolithic; finally, there’s a decent program and there’s an avalanche of popularity. In the future, there will be a multitude of very good programs and anybody will be able to make them.
INTERVIEWER: Are you saying that an individual will write his own programs in the future?
LANIER: Well, of course most people won’t, but a lot of people will. It will be just like books today. Everyone’s basically literate, and the difference between a person who writes a book and the person who doesn’t is a question of drive, enthusiasm, and business sense, not a question of being able to write or not. Creating programs will be the same way.
Also, if somebody wants to come out with a new computer that isn’t MS- DOS standard, they won’t be scared to do so because it will be so much easier to have a software base for it. Today, people are building whole computers just to run certain software. In the future, it will be totally the opposite. Compatibility won’t be as important when it’s easy to make software. It will be like transcribing a piece of music from clarinet to violin: You have to change little things here and there, but it won’t be that big a deal.
INTERVIEWER: Do you see the computer becoming more of a creative tool and less business oriented?
LANIER: Actually, the business world is, like computers, entirely created by people. God didn’t come down and say there will be a corporation and it will have a board of directors. We made that up. Business is a very ritualistic thing. It changes very slowly, so it’s hard to say what will happen in business in the future. It won’t necessarily be rational. You know that legions of business people still use Cobol programs on big machines, even though it doesn’t make any sense at all? What can one say about that?
In general, I think computers will be used for creative purposes more, and for business more. Right now, computers don’t do that much for people. Word processing is somewhat better than typing, and databases occasionally work pretty well, especially for large companies. The problem is that software is so hard to write. Instead of a constantly evolving body of software getting better and better, we have a situation in which evolution just freezes when a certain software reaches a certain adequate level. Everyone’s so glad just to have something that works.
INTERVIEWER: Do you think that eventually programming languages will replace spoken and written languages like English?
LANIER: Not at all. English will never be replaced by a programming language, because there’s so much built up around it. There’s Shakespeare, there’s all our expressions. . . .
INTERVIEWER: Well, what about hieroglyphics? They disappeared.
LANIER: That’s because all the people who used them were killed. But even if they were still around, they probably would be using a descendant of hieroglyphics. I think computers will provide a new form of expression, and people will recognize that both English and computers are good for expressing different things. In some areas today, English is being stretched. When you talk about ideas, in philosophy, economics, politics, people almost never understand what they are saying to each other. With computers, you can actually build models of whole interactive systems of ideas, or concepts, or even styles of thinking. These will be better expressed by models built on a computer. English is very good at describing, and computers are good at modeling. In the future, the two will become mixed and each will be just a part of the way we communicate with each other. Together they will improve the way we communicate. And whenever people communicate, they stand a better chance at having empathy for one another.
INTERVIEWER: Do you think the concept of the Dynabook will ever become a reality?
LANIER: Only as a passing phase. The Dynabook is one thing that will move through pretty quickly, probably within the space of a decade.
INTERVIEWER: Move through into what?
LANIER: Well, I’ll tell you, but you might not believe me. Let’s see, right now your interaction with a computer is confined to a screen. What if the computer affected the way you perceive things? Not changing the real physical world, but creating essentially 3-D objects around you. They aren’t really there, but people can see them and share the experience. People’s daily lives will include images that are computer generated. That’s the technology we are working on. And, unfortunately, I can’t really describe it in any more detail now. I know it sounds totally loony, but it’s really going to happen.
INTERVIEWER: The computer will generate objects that aren’t really there? Perceptual models?
LANIER: I know that sounds very confusing. Let’s just say they are well-controlled hallucinations. Actually, it’s nothing that weird. It will all be very straightforward. One thing I can say is, you’ll probably wear special glasses that will help create the images. And people will be able to share these images and talk by radio.
INTERVIEWER: So, instead of having a printer next to our computers, we’ll have some kind of image generator?
LANIER: Yes, right. There was a cartoon in the Sunday paper last week that showed a hacker turning into different forms. Did you see that? It was amazingly on track. I think that, ultimately, computers will generate additional realities for us. And they will do this in ways that won’t detract from the physical world, but will in fact help us appreciate the real world.
INTERVIEWER: Do you see genetics merging with computers?
LANIER: Maybe. There may be optical computers, there may be chemical computers. There are obviously biological ones because parts, if not all, of our brains are computers. It’s a hardware question. How do you get the actual technology of the computer to work? But there’s an entirely different question that is the important one to me: What culture do you make up so you can actually use this technology? The technological question is basically engineering, you know. If we want to make a Dynabook, someone will figure out how to make one. If you want to make a very, very dense memory that’s based on enzymes or some ridiculous thing, who knows, somebody can figure out how to do it. But when it comes to culture, you really have to invent it–invent a whole new world out of nothing. When you have a Dynabook, what do you do with it? How does it operate with the rest of the world? And how does it fit in with regular books? With lunch boxes? With video games?
INTERVIEWER: Don’t cultures just evolve?
LANIER: No, they’re invented. They’re made up, either by people who are conscious of making them up or by people who aren’t conscious that they’re making them up. The twentieth century is full of examples because we’ve made up so much stuff. TV didn’t exist before, and now a lot of people in America spend more time watching TV than anything else except sleeping.
INTERVIEWER: But when TV was invented, they didn’t know they were creating a culture.
LANIER: No. The people who invented the actual TV tube and stuff didn’t do it. The people who created the culture were the people in Hollywood who made up the programs, and the people who figured out how to sell TV. There’s a whole bunch of people who did it, not just one person. Photography was made up from scratch by people who had to figure out what the hell a photograph was and what it meant. The same with movies. What’s happened to computers so far has been made up by people. I think it’s good to be conscious of the process. I’m glad that, from the beginning of computers, people have been thinking about the political and ethical implications. The computer world itself has really benefited from that. I think so far it’s been a somewhat more self-conscious endeavor than TV or photography.
INTERVIEWER: What about the power of information? Is that the most important aspect of computers? So many people say that’s what makes the computer so important to our society.
LANIER: Well, all the computer can really do is manipulate information. And information is a rather broad term. It basically covers all human experience. But when people talk about the power of information, I think what they mean is something more specific, that our society is organizing itself more and more around things that don’t physically exist. Information, concepts of living, computer memories, the true existence of a corporation, one’s wealth, or power, or status, or one’s job–all these can be defined by information held in a computer. We are in a transition period. Until now, what we have wanted from life had to be got by manipulating physical matter. Now, we are just starting to organize our lives according to information. Eventually, our very experiences will be generated by information instead of the other way around. That will be the true information age.
INTERVIEWER: Won’t the physical world’s importance be diminished?
LANIER: No, not one bit. Not any more than having computer music threatens acoustic instruments, or having photography threatens the existence of painting. I believe it will help us get a much more objective view, a more appreciative view, of the physical world. And of nature. I think it will spawn a stronger ecology movement, for instance. Just on a practical level, people will be able to have experiences without having to change the physical world, without screwing it up. This is a very large topic. I’m writing a book on it, by the way. It’s called New Natures, about what it will be like to have arbitrary worlds.
INTERVIEWER: Do you think some young people who are interested in computers today are thinking along the same lines?
LANIER: Yes, I think so. It’s very hard to generalize, in spite of the stereotypes. Like the one that programmers all dress terribly, stay up all night, and all that. There’s a current generation that is pretty much oriented toward the Macintosh type of thing. I think we are at a very fundamental level right now. People are still trying to come up with a decent piece of software for somebody to use. In the next few years, life is really going to change. It’s very exciting to think that it’s the young kids, the generation being born right now, who are going to grow up with this new technology. They are going to be the ones to really benefit from what we’ve been talking about.