Thomas P. Hughes, "Creating Open Systems: From Thomas Edison to ARPANET/INTERNET” - MIT SDM Distinguished Lecture
[MUSIC PLAYING]
MAGNANTI: Welcome, everyone. Delighted to have you with us, those here at MIT and those at a distance. My name is Tom Magnanti. I'm one of the co-directors of the System Design and Management program.
And it's my pleasure to welcome you to our first distinguished lecture series in complex systems. We are very fortunate to have with us today and over this semester Professor Thomas Hughes, distinguished professor, and I think it's probably appropriate to say, dean of the history of technology in the United States and maybe in the world.
Professor Hughes is well known for his insightful look at technology, particularly look at large scale projects and of developing a framework for looking at these projects and for providing insight for all of us at technical institutions such as MIT and elsewhere. We're just delighted to have Professor Hughes with us on this occasion.
He's going to give two public addresses. The first address will be today, and the second address will be on October 30. And he will also be meeting with the System Design and Management students in a more intimate setting.
I'd just like to, for those who are not familiar with the System Design and Management program, it's an educational program at MIT, whose intent is to educate future technical leaders in architecting, engineering, and designing complex systems and complex products. The intellectual content of this program revolves around complexity, complex systems, and complex products.
There clearly are many ways to look at complexity. There's various modeling approaches to looking at it. There's engineering ways of looking at it. But certainly among those viable ways of looking at technology is a historical perspective of trying to understand lessons from history, what we've learned from the evolution of our understanding of complex systems, our understanding of technology. And Professor Hughes, I think, brings a rather unique perspective on that in terms of his understanding of particular large scale projects.
He has written many books and many articles. His most recent book is Rescuing Prometheus. It's a very notable book.
AUDIENCE: I hate to interrupt, but the audio is very poor.
MAGNANTI: We hope we'll improve it. I'd like for the control room to see if we can improve it.
His most recent book, Rescuing Prometheus, has gotten a great deal of attention. It was, I think, a month, a month and a half ago was reviewed in the New York Times. Two weeks ago, he was on National Public Radio to talk about this book and got quite a rousing responses, I understand from my conversation with him a few minutes ago.
He is also very much intertwined with the system--
AUDIENCE: We can barely hear.
MAGNANTI: Let me try another microphone then.
HUGHES: You want this?
MAGNANTI: I hope this is a little bit better. He's intimately involved, has been intimately involved with the System Design and Management program. I think about six years ago or so now, we had a committee at MIT that was convened by our previous dean of engineering provost Joel Moses and Dan Roos, our associate dean for engineering systems, that was looking at large scale systems. And we brought together, at that time, engineers, management people from MIT, anthropologists, historians to try to get our arms around this whole issue of large scale systems.
And I think that was a previous occasion in which Professor Hughes was visiting MIT. And he joined us in those deliberations. So I think it's fair to say that he's had a touch, from the beginning, on the overall direction and orientation of this program System Design and Management.
I think those of us involved with the program couldn't think of anyone better to inaugurate this distinguished lecture series on complex systems than Professor Hughes, who will tell us a little bit about, I think, some of the elements of his book and some of the projects that he's been involved with and provide us with his own unique perspective on these issues. Professor Hughes.
HUGHES: Thank you, Tom. This all right? Coming through all right? Very generous introduction.
I'm very happy to be here talking to the people about whom I write, that is engineers. Historians do not have this opportunity very often. I'm very fortunate to have it this afternoon and on other occasions.
Tom mentioned this National Public Radio. He said I got a good response. I'm not sure it was a good response. I got an interesting response. I'll tell you about it very briefly before I go into my prepared remarks.
The program is called the Public Issue, Public Concerns, Public Issue. It comes on every day in the week. And after the guest described is interviewed about a book or some other activity, then the audience out there in radio land has an opportunity to call in and ask questions.
And I was getting a rather expected, what I expected by way of questions, until somebody from Pennsylvania named Mike, I believe, called in and he said, professor, I don't want you to think that I'm a crock. But he said, when I was young, aliens abducted me. This is on National Public Radio. Aliens abducted me. I spent time with them.
And I know in your book, you write about intercontinental ballistic missiles and computer networks. And I happen to know from being with the aliens that all of this is reverse alien engineering. Well, he said, did you find any evidence to support this?
So the person who was interviewing me was rolling her eyes. And all I could think of saying was, I've only been able to interview earthlings. And I said, but you have these contacts. And if you can renew them and do some serious interviewing, then turn these over to historians, we can use this to revise our interpretation. So this is true. I mean, if he comes through with those interviews, we will revise our. Interpretations. So he seemed quite satisfied.
Today, my subject is systems, does creating systems, creating large complex systems, as Tom said. First, a word about the historical approach of which you will share with me today. I think what the historian can do can help engineers in several ways.
One way is to put an engineer's present work into some perspective. I hope today, when I describe the creation of several very large systems, that you will be, those of you who are into system design and management, will be drawing analogies on your own with what I'm describing historical cases with what you experience in your work today and yesterday and tomorrow.
And incidentally, if you find that my description of these two systems, the Edison electric light and power system and the ARPANET system, their creation, I should stress, don't hesitate to interrupt me and point out where you think I might have gone astray in my interpretations of the creation of these systems. I'll be giving, as Tom said, several lectures, so I have a few opening definitions. These definitions I wish to use initially are definitions of closed systems and open systems.
Now I've read the biographical sketches of a number of those of you who are now here in the open systems program or in the Systems Design Management program. And I know some of you are in product design, and others of you are in what I am talking about today, managing projects. But I assume that those of you in product design, often the activity you are engaged in is called a project.
And so I believe what I will say today, even though it's not about products, say, in the automobile industry or products in the photographic industry, it is about products. Now here is a product. I call this a closed system. It could be called a technical system. In a moment, when I give you a visual of an open system, you will see the distinction.
Now this diagram is from a technical journal around 1892, I believe, something in that period. It's an electric light and power systems, obviously. What I think you might note here is that, as early as 1982, engineers and managers were quite capable of combining heterogeneous components into a general system.
You will notice that some of the appliances here are run on very high voltage, like the industrial motors. Some on low voltage like the electric lights. Some have one frequency. Other appliances or components have another frequency.
But by 1895 or thereabouts, engineers had developed what I think you might call gateway devices, transformers, motor generator sets, and so forth that allowed heterogeneous components to be put on a single system. So it is a closed system in that it's a technical system. But it is complex.
And now I would like to show an open system. Now what I hope I can contribute to your program and through other talks that I shall give at MIT is opening your minds, if they're not already open, to the importance of open systems. Open systems, as you see from the diagram, include organizational components along with technical components.
And a number of system builders, engineers, and managers over history, over a long period of time in Germany and in the United States and elsewhere in the world, some of these people have been quite good in designing open systems and creating open systems. Now my impression is that in most engineering schools, I'm not sure about MIT, I'm fairly sure though, the emphasis is not put upon open systems. They are considered soft. The emphasis is on the closed technical systems.
But from my interviewing of a number of engineers in the large complex systems world, I find that a number of you, perhaps certainly they're compatriots, your compatriots, your peers, whom I have interviewed, are in fact designing, developing open systems. And as a matter of fact, they tell me they enjoy it very much.
They are messy. And some people consider their work soft. It's not eloquent and elegant and as quantitative as closed systems. But it is designing these is a challenge, which young engineers enjoy. I'm thinking especially of engineers I enjoyed-- I'm sorry, yes, I enjoyed. I interviewed on the Central Artery and Tunnel Project.
And I would ask them, how do you feel about this bringing into the design of the system, environmental controls, taking into account the neighborhood politics, actually rerouting the system because of community concerns? And generally, they said it makes the design project much more interesting. But they added, several of the people I interviewed, that some of the older persons did not find this congenial and would not continue the project.
Now let's take a look at this for just a moment. What I wish I had up here in this diagram was a consulting engineering firm and also a utility. A utility organization should certainly be there.
And if I'd had those two and I'd put those two into my chart, then I could say this is very much like the system that the head of German General Electric, which was called Allgemeine Elektricitats-Gesellschaft. The open system that Allgemeine Elektricitats-Gesellschaft, AEG, created in Germany around 1910, it was a system presided over by a man named Walter Rathenau, who was an open system builder and very good at it.
Now let me just spend a moment on this. The point here is let's look at the bank. A man like Rathenau, when he created the system and then later used it, and he was trained as a physicist and an engineer. He was the head of German General Electric, which was not related to American General Electric.
Rathenau wanted some control over the bank. He also wanted some control of the consulting engineering company. And let's take the bank, for example.
If his consulting engineering and construction company, which he owned stock in, or his company owned stock in, he have them stock in, if construction activity had fallen off, then if he had influence over the bank, he would urge the bank to lower its interest rates. And so that those considering construction projects would be more enticed by the money market. And so there was interaction, and Rathenau saw it, between a bank and a construction firm in the field.
And also in this diagram there should be a manufacturing company. And Rathenau, through sitting on a number of boards of directors, had influence in manufacturing companies and in utilities. And he brought about a symbiosis between what the utilities were needing by way of equipment and what the electrical manufacturers were producing or would produce.
So what we have here are functional interrelationships. And some persons have the vision, as we will see in a moment in our world today, to put together these various components into an interacting system. One thing driving these people who do this, and to me they're fascinating people, and I think, with all due respect, that Mr. Gates is one of these people, these people want control. And they want control not for nefarious reasons, but they want control in order to get the system to function in an optimal way so that they drive control.
Now if I can have the next visual. This is from The New Yorker magazine several months ago. And here, we have media companies. And you know one of the media companies is Microsoft.
And I think most of you are well aware of this, but maybe I should remind you of it, that Microsoft is creating these functional interrelationships. And perhaps one of the driving instincts of Bill Gates is to create these kinds of interactions to have this type of control. If you run down Microsoft, just Microsoft either through direct ownership, through stock ownership, or some other mode of influence, as you will see, is into cables, TV and film production, internet technology, internet content, and so forth. And you see these are all media engagements, media activities. And you can understand that he has a mode of distribution, say, the internet, it's an instinct to get control of the content that goes over the distribution, over the media, over the modes of distribution.
But if we went back to the other diagram, German General Electric in 1910, you had the same phenomenon. You had the distribution system that was also the generation of content, that is electrons, and then et cetera, et cetera, et cetera. And so these are open systems.
I have another very complex diagram here. We can show the next one Leon. This also is from The New Yorker. And the question being asked by the person writing the essay and doing the diagram is, are these media concerns, which I showed you a moment ago, are they competing? Are they collaborating?
And this diagram shows companies that are owned or companies in which control is influenced that connect the spiders, the bugs across the lines drawn between them. And you could study that. And you'll see a number of companies jointly owned, et cetera, et cetera, et cetera. So this is a very complex open system. And again, the reason it's an open system is because organizations are part of the system, I repeat, and there are technical components in the system. Now this is what's called a web of collaboration.
Now a word about projects. I would argue, and perhaps some of my friends here from the Sloan School would not accept my argument, have some in the audience who would not accept the argument, but it seems to me from reading the history of engineering management, my concern is with the management that has a engineering core. And often the people I interview and the companies I write about are managed by persons with engineering backgrounds. So that's the type of management I'm speaking of now.
It seems to me looking back over 100 years that projects are becoming increasingly the activity in which engineers are involved. Projects, and as I said, projects can certainly include product development. Let me spend a few more moments on this.
If one were an engineer, imagine yourself an engineer in 1920, and you'd accepted a job. This is my hypothesis. I'm happy to have it critiqued. And you accepted a job at the Ford Motor Company and went to Detroit in, say, 1925. Still making the Model T.
I think you would expect, as an engineer, that you would be asked to improve the functioning of the assembly lines at River Rouge and Highland Park. You would also be engaged in some improvement in the model, Model T. But you would expect these changes to be accumulative as an engineer.
I don't think you would expect to be involved in projects that would bring about radical changes in the assembly line or in the product. And also, you would not expect a need to go back to MIT or some other institution in order to keep abreast of the discipline in which you were working. That's what I believe a 1925 engineer would feel going to work he or she at the Ford Motor Company, mostly hes then, of course.
But today, one finds that you may become involved in a project. And the project might bring about a radical transformation in product, software, or hardware, or a radical transformation in the way the production line is functioning. And these projects remind me in many ways of the theater.
A friend of mine who is a director and a producer, when I told her about these projects and engineers are so often involved in she said, oh, what you're telling me about is open casting. And I asked her to elaborate upon that. She said, all right, we're doing a play.
We're not a repertory company. We're doing a play. We announce open casting. And I know that companies like Andersen and others do this. An announcement is made, such and such a project will be put into place. The project might engage you for two or three months or maybe even a year or more. And then after you come join this project, and then when the project fulfills its goal, then we will disperse. And you may become involved in another project.
In the case of the theater, open casting, you make an announcement. We are doing "The Iceman Cometh." People from all over the Manhattan area will come and try out for parts. You put these people together for a while. They rehearse for months. They put on a production. And after the run is complete, they disperse. Incidentally, that this open casting when you announce that you are going to do a play, that's a cattle call, incidentally.
Now they also, repertory companies, they do not have cattle calls and open casting. But they have the same people, as you know, doing different projects with the same people. And you'll find both of these forms in the industrial world, in the military world, and in the university world today. Virtual projects as you know they're sometimes called.
And I will describe in a matter of a few minutes, a project. One of the classic projects of history. I think most of you have heard of it. Perhaps some of you have read about it. I note that people in the Systems Design and Management, some of you are interested in history. Well, this is one of the great historical episodes. And that is the creation by Thomas Edison and his group during the period from about 1878 to 1882 of an electric light in and system.
Now what I hope I will be able to do in briefly describing this project is give you what I consider to be the essence of project, the project essence. And the projects you may be involved in differ in detail, of course. And projects done throughout history differ in many ways. But I am hoping to capture the essence of a project by going through this historical episode.
First, program management, I am contrasting program management and project management. I am drawing upon some NASA policies in this regard. NASA, in the past at least, and many of the things I say come from the past, NASA defined program management differently from project management.
Now first, program management in the case of the electric light and power system, program management, people involved in this are concerned about funding and are concerned about politics. And sometimes, they must become involved in organizational creation as well. Now in the case of this particular project, an organization had been formed.
And as I go through this, perhaps you'll be impressed, as I am every time I involve myself with the history of this project, how contemporary this Edison story is in its detail. Perhaps you will agree with me. Perhaps not.
But a company was formed 1878 called the Edison Electric Light Company. Now the Edison Electric Light Company was formed by the program manager types. One of these was a man named Grosvenor Lowrey, who was a very prominent New York City lawyer. And it's not unusual to find people from law and other disciplines involved with engineers in these open system creations.
Now Grosvenor Lowrey was the lawyer for the Western Union Telegraph company. So he was working in the world of technology. Lowrey said, I think we should create an electric light system. I'll explain why he chose that problem in a moment. He said this to Edison. He said, first, we must create this company. And this company will be an investment and patent holding company.
What's new? It will fund Edison's activities. It will take over his patents. It will get a return on the investment. This is venture capital in 1878. And the company was formed.
Another thing Lowrey did, and this is another requirement of open systems, as some of you here I know are familiar with this, the political stage has to be set for the project. And many engineers in projects are not aware of the political maneuvering that has gone on before the project comes into being. In this case, the councilmen in the city of New York in 1878 were very much under the influence of the gas lighting people.
And so the gas lighting people put a lot of pressure on the councilmen in the city of New York when the gas lighting people learned that there was to be an effort to create an electric lighting system. And the gas people said to the councilmen, don't let these people open up the streets, because the distribution lines would be put underground in New York City. So one way to block the project was not to give the Edison Electric Light Company permission to open the streets.
Well, Grosvenor Lowrey went to work lobbying and in other ways in bringing about a change in attitude on the part of the councilmen of New York City. And I think you can well imagine what he might have done to change their minds. So the political stage was set.
And very quickly, I don't want to dwell on this, but it's an anecdote. One thing Larry did, he invited all the councilmen out to see a test of the Edison electric light system in 1879. And he was taking the-- he brought them out in a handsome railway car, took them around and showed them a proto electric light system that Edison had put up at his Menlo Park laboratories. And these people were becoming very bored, the councilmen, with what they were seeing.
And then the councilmen were moved into Edison's laboratory. It was completely dark. Suddenly, the lights came on. The first display of indoor lighting for these people. And when the lights came on, what was displayed? A marvelous meal from Delmonico's with much liquid refreshment to go along with it. And incidentally, that banquet had to end within an hour and 35 minutes because Edison's electric lights would not burn for longer than an hour and 35 minutes.
Now this is program management. We have to respect it. It sets the stage. Now we go into Edison as the project manager.
Now let's forget the heroic Edison of the books we read when we were young. This man was not working alone in a garret. He had one of the best equipped laboratories in the United States.
He also had his team. His team included a physicist, who had studied in Berlin. It included marvelously talented machine tool people. There were also chemists in his team as well as glass blowers who were brought into it for this project. I could go on and name the craftsmen.
But the point to be made here that the historian wants to make, Edison had brought this group, this team together. And I know many of you have been involved in teams. Edison had brought this team together in earlier years to work on the telephone and to work on the phonograph. Now he had this team in place, so he had to choose a product. He had to choose a system that could utilize the resources in the personnel that he had at his disposal.
So I think we need to remind ourselves that we are constrained by the organizational concepts that come out of the past and in which we are working now. We are constrained. There are only certain projects we'll do. There are only certain projects that are suitable.
So Edison decided that electric lighting was suitable for the team that he had in place. Now once he had his team in place and he had his equipment in place, then there was the problem of conceptualizing the electric light system. And I think this is where Edison's brilliance along with the intelligence of some of his team comes into play.
And if I may have the other diagram. Yes. There we are. There's nothing in Edison's notebooks that gives me the evidence to feel confident in saying that what I'm telling you now is other than a hypothesis.
But from circumstantial evidence, this is the way that I believe Edison reasoned his way to his concept, his design concept. As you know in a project, a design concept comes before a preliminary design. Well, from this little exercise in following Edison's reasoning, I think you may better understand the man's so-called and I would say, yes, his genius.
You'll note the interdisciplinary character of the thought processes. He realized he had to analyze the market and with whom was he competing. He was competing with gas lighting.
And from gas lighting, he drew an analogy with regard to the overall architecture of the system that he would design. He had to be concerned about the economics of competing, but also this analogy with the architecture. It was very helpful to him. Gas lighting or the gas companies were then distributing from central plants. They were distributing through an entire city from one central plant.
And Edison decided that his architecture would be like that architecture. But let's go back to the economics of the system. He had a general concept of what the system would be like. He would have distribution lines of copper. It would have generators, of course. And it would have the electric lights.
And he reasoned with the help of some of probably with Upton, the man who'd studied in Berlin under Helmholtz, he reasoned that the reverse salient in the system, the bottleneck, if you prefer, the reverse salient in the system was the cost of copper. So here's a man who is into designing an electrical system but he realized that materials are his problem.
And Edison didn't hesitate to move from one area from materials area into the electrical area into the chemical area. Wherever the problem was, he would move. He wasn't bound by any disciplinary restraints. So he said, the problem is the cost of copper, the cost of copper in the distribution lines.
Then he turns from materials and cost to the science that he had at his disposal. Now remember this is 1878 and the science available to electricians is very limited. He had Joule's Law and he had Ohm's Law.
And from thinking with Joule's Law and thinking with Ohm's Law, he knew that the losses in distribution would be an equal to the square of the current times the resistance. Or to put it another way, the energy loss in the conductors would be proportional to the current squared times the length of the conductor divided by the cross sectional area. If one increased the cross sectional area of the distribution lines, then one would reduce the energy loss in the distribution lines.
But to increase the cross sectional area was to increase the use of copper. Therefore, this was not an option for him to continue to increase the cross sectional area until he lowered his energy lost in distribution. So assuming a fixed length and keeping the cross section area low, how was he to solve the problem of getting the required amount of energy into his electric lights?
Now he knew from Ohm's Law that the electric lights had to have a power on energy consumption which was equal to the current times the voltage. Now he knew from the energy loss in the distribution lines, he had to keep his current low, because the energy loss was current square. I'm going across into this in such detail, it may seem elementary, but this is the way I think the man's mind was working.
And so therefore, he has to keep the current low because current square. He'll raise the voltage, so he will have sufficient current times voltage to give him the energy he needs at his electric lamps. Now how does he raise the voltage and lower the current?
I went into an old Encyclopedia Britannica for that period. And I found that the way that resistance, Ohm's Law was stated in those days was that the resistance equals the ratio of voltage to current. And there's the obvious answer. I think that's the eureka moment for Edison was when he saw that by raising the resistance of the electric light element, he could increase the voltage that would be going through his transmission distribution lines and reduce the current. So that is why the man, and we go into popular stories now, spent months trying to find a light filament that gave a maximum of resistance.
And what he ended up with and that determines even today the standards of our electric lighting system, he found that he could get a filament that would take a current of one amp, that it would have a resistance of 100 ohms. And so that would give him 100 watts. And so there you have the standards that we have in our electric lights today because the highest resistance he could get was 100 ohms.
He needed one amp to get the wattage he needed. Why that wattage of 100 watts? Because 100 watts gave him the same lighting intensity as a gas light was giving at that particular time. So Edison's conceptual design.
I hope that you are as impressed by the man's thinking process as much as I am, even though the scientific laws are not esoteric. The quantitative activity is not remarkable. But putting together the various aspects of the problem is what I find extremely impressive.
Now let's go on to the open system. He now has the essence of his electric technical system, his closed system. Now we go into the organization of the Edison company.
I have interviewed a number of persons in the field of complex systems. And they have told me that it's necessary if the closed system is radical, if the technology is radical, it is advantageous to create a new organization to nurture the radical technology. They tell me, and I'm thinking about an interview with Simon Ramo, who worked on the Atlas missile, did systems engineering for the Atlas missile. And I'm also thinking about General Schriever, who presided over the Atlas missile. And I'm thinking about the people who presided over the Polaris missile.
And in both those cases, the Polaris and the Atlas and numerous other system designs, the system builders not only produce the new technology, but produce new organizations in order to get the radical technology outside of the bureaucratic structure. General Schriever told me, and I'll talk about Atlas next time, General Schriever said, I had to get out, away from those bastards at the Pentagon. He said, I was dealing with people who could say, no, and could not say, yes, the assistant secretaries. I had to get outside the system. So he forms a new organization, the Western Development Corporation, part of the Air Force, but outside of the Air Force and the Pentagon bureaucratic structure.
Well, look what Edison did. Edison and his advisors did not depend upon existing manufacturing companies. But they formed this consortium of companies in which the parent Edison company, the Edison Electric Light Company, the venture capital patent holding company had interests through stock, in most cases. So you had the Illuminating Company, which is a utility, the Edison Electric Illuminating Company in New York and Brooklyn, and other illuminating companies, the Thomas Edison Construction Company to construct the electric light and power systems, the Edison Machine Works that manufactured dynamos, the Edison Tube Company, underground distribution, lamp works obvious. Bergman was fixtures, and other manufacturing came from United Edison.
What it is worth noting here, this system builder, Edison and his team, they have control of the manufacturer, like the German General Electric I told you a moment ago, they're controlling the manufacturer. And they're controlling the market for the manufacturer. That is, they're controlling the utilities that buy the generators that are made by the Edison machine works.
What's new Mr. Gates? In this effort to get control not only of other production, but the content and distribution and so forth. And so we see this control at a very, very early day. Incidentally, most of the manufacturing companies here were combined in the early 1890s into the General Electric Company.
Now I want to turn to a more recent-- well, I should say that the system was deployed in New York, in Brooklyn, and elsewhere. And the company, the Consolidated Edison Company flourishes in New York. It was established by Edison those many years ago, et cetera, et cetera, et cetera. I don't think we need to go into the further development and deployment.
Let's now turn to the ARPANET, which I think you all know is the beginning or the predecessor of the internet. And this story is I think one of the remarkable accounts of system building in the last 30 years. So I would spend some time telling the story of ARPANET.
Again, I mean to say telling a story. Because I know most of you are involved in very elegant problem solving. Your reports are rigorous and quantitative. And I'm telling a story, which you usually do not do in your professional work.
The reason I'm telling a story is because stories can-- well, there are many reasons. I couldn't do the kind of work you're doing. But another reason is stories encompass messy complexity in a more embracing inclusive way than a more scientific elegant account can do. So a story is one way of conveying messiness.
And I'm arguing and I'm arguing with some of my friends at MIT that MIT students need to hear more stories of messy complexity. They may find it soft. And they may find it at first not congenial. But the people I've talked to out in the field often say to me, yes, yes, do tell messy stories because the mess is what many of the young engineers will soon be involved with.
All right, turning to the ARPANET. In following the pattern that I have described in the case of the classic Edison story, let's begin with program management of the ARPANET. Now the program manager of the ARPANET, this is my terminology, some people in the audience may disagree, the program management was the responsibility of DARPA, the Defense Advanced Research Projects Agency. And I trust most of you are familiar with it.
It has been a remarkably successful funder and nourisher of projects since 1958. It is still flourishing. It has changed in character, in policy, but it flourishes. It has been called at times Advanced Research Projects Agency, ARPA. At other times, the Defense Advanced Research Projects Agency. I refer to it as DARPA.
It was founded, established in 1958 in response to the Sputnik. If you don't remember, I do remember. You may have heard that after the Russians launched Sputnik, there was great anxiety in this country that our technology was falling behind and there were some measures had to be taken the Eisenhower administration felt in order to respond to this problem. And one of the responses was DARPA, which was to nourish and develop and fund radical technology. I'm using radical in a very general way, but highly innovative technology, if you prefer.
Now note that DARPA is a Pentagon agency. DARPA gives military funding. But I stress that because DARPA has funded a number of projects that in their impact and even in the process of being created seem rather distant from military needs.
But let me go on with the project management that DARPA was engaged in when the organization decided to do a computer network. All right, now first, what's momentarily distracting me, I see a quote I want to give you here. One of the major DARPA personalities, Licklider, said, what the military needs is what the business main needs is what the scientist needs. So what the military funds can be useful of advantage to businessmen and scientists.
And this statement can be made as long as the level of abstraction is high. Where the businessman and the scientist and the military find their needs differ is when one lowers the project down the ladder of abstraction to very practical, specific applications. But when working at a general level, a computer network, general, all of these contingencies, these clients can be responded to at this level.
Another person I interviewed told me that the military is a very big society. The United States is a very big society. This is a DARPA person. So is it any surprise when we fund a military society need we also funding a civil society needs? Both, for example, need communications. Both need calculations, et cetera, et cetera.
All right, another characteristic of this program manager, DARPA, besides its willingness to innovate on a rather abstract level and respond to a number of interest was that the people in the early days who came into DARPA to manage as program managers, they were often on leave for two or three years from industry, on leave from universities for two or three years. So what were they bringing? They were bringing with them a university style or an industry research style.
What else? Two or three years, and then they left. The project was complete for them. Their performance was over. They left and others came in.
It is difficult to argue that DARPA was a bureaucracy with hardening of the arteries when you have these people coming in for short periods with new ideas soon to be replaced by other people with new ideas. Another characteristic of DARPA, the program management, the staff was extremely lean, relatively few people compared to other agencies. And another characteristic was that DARPA preferred to fund very large expensive projects. And this meant that a small staff could manage, instead of having to manage 1,000 little projects. You had a small staff, five or six people in case of one of the branches of DARPA, that is IPTO, which I'll talk about in a minute. You have five or six people managing millions of dollars, but only a few projects.
Now I want to say something about a remarkable project manager. I'm sorry. I've been talking about program management. Now I want to talk about a project manager, a person who worked within the context of DARPA. And that is Joseph Carl Robnett Licklider, who is usually called Licklider. He set a pattern of project management that I think has had enormous influence in the world of project management and engineering to this very day.
He had a number of devoted followers who admired his style, saw him as a visionary, and put into place and deployed the kind of project management style for which he became noted. Let me say something about Licklider in the time I have. Note the man's pedigree.
This gives us insight into a very interesting period of his history and some fascinating people. Licklider was a psychologist by training. I should say here that he was the first head of the information processing techniques office, IPTO, which was that part of DARPA, ARPA, which presided over the creation of the ARPANET, IPTO. Licklider later from 1962 to 64 was the head of IPTO, information processing techniques office of DARPA. OK.
But what I want to stress, this man with a background in psychology had a remarkably rich career that gave him to experience from unusual capabilities. And maybe some of you have had similar experiences. During World War II, and many of the people who came to the fore in the period about which I'm speaking, the '60s and the '50s, had an interesting World War II experiences in the realm of science and technology. World War II for scientists and engineers was a remarkable learning experience.
Well, Licklider, during World War II, had been a part of a psycho-acoustics laboratory at Harvard University. Then after the war, he moves to MIT to head an acoustics laboratory. How does a man move from acoustics to computers? Well see.
Now while at MIT, he became involved in those organizations that were providing rich experiences for those who would take the risk of becoming part of these new and risk taking organizations. For example, he became a research associate in the MIT research laboratory of electronics. Now that laboratory was famous in the 1950s and '60s for an interdisciplinary approach. Interdisciplinary with something still fairly new in the engineering world.
So he becomes a part of MIT research laboratory of electronics. He also takes part in Project Charles. Project Charles was a summer study project. We don't have these projects anymore.
Summer studies were very popular at MIT in the '50s and '60s. What was a summer project? A summer project was funded by the military, generously funded. The project involved inviting a number of academics and industrial people and military people, maybe 30, maybe 40, that neighborhood number, to just take off from their responsibilities for an entire summer. Later, it became sometimes six months. And focus at a bucolic site, geographical site, somewhere up in the mountains, down to the seashore, these 40 or 60 people free from their responsibilities would concentrate upon one particular problem of interest to the military for six months or a summer.
And one of the major projects was called Project Charles, which dedicated the people who were members of Project Charles dedicated themselves to setting up the air defense system called SAGE. Well, Licklider took part in one of these summer projects. And I'm told by people whom I interviewed at the summer projects, that when you entered the door wherever the meeting was being held, that you were told to forget your discipline, to forget that you were an expert, and that just focus on the problem and go wherever the problem took you into whatever field of engineering or whatever field of science because there are no experts in this area we're exploring.
And incidentally, that was the same technique used at the Rand Corporation about the same time. Forget your expertise. Let the problem decide where you would do your thinking. So Licklider was a part of Project Charles.
Also, he heard about Norbert Wiener. I think all of you have heard about Norbert Wiener. And so he became a part of the group that would mix and meet with Norbert Wiener. So this man had a rich, rich MIT experience. And after MIT and in electrical engineering department and acoustics lab, he went to Lincoln Laboratory. That was another great learning experience in the '50s.
Lincoln Laboratory was established to do the research and development for the SAGE air defense project. But it was funded by the Air Force. It had some of the most advanced computers in the world. And so if one had an opportunity to go to Lincoln Laboratory, as you know which is still administered by MIT, then one had hands on with the most advanced technological and specialty computer equipment.
Then from there, he went to Bolt Beranek and Newman, a high technology firm founded in 1948. I think you're familiar with this firm today. It's a major player in the acoustics and the computing field. So he moves on to Bolt Beranek and Newman. Rich experiences which prepared him for his work at IPTO.
Now Licklider was visionary. And what was his vision? He had a vision in the mid 1950s, he had a vision of time sharing on a computer. And he also had the vision of interconnecting computers. And he often gathered together people in the Cambridge area, maybe over a few drinks, to talk about the future, the possibilities of time sharing, the possibilities of computer interconnection.
He spoke a man-computer symbiosis. Now Licklider, in this earlier period, the 1950s, was not moving towards the development of the computer as a calculating arithmetic device. He was one of the pioneers. He saw that a computer could be used for control and decision making and other purposes. So the man has vision. He has a number of followers.
And now I would like to show you the Licklider network. I hope you have it before you on one of the charts. To me, this is a remarkable network. Please note the movements of these people. These are people with, most of whom, Licklider is intimately associated.
And the majority of them have similar experiences to which he had, MIT, electrical engineering, Lincoln Lab. Some of them at Bolt Beranek and Newman. This was the context in which computer pioneers who made the ARPANET learned the trade. And note the movements of a relatively few people and the organizations with which they were associated.
Now turning to the choice of problem, choice to work on the ARPANET. Rather simple origin, ARPA was funding a number of computer research centers in the United States in major universities, such as Carnegie Mellon, MIT, UCLA, Stanford Research Laboratory, and so forth. ARPA was funding these computer research centers. Actually, these were computer research centers which Licklider had his network located. And he funded his network through these computer centers.
Now the cost of the computers is very high. And it was thought that if one could bring these centers to share their computers, to interconnecting the computers, ARPA costs would be reduced. And according to Robert Taylor, who was one of the IPTO heads, the idea for this cost saving interconnection came to him when he was sitting in his office in Washington. Remember he's an IPTO head.
And he had terminals from three time sharing computers in his office. I think one of the time sharing computers was at MIT. Another may have been somewhere in the Washington area. I'm not sure. We had three terminals from three time sharing computers. And they were not interconnected. And he said, well, the obvious thing is to interconnect the other time sharing computers. And the need, of course, is to reduce cost by sharing computer capacity and sharing software.
Now a word about this is the inauguration of the project. The conceptual design of the project was worked out by a distributed committee. In the case of Edison, who had the people who came up with the conceptual design, that thought process, that reasoning I went through, they were located in one place. They were in a hierarchical structure headed by Edison.
But it is remarkable that the conceptual design for the ARPANET came about through informal meetings of the heads of these various computer centers, informal meetings called by Licklider. And Licklider, in one of the interviews that I've used, said, all of the heads of the research centers would gather at some place for a committee meeting, a committee of research heads. And we'd have some alcohol in the late evening.
And then I might bring up the subject of computer networking. And we'd stay up into the late hours. And then perhaps an idea for the IMP, the information processor, Wesley Roberts would come up with that. And then somebody might mention packet switching, and we would talk about that.
So what Licklider was doing is with his network, he was drawing out of them the ideas through discussion that would become the conceptual design of the ARPANET. Now the next step in that procedure was that IPTO, the office that would preside over this development, IPTO sent out a call, a request for bids to a number of companies. And the request for bids had this basic conceptual design in the request.
We want a computer network with so many nodes. We want to use packet switching. We also are interested in having interface computers that will allow their host computers, although they have different design, to interconnect. Even though they have different characteristics, they will interconnect to the telephone network through these IMPs, these interface computers. These ideas were coming up through these discussions that took place over a number of months.
Now the fundamental contract, the basic contract for the design of the IMPs was given to Bolt Beranek and Newman, the company which you remember Licklider was a researcher at and then later a vice president of. So this company had the responsibility for developing the interface computer, the IMP.
Now I want to just say a word as a move to conclusion about the style of research at BBN because I think this style of research at BBN has become a rather commonplace style in the world of projects today. You can tell me whether this is true or not. And let me read a quote from Robert Kahn who was at BBN.
He said, BBN-- this is from an interview. BBN was a kind of hybrid version of Harvard and MIT in the sense that most of the people there were either faculty or former faculty at either Harvard or MIT. The role that MIT plays in this episode is a remarkably active one.
If you've ever spent any time in either of those places, you would know what a unique kind of organization BBN was. A lot of the students at those places spent time at BBN. It was kind of like a super hyped up version of the union of the two, Harvard and MIT, except that you didn't have to worry about classes and teaching. You could just focus on research.
It was sort of the cognac of the research business, very distilled. The culture of BBN at the time was to do interesting things and to move on to the next interesting things. And there was more incentive to come up with interesting ideas and explore them than to try to capitalize on them once they'd been developed.
So BBN will develop the hardware and some of the software for the ARPANET. And also one more quote about the style and BBN. One of the team managers, Hart, said, what we were doing in developing this hardware was a labor of love. It was all fun.
And I have enough collaborating evidence to accept this. And we all on the team, a team of 10 or 15 at BBN, we thought holistically. We reached our decisions by consensus. The team leader would only come in when you couldn't reach a consensus. There was no micromanagement here. We were suitably black boxed.
Now another interview with one of the persons at BBN said, when a breakthrough occurred, we'd run in and say, look, I got this running. Somebody come and type this on the teletype. This is exciting. Something is cycling.
Now another characteristic of this research that was going on at ARPA was the meritocracy. It wasn't a hierarchical activity. And one more quote. A group of graduate students were set up as a committee at another site than BBN, but this type of thing went on with BBN too.
A group of graduate students was set up to develop the protocols for the internet. And here's a quote from one of the graduate students. We were just rank amateurs and we were expecting that some authority would finally come along and say, here's how we're going to do it. A professor would come along and say, here's how we're going to do it. And nobody ever came along. So we were sort of tentatively feeling our way into how we could go about getting this software up and running. And there are many, many, many examples of this sort of flat organization.
Now in conclusion, I want to show you-- don't show the ARPANET. Well, yes, quickly the ARPANET diagrams. These are historic. This is the first diagram I encountered. It was done in longhand. And it showed one IMP. That's the interface computer that connects the host with the telephone network. And one host computer.
Let's see the next one. This is by August 1972. The other was in a few years earlier, three years earlier. And this is a network as it was in 1972. I like to show this because the growth of the computer network like the growth of a railroad network.
Next one please. And in conclusion, something I hope we can talk about it at another time. I wasn't able to give you all of the information I'd like to have given you about the style of BBN, the ARPA style of management. I hope I indicated and suggested it to you the style.
But out of the study I've done, I've come up with a contrast between the kind of production management that took place, say, at Ford Motor Company in the 1920s and the kind of project management that I count on these projects today. And let's run very quickly through the contrast.
Before World War II, hierarchical. Now today, flat. Specialization was well regarded. Now it's interdisciplinary. Integration over coordination. Rational order was the objective. Now messy complexity is accepted. Heterogeneity is accepted. We could go on down the list.
Experts were highly regarded. Now it's more of a meritocracy as the anecdote I gave you about the graduate students. Tightly coupled to a network. Let's have the next one very quickly. And then program control against feedback control. Systems engineering against taylorism. And it seems to me as we move into the world where projects play an increasing role, these are some of the characteristics of project management.
Thank you, Tom.
[APPLAUSE]
MAGNANTI: We have a few minutes for questions or commentary either from people here at MIT or people at a distance. Anyone like to add or ask any questions or comment?
AUDIENCE: Professor, I'd like to know if there's plans to post the slides that were shown? Some were a bit difficult to read from a distance.
MAGNANTI: Yes. Yes, we do plan to post the slides. So they will be available for everyone, yes.
AUDIENCE: Secondly, I have a question about the Thomas Edison slide showing the organization that was early in the presentation. You showed that as being separate companies, or was that the forerunner to Ford's concept that was very highly vertically integrated? It wasn't unclear to me?
HUGHES: You're speaking of the Edison companies? I think you're speaking of the Edison companies, the ones that were established, like the Edison Manufacturing Company and the utilities and so forth. These companies it wasn't a vertical. No, it wasn't vertical. It was horizontal.
The companies interacted. But there wasn't a central management in 1882 when these companies were established to manufacture the generators, and the companies established to manufacture the light bulbs and the underground distribution and so forth. No, it was a horizontal. It was a horizontal arrangement with the parent company, the Edison Electric Light Company, the patent holding company, the investment company having stock interest in these companies and some control through stock interests. But it was a loosely coupled organizational system.
AUDIENCE: Thank you.
MAGNANTI: Please come down to the mic, Joann.
AUDIENCE: One of the terms that I was looking for in this comparison that I didn't see was emergent. The project management that you described in ARPA seemed to me a much more, at least in this project, a much more emergent kind of thing. Because as we all know, the ARPANET wasn't intended to be a communication network originally, and of course that emerged and so on. The graduate students who sort of developed their own standards and so on. Could we add emergent to this project management?
HUGHES: We certainly could if you would define.
AUDIENCE: Emerging over time out of the interaction of the various socio technical elements, but not managed, not planned in advance. Yeah.
HUGHES: Yes. Yes, it certainly was, Joann, it certainly was a emergent project. Because at the beginning, as I said, the objective was to lower cost, the primary objective. I haven't talked about the military background. The primary immediate objective was to lower costs through the sharing of computer resources and software resources.
But as you point out, and within a few years after that demonstration of the ARPANET in 1972, within a few years, email became the major traffic generator on the ARPANET. And there was no intention at the beginning of it using it primarily as a mode of communication. And then also, I think as you know, I didn't go into this because I assumed you knew it, the ARPANET after a while was turned over. The military took over the military aspects of the ARPANET. And the other parts of it were turned over to civilian agencies. NSF, for example, became a major funder of computer networks.
And so the ARPANET was the one at the core. But then a number of other networks funded by other agencies were joined through protocol development to the ARPANET. So that wasn't expected either. It's good to stress this.
And Edison didn't foresee many of the future developments in electric light and power. He didn't see alternating current. He did not see regional electric light and power systems. I'm glad you made the point, Joann.
MAGNANTI: Other questions, comments?
AUDIENCE: Tom, I'm trying to get a better grasp on the difference between an open and a closed system. Would you characterize the ARPANET project as an open system or a closed one originally as it started out?
HUGHES: Yes. I want to be sure this is on. The ARPANET I call an open system because IPTO established a number of these research centers. And the research centers were the organizational framework within which the technical network was housed. So the ARPANET involved not only the technical aspects, the computers, the telephone system, et cetera, but it also involved the organizations, the research centers at the various universities. And you couldn't have one without the other.
So people at IPTO had to nurture the organizations at the same time they were nurturing the technical content. And it was also an open system because the designers of the system had to clear the funding through Congress. Also, the Air Force, the military generally was funding and funding had to be arranged, et cetera. So it was a complex system that involved technical components, funding components, political problems, organizational components.
And that's the point I am trying to stress is most of the technical systems you deal with are embedded in a larger structure that can be called an open system. And as Joann points out, these open systems often are emergent systems and have many of these complicated characteristics, extremely difficult to manage once they're in place.
AUDIENCE: Great. Thanks, Tom.
HUGHES: And because of this audience, I'll ask you. Do you find that projects are increasingly common in the organizations within which you work? I gave you the analogy of the theater where a group is brought together for a specific length of time to solve one particular problem. And the people on the group may come from various parts of the organization or company. Then after the project is fulfilled, the goals are met, then the project disbands and people go back to their respective areas of commitment or wherever their home base is.
AUDIENCE: I have a question, but my thoughts are spinning such, I hope I can clarify it in a way. I'm affiliated with the Air Force right now. And my questions go back to the idea of the conceptual design and how you talked about the network that was among these people and the loose style that was there.
In the research I've been doing, I've been seeing the Air Force has tried to capture these networks of people. But they're not being very successful. Instead, what's happening is those projects that are getting funded are from a different kind of a person that has another network, an informal network.
I guess what I'm trying to say, the Air Force has tried to formalize and structure and establish networks to emulate some of these attributes that you've talked about. But they're failing. And it's this informal network is the one that's still working. And I'd be interested in your comments on that and maybe because it's something the Air Force is struggling with.
HUGHES: I'm glad you brought that up because it allows me to go back to Licklider. I think I showed this diagram, which you have, of the Licklider network. And Licklider cultivated this network informally. But he also structured it formally in that he would help the people in his network become located in these research centers.
And then he would also, when he had this funding resource at IPTO, which is part of ARPA, he would fund people in his network who were located in these various research centers. But again, on the informal level, again, as I said, his people would often gather together at meetings he would call and talk into the late hours about concept for time sharing or concept computer networking.
But so he was playing the informal and the formal. Formal being, again, placing the people in an organizational structure, these research centers, for example. So I think the most direct answer to your very interesting question is in government today, project managers, and that's what Licklider was, he was a project manager for ARPA, are extremely important. And there are project managers in the Air Force, as you know.
But I've been told by some of the people whom I've interviewed that the project managers today are not given the freedom and the leeway to organize informal networks, to draw out conceptual designs as freely and imaginatively as they once were able to do. So I think the answer to your question is put your attention upon the functioning of the project managers, people like Licklider in Air Force, and what kind of success are they having? What kind of reinforcement are they getting? What sort of freedom of action do they have?
And again, I repeat, I've been told that the project managers do not have the freedom to maneuver and to develop and feel imaginative concepts that they once had because of a number of bureaucratic constraints and regulations. And I think I know what you mean by an informal network. That is a special interest group that is responding to the needs of, say, a particular small group. And you don't have this drawing upon a community that Licklider drew upon. Yes.
MAGNANTI: Let me, on behalf of all of us, thank Professor Hughes for this fascinating account of two remarkably different projects. I think as you explained them. Whether one is the old fashioned way of doing things, the Edison way, and the ARPA is the new fashioned way, I think is one for all of us to think about. But the contrast is rather stark, I would say, in terms of the differences in the organizational style and the approach for these two projects. Both eminently successful in terms of their projects.
I would recommend, again, to everyone to Professor Hughes' book, particularly those of you who are interested in MIT, because there's a lot of MIT in this book. I'm really impressed as you look through it the impact I think that MIT has had. And all of us should feel good about this in terms of the way of thinking about complex systems. This has been an organization that's been deeply immersed in this for many, many years.
As I said at the outset of this presentation today, Professor Hughes is on campus for the semester as a guest of MIT. He will be around in the MIT community. I think delighted to meet with anyone who might like to talk to him further about these. He will be giving another public lecture on October 30 to pursue some of the topics that he talked about today and I think talk about some other of these large scale projects. And he'll be meeting with the SDM students in private.
So please join me again in thanking Professor Hughes for this very illuminating presentation.
[APPLAUSE]