Email Servers

The NSFNET Backbone Project, 1987 – 1995

NSFNET: A Partnership for High-Speed Networking

The Role of Government: The National Science Foundation

Image of 3-D NSFNET Traffic Map How did the National Science Foundation become the Federal agency tasked with managing the nation’s high-speed backbone network? Other Federal agencies, notably the Department of Defense, the Department of Energy, and NASA, had their own networking projects. According to an influential early article in Science, “Computer Networking for Scientists,”(1) in 1986 there were various Federal wide-area networks operating, using protocols from DECnet to SNA and linking hundreds of Federal researchers at sites around the country. Competition between agencies regarding these projects was not uncommon. However, the National Science Foundation had several things in its favor, including the presence of the 1986 version of the NSFNET and support of another network service for computer scientists called CSNET. The first NSFNET backbone service went online late in 1986, linking six sites, or nodes, together at the five NSF supercomputer centers (San Diego Supercomputer Center, National Center for Supercomputing Applications, Cornell Theory Center, Pittsburgh Supercomputing Center, and the John von Neumann Supercomputer Center) plus the National Center for Atmospheric Research, or NCAR. When the solicitation for the new and improved version of the NSFNET backbone was released in 1987, it sought to build on the success of the 1986 backbone by keeping the basic form of a centrally managed backbone network while expanding the number of backbone sites from six to 13; accelerating the speed (from 56 Kbps to T1 and eventually T3); and improving the reliability and robustness of the service. NSF staff in the Computer and Information Science and Engineering (CISE) directorate, created on October 1, 1986, envisioned the overall design of the new NSFNET as consisting of a logical topology, or architecture, composed of three “tiers:” the national backbone, various regional networks (such as NYSERNet and Westnet) and campus networks. The overall “internetwork” formation, linking networks of different sizes and running different protocols via TCP/IP, had been borrowed from the ARPANET for the original 1986 NSFNET and would become even more complex with the proposed three-tiered 1987 network. Various organizational and managerial characteristics also distinguished the NSF from other agencies. The NSF’s primary duty was perceived to be broader than other agencies. Jane Caviness explains: “While NSF serves science and engineering generally, other agencies focus on mission-specific needs. The CISE directorate in particular is charged with promoting technology development for all of US research and education.” Because of this, the NSF also placed fewer restrictions on the use of the NSFNET backbone by researchers and scientists of every stripe, within the limits of NSF’s Acceptable Use Policy (AUP) requiring the network to be used in support of research and education. “The way we ran the program made it very easy for people to use the backbone, to connect, to do all kinds of tasks,” says Caviness. Other organizational processes (such as peer review) and instruments (such as the cooperative agreement), while not unique to the NSF, were used to good effect for the NSFNET backbone service. The use of peer review, both in the initial selection and as the project developed, kept the best resources and advice in the networking community available to NSF for the project; it allowed the NSF to stay close to the “cutting edge” of thinking in academia and industry. According to Steve Wolff of NSF, the value of this institutional structure to successful R&D initiatives is undeniable: “The NSF has been at its best, I think, when it has followed a very rigorous and formal and open policy of peer review. It not only funds the academic community, but it tries to achieve consensus on the worth of what it supports, from the very community that it’s supporting. In that sense the community is policing itself. And when the community is a good community, with high standards, it produces some of the very best work that has ever been done.” Even before the NSFNET, participation in the development of supercomputing and networking initiatives by the academic community had been strong: numerous advisory committees, including the Federal Coordinating Council on Science, Engineering and Technology, and the National Research Council’s Computer Science and Technology Board as well as various NSF-sponsored advisory committees contributed ideas, advice and comment to the NSF. In addition, since the NSFNET was created as a resource for the U.S. research and education community, NSF was able to use a cooperative agreement as the award vehicle. Utilizing the cooperative agreement enabled the NSF to build and maintain a backbone network service in support of the research and education community while maintaining “a substantial involvement in the process,” says Don Mitchell, Staff Associate at the NSF. A cooperative agreement was chosen as the award instrument for the NSFNET backbone because a contract is used only when procuring specific goods and services for the government, and a grant does not allow the government to significantly guide the management of a project. The organization within the NSF specifically tasked with overseeing the NSFNET program was the Division of Networking and Communications Research and Infrastructure (NCRI). Foresighted and flexible, NCRI staff seemed to exhibit that most precious commodity of managers: knowing when to step in and when to keep out. NSF participated in the biweekly Partner’s Meetings, in addition to the more formal quarterly Executive Committee meetings composed of representatives from each of the partners. But this level of involvement was balanced by a reluctance to micromanage, in addition to the flexibility of the cooperative agreement structure. Hans-Werner Braun puts it this way: “Having a cooperative agreement with the National Science Foundation, as opposed to a grant or contract, was a real advantage. Often when I needed something done I could call Steve Wolff and say ‘Steve, I want to do this, do you have any problem with that?’ And he would say, ‘No, go ahead.’ And I just did it, without any paperwork or additional funding.” The first NSFNET Program Director, and the driving force behind the inception of the NSFNET backbone project, was Dennis Jennings, who came to NSF in 1985 from University College, Dublin. Jennings’ immediate successors, Steve Wolff and Jane Caviness, were both closely involved with the NSFNET project. Wolff had been a communications engineer working for the Army research laboratory on various projects in communications and computer systems when he was brought to the National Science Foundation as the second NSFNET Program Director in June of 1986. In September of 1987, just before the award to Merit was announced, Wolff became Division Director of NCRI and Jane Caviness, Director of Academic Computing Services at the University of Delaware, replaced Wolff as NSFNET Program Director. Caviness held that position until 1990, when she became Deputy Division Director of NCRI. Wolff and Caviness together worked hard to support the NSFNET partners within the NSF, Congress and through two Administrations: “Steve and Jane really have been the driving force,” says Ellen Hoffman. Mark Knopper gives Wolff credit for advancing the cause of the NSFNET: “The National Science Foundation was very visible and very active in this project. Steve persevered at NSF and at other levels of the government at the time, including then-Senator Al Gore. It was Steve who told them all that we’ve got to do this. And they said, ‘OK’. Steve was able to leverage all of these people and get corporate involvement as well.” Al Weis puts it this way: “There was one person with a lot of guts, a lot of brains, and a lot of vision. And that person was Steve Wolff.” Additionally, according to Wolff, “Jane really held the program together for a couple of years:” “When we were building the regional networks, we were empowering them, but with grants that were not enough money to build the networks. Jane was well known in the academic computing community; she was an officer of EDUCOM, and everybody knew her as honest, straightforward, and a hard worker. She went to each regional network, to each person in charge, and said, ‘Doesn’t matter. We won’t be able to get you the money, so you’ll have to support it however you can.’ And the academic community came through.” Caviness was succeeded by Doug Gale as Program Director in 1990. Gale did much of the background work for the information services component of the 1993 NSFNET solicitation, and set up new organizational procedures for the NSF “Connections” program. George Strawn succeeded Gale in 1991. Strawn, who was instrumental in realizing the vision for the new NSF architecture, was Program Director until March 1993, when Priscilla Huston, director of Computing Information Services at Rice University, joined the NSF staff. At that time, work had just begun on evaluating proposals for the transition to the new architecture. Huston, a past Chair of EDUCOM’s Board of Trustees, was proud of the way the NSFNET project proved the value of increasing access to the NSFNET to the research and education community at large: “There was a very practical reason for introducing networking, but once it was introduced, it became a much more powerful tool than simply in support of supercomputing. Networks began to form the basis for a larger, even international, communications infrastructure,” she recalls. NSF provided funds for the creation of the expanded NSFNET backbone service in the amount of $57.9M for the duration of the agreement, originally awarded for five years but extended for a total of seven and one-half years. Although for many years the NSFNET program was the fastest growing program in the NSF in terms of percentage increase in its budget, in comparison to other Federal R&D programs, the NSFNET backbone service, for what it accomplished, represented a relatively modest government investment: “They didn’t spend a tremendous amount of money; if you look at the budget of the United States, it was a drop in the bucket,” says Mark Knopper. The State of Michigan also played an important role in the NSFNET project. Before the solicitation was announced, Doug Van Houweling had a meeting with Jamie Kenworthy, at the time an employee at the state Department of Commerce working on the Michigan Strategic Fund. The Michigan Strategic Fund was tasked with allocating funds from a pool derived from various tax revenues in a way that would benefit the overall development of the economy of the state of Michigan. According to Van Houweling, Kenworthy specifically mentioned that the Fund was interested in initiatives that “combined support from business, education and government” that would affect the future of the state. When it became clear that IBM and MCI would partner with Merit for the proposal, Van Houweling introduced the partnership to Kenworthy and as a result, “one of the very strong parts of the proposal was a letter from Governor Blanchard, in which the Governor essentially pledged that he would use his best efforts to enable an allocation from the Michigan Strategic Fund.” The Strategic Fund contributed $5M to the NSFNET project, and, according to Van Houweling, “as it turns out, it was strategic for the state of Michigan” because of the impetus Merit provided to the development of networking and Internet connectivity throughout the state.


The Vision: Networking the Research and Education Community

The history of the NSFNET program shows that the overall design and intended uses of the NSFNET proceeded through several stages, from its original conception of a network to link researchers to the NSF- sponsored supercomputer centers all the way to a broad-based networking communications infrastructure connecting the entire research and education community. Various developments in computer science and networking technology converged with the evolution of Federal programs in computing and communications (as well as similar initiatives in other countries) in the early 1980s to create the context for the NSFNET program, of which the NSFNET backbone service itself was the centerpiece. These factors combined to create the environment and policies that produced the NSFNET and the Internet, and were an important part of the motivation leading the NSFNET partnership to initiate and respond to a project for a national research backbone network. Although the guiding vision of what such services could provide changed over time, in 1987 the NSFNET backbone service solicitation and cooperative agreement were fashioned in response to the perception of specific needs of the research and higher education community for high-speed networking. High-speed networking was originally conceptualized as a support service for scientific researchers to access high-performance computing resources at the supercomputing centers and elsewhere. In 1985, the primary needs of the scientific research community for a national high-speed network were characterized in the following terms: getting researchers access to supercomputers and large databases, and facilitating collaboration via electronic communication. In fact, to help meet those needs, NSF in 1985 gave DARPA $4M to install new ARPANET nodes at 40 colleges and universities to be selected by NSF. Steve Wolff remembers, however, that “DARPA had just turned over management and operation of the ARPANET to the Defense Communications Agency, and the bureaucracy was such that it took until 1990 to get all the nodes in place. By that time the T1 NSFNET backbone service had been in use for two years, and the connections to the 56 Kbps ARPANET were redundant. As DARPA decommissioned the ARPANET during 1990, some of its nodes were actually installed and de-installed in the same week.” In 1986, the five NSF-sponsored supercomputer centers began operation linked by the 56 Kbps NSFNET backbone. This network clearly filled a need: overwhelming demand for networking services quickly saturated the backbone, and the NSF realized that the network would have to be upgraded to support the increase in users. At the same time, however, individuals within NSF’s NCRI as well as the segments of the research and education community that had benefited from other networking research and services (from ARPANET to BITNET) were beginning to express the desire to expand the vision of high-speed networking. Jane Caviness was at the University of Delaware’s Academic Computing Services in the early 1980s, and remembers hearing the community’s needs described as a “national problem:” “People were using BITNET and it was expanding, but there were limits to what you could do with it. I was part of a group trying to articulate academic networking needs. We realized that this was a national problem that had to be addressed, and the community was organizing to express that need. A set of groups came together at that time to help get the effort started.” “NSF started to create operational TCP/IP infrastructure initially to interconnect the supercomputer centers, but very shortly thereafter the vision of the NSFNET backbone service was broadened to include the research and education clientele at large,” says Hans-Werner Braun. “In fact, shortly after Steve Wolff started at NSF he made comments to me, and I assume to others as well, saying ‘I do not want to see this network only as a supercomputing center network.'” A paper released in 1987 by EDUCOM noted that “as limited budgets force us to examine closely the means to improve research and scholarship, information technology emerges as a key tool … faculty in all disciplines have a need to use networks for communications among their peers and within research projects.”(2) Thus, while NSF staff at NCRI was primarily concerned with expanding the design and operational aspects of the NSFNET backbone at the time of the 1987 solicitation, they were already planning to extend the vision of the NSFNET program by attempting to build a more comprehensive infrastructure and by encouraging all of the higher research and education community to connect. The vision for national high-speed networking in 1987, then, contained two main elements. Within NCRI, there was a growing consensus that the best way to create a ubiquitous national networking program was to build infrastructure on many levels at once. Over time, the NSFNET program would include funds to construct, operate and maintain the NSFNET backbone network, to support the regional and midlevel networks, and to fund access to the NSFNET for individual colleges and universities. According to Steve Wolff, following this strategy was the best way to build the network in a way to benefit the most people. “The lucky thing that we did was to say why don’t we just make a cross-country transit network, and then empower the regionals? We’ll do the backbone, we can manage that, but we’ll let the regionals connect everybody else. In this way NSF empowered hundreds of people all at once.” Accordingly, the NSF’s first goal for the new NSFNET was to construct a national backbone network service, linking the supercomputer centers and the emerging regional and midlevel networks at T1 speeds. Secondly, expanding use of the network from scientists and researchers to the general academic community would also coincide with the growing understanding that in America’s universities, research and educational functions were thought to benefit one another. New networking technologies might further erase the boundaries between research and teaching, and even between teacher and student. In this environment, then, separating “research” use of a network from “educational” use would not only be technically difficult, but it didn’t seem to make much sense, not when networking the entire research and education community could improve the learning process at all levels. According to Priscilla Huston, “Very early on, both the campus communities and the government realized that it would be very difficult in the networking world to separate one thing from the other. Historically we have tended to think of teaching and education as separate from research. We have always thought there was value in the teacher being a researcher as well as a professor, but networking has facilitated a broader learning experience where students are empowered, students learn better, and may in fact make contributions themselves. In this way, networking can help to foster a more interactive learning process.” During 1987, the specific vision for the NSFNET backbone solicitation began to take shape, taking into account this expression of the community’s needs.


The Solicitation

As the currents of opinion regarding networking for the research and education community swirled around computer scientists, scientific researchers of every stripe, government policy makers, and members of the computer and communications industries, the NSF’s specific, immediate needs in creating Solicitation 87-37 centered around expanding and building upon the 1986 NSFNET. As more of the scientific research community became connected, they realized just how helpful networking could be, and this encouraged a dynamic process of innovation in networking. New applications such as NCSA Telnet were written to take advantage of the growing information resources available on what was becoming the Internet. This in turn attracted more users to the NSFNET. Moreover, NSFNET users increasingly incorporated networking into their daily lives, relying on their ability to communicate electronically in all aspects of their work. Reflecting this increasing dependence on networking, one of the basic rationales behind the creation of the NSFNET program was beginning to shift. Rather than creating and supporting a “research” network for computer scientists and networking researchers to test new networking technologies, as it had been envisioned in the early 80s, by the time of the 1987 solicitation the new NSFNET backbone was conceived to be a “production” network, meaning that the network would be built, operated and maintained at a high level of performance and reliability. According to Hans-Werner Braun, the NSF had high service expectations for the network, and did not intend the NSFNET to be used only for research: “The NSFNET backbone was really there to provide infrastructure. Some of the networking researchers did not understand even until 1988-89 that we were operating a network with very high service expectations that people really depended on. They needed to be sure that the network was always up and that networking researchers didn’t experiment with it.” The NSF also realized that in order to scale the network backbone to the level demanded by its exceptionally high growth rates, significantly more investment would be needed. Thus the solicitation itself was drafted to encourage participation by companies like DEC, AT&T, Sprint, IBM, and MCI, by not specifically excluding private companies from bidding, and by framing the technical requirements so that by meeting them, companies would gain experience and have a better competitive position in the future. Campuses spent approximately $100,000 per year per campus for connectivity to the 1986 incarnation of the ARPANET, according to EDUCOM, but other significant costs included extensive installed bases of incompatible computing systems, which often had to be adapted or replaced. The NSF knew that it would not be able to fund a new, expanded NSFNET in its entirety; hopefully, the careful administration of funds to various levels of the NSFNET (the backbone, the regionals, and campus network connections) would seed the development of the overall program, while cost-sharing arrangements with industry partners would provide the remainder of the necessary investment. The solicitation was drafted at the NSF by Steve Wolff and Dan Van Bellegham. Funding provided by NASA made it possible for Steve Goldstein, at the time an employee of MITRE Corporation, to assist by preparing an analysis of prospective costs and performance of the new backbone. One of the firm requirements of the solicitation concerned the communications protocol standard: it had to be TCP/IP, created by Vint Cerf and Bob Kahn in 1973 as a research project on behalf of ARPA. Dennis Jennings chose TCP/IP, as opposed to other protocols, for the new NSFNET backbone because it was an open and non-proprietary standard, and also because the ARPANET used TCP/IP: “One of the things that helped jump-start the NSFNET was an agreement with ARPA to allow us to use the ARPANET for NSFNET traffic,” recalls Jane Caviness. In addition to the communications protocols, appropriate hardware and software had to be specified, which was difficult at the time because there were few, if any, off-the-shelf TCP/IP products available. The fastest routers available at the time switched fewer than 1,000 packets per second, compared to today’s high-speed routers with performance levels of up to 100,000 packets per second. An advisory panel of scientists and engineers from academia and industry, including those involved in CSNET, assisted NCRI staff in preparing the solicitation. Finally, in an open meeting at the NSF’s offices on June 15, 1987, the solicitation was released. Its architects now had only to wait. For those interested in accepting the challenge put forth by the NSF, however, the long journey toward a national research and education backbone network had only begun.


Drafting the Proposal and Winning the Award

For about six weeks before the due date for proposals, a group of people from MCI, IBM, and Merit met at the University of Michigan, Merit’s administrative home, to put ideas together: Eric Aupperle, Hans-Werner Braun, Walter Wiebe, Paul Bosco, Rick Boivie, S.S. Soo, Mathew Dovens, Bob Mazza, Len Dedo, Doug Van Houweling, and Greg Marks led the proposal development team. “It was exciting stuff, hammering the proposal together. There were a lot of bright people and a lot of enthusiasm to go with it; we burned both ends of the candle on that one,” recalls Mathew Dovens. Walter Wiebe and Paul Bosco worked constantly within IBM to solicit the necessary commitment: “For most of the summer the phone was glued to my ear, begging, praising, eliciting, humbling, whatever it took to get whatever we needed from within IBM to move forward,” recalls Paul Bosco, manager of the team at IBM responsible for building routing systems, adapters, and technologies for the NSFNET backbone service. At the IBM senior executive level, too, persuasion was the order of the day. Bob Mazza, who led the NSFNET project for IBM, found himself on the phone with vacationing executives, selling them on the proposal. “We were in a period of major changes and cost-cutting, so it was a particularly tough time to get an agreement,” he recalls. But IBM had a long history of interest in supporting the academic community, with influential joint studies already in place with MIT, Carnegie-Mellon University, and the Cornell Supercomputing Center. “The NSFNET backbone service was an ideal adjunct to those projects,” says Mazza. At the same time that the partners-to-be had definite ideas about how to respond to the NSF solicitation, making all of the pieces fit properly into a cohesive, integrated design for a backbone network service required them to break new conceptual ground. There were three main components of the basic architecture in the proposal: the packet-switching nodes, called the Nodal Switching Subsystem, the circuit switches interconnecting these nodes, called the Wide Area Communications Subsystem, and a network management system. In addition to overall engineering, management, and operation of the project, Merit would be responsible for developing user support and information services. IBM would provide the hardware and software for the packet-switching network and network management, while MCI would provide the transmission circuits for the NSFNET backbone, including reduced tariffs for that service. A team from IBM, MCI, Merit, and the University of Michigan labored for many late nights to write the proposal, which was completed and submitted to the NSF on August 14, 1987. As the NSF reviewed the proposals, the team members had no idea if they would win. Through the fall of 1987, however, the partners did not rest while waiting for the award announcement. Anticipating the NSF’s desire for more concrete examples of the potential of the Merit proposal, IBM built a simulation network to test some of the hardware and software systems to be used in the NSFNET backbone service. The partners also worked on development of hardware and software systems for the packet switches to be located at each node, as well as testing of the proposed fourteen MCI T1 circuits. On November 24, 1987, the NSF announced that they had a winner: Merit and its industry partners, together with the State of Michigan’s additional funds, would build a new NSFNET backbone. The new Cooperative Agreement NSF 872-0904 was announced in a press conference convened at Wayne State University, one of Merit’s member campuses, and a press breakfast in Washington D.C. for members of Congress, both of which were attended by representatives of the new NSFNET project team. At that time, networking still represented barely a blip on the radar screen of most news media; those members of the press that could be cajoled to come received something of a crash course on “internetworking,” an introductory lesson to be repeated to ever-expanding groups of people over the course of the award. Today, of course, the words “information superhighway” make front-page news in national newspapers and magazines. Back in 1987, however, Merit and its partners were charged with planting the seeds of that future.