High Performance Computers
Summary
High Performance Computers (HPCs) are important for many military applications and essential for some. Although there is limited information on how the PRC is using HPCs for military applications, HPCs could facilitate many of the PRC’s military modernization objectives.
PRC organizations involved in the research and development of missiles, spacecraft, submarines, aircraft, military system components, command and control, communications, and microwave and laser sensors have obtained HPCs from the United States. Given the lack of a proven and effective verification regime, it is possible that these HPCs have been diverted for unauthorized uses, which could include the following:
To realize the full potential of the acquired HPCs, the PRC must be able to perform system integration, develop or procure application software, obtain weapon systems test data, and institute quality-controlled production processes. The contribution of HPCs to military modernization is also dependent on related technologies such as telecommunications and microelectronics.
The Select Committee judges that the PRC has been using high performance computers for nuclear weapons applications. The computer workstations recently acquired from the U.S. represent a major increase in the PRC’s computing power. Although not necessary to design nuclear warheads, HPCs of 2,000 million theoretical operations per second (MTOPS) or more can be used for such applications. In addition to nuclear weapons design, another major concern is how the PRC can use U.S. HPCs to improve and maintain its nuclear weapons.
If the PRC complies with the Comprehensive Test Ban Treaty, then its need for HPCs to design, weaponize, deploy, and maintain nuclear weapons will be greater than that of any other nation, according to the U.S. Department of Energy. The exact extent to which HPCs can assist the PRC depends in part on the goals of the PRC nuclear weapons program and the degree of uncertainty it is willing to accept in warhead performance.
HPCs are useful to the two- and critical to the three-dimensional computer modeling that is necessary for the PRC to develop, modify, and maintain its nuclear weapons in the absence of testing. The utility of such modeling depends on the amount of data available from tests, the computing capacity that is available, and programmer expertise. Complete three-dimensional models, critical to stockpile maintenance and assessment of the effect of major warhead modifications in the absence of testing, require HPCs of one million MTOPS or more. Assessing the effects of a new warhead without testing would require three-dimensional modeling. In the absence of physical testing, two dimensional models are important for estimating the effects of less substantial changes to warhead designs, although the utility of such modeling decreases as the designs become more sophisticated. However, the fidelity of any two-dimensional model is inherently limited, and some level of uncertainty will always remain. Should the PRC resume physical (rather than virtual) nuclear testing, the resulting data would permit more accurate two-dimensional modeling of subsequent design changes. Although HPCs in the 2,000 to 10,000 MTOPS range are useful for such modeling, their precise utility for such applications is unclear. These HPCs may be powerful enough to help the PRC make use of design information that it stole from the United States, including design information for the W-70 neutron bomb and the W-88 Trident D-5 thermonuclear warhead — without further physical testing.
The U.S. Government, citing rapid advances in computer technology, has steadily relaxed export controls on HPCs. A Stanford University study commissioned by the U.S. Government was a key element in the relaxation of export controls on HPCs in 1996. The study concluded that U.S.-manufactured computer technology between 4,000 to 5,000 MTOPS was uncontrollable worldwide and would become available worldwide at 7,000 MTOPS by 1997. The study also concluded that many HPC applications used in U.S. national security programs occur at about 7,000 MTOPS and at or above 10,000 MTOPS. Criticisms of this and other studies that were used to justify the 1996 HPC export control policy changes focus on flaws in the methodology of the studies and the lack of empirical evidence and analysis to support their conclusions. These critics also claim that the U.S. Government revised the export controls on HPCs without having adequate information on how countries of concern would use HPCs for military and proliferation activities.
Until June 1998, the U.S. Government’s ability to verify the location and use of HPCs in the PRC was blocked by the PRC’s resistance to post-shipment, on-site verification visits. A new agreement affords the U.S. Government the right to request access to some American HPCs, but includes substantial limitations on such requests and any visits. Moreover, the post-shipment visits that are allowed can verify the location of an HPC, but not how it is used.
Rapid advances in computer technology have altered traditional concepts of what constitutes an HPC. Observers in the computer industry and academia state that HPC-level performance can be obtained by linking together inexpensive commodity processors. For some applications the efficiency and effectiveness of the linked commodity processors depends on the application, skill of the programmer, and interconnection software. The resources and time needed to effectively modify and operate significant defense applications for such linked systems have not yet been demonstrated. Nonetheless, the U.S. is pursuing research and development on the use of linked systems for three-dimensional modeling for nuclear stockpile maintenance.
While it is difficult to ascertain the full measure of HPC resources that have been made available to the PRC from all sources, available data indicates that U.S. HPCs dominate the market in the PRC and there really is no domestic PRC HPC industry. While the PRC has a large market for workstations and high-end servers, there is a smaller market for parallel computers that is entirely dominated by non-PRC companies such as IBM, Silicon Graphics/Cray, and the Japanese NEC. However, there continues to be significant market resistance to Japanese HPC products in Asia, especially as U.S. products are beginning to have significant market penetration. The PRC has assembled several HPCs in recent years, using U.S.-origin microprocessing chips. The latest such HPC may perform at 10,000 MTOPS. However, the PRC’s HPC application software lags farther behind world levels than its HPC systems.
Since the 1996 relaxation of U.S. export controls on HPCs, U.S. sales of HPCs between 2,000 and 7,000 MTOPS to the PRC have burgeoned. Of computers not requiring licenses under the 1996 regulations, 23 HPCs in this performance range were exported in 1996 and 123 in 1997. An additional 434 HPCs were to be exported in the first three quarters of 1998. Between 1994 and 1998, the U.S. Government approved licenses for 23 HPCs greater than 2,000 MTOPS.
Thus, the PRC may have received a total of 603 U.S. HPCs since 1996. In 1998, the United States approved licenses for two HPCs in excess of 10,000 MTOPS. Approximately 77 percent of the U.S. HPCs that have been exported to the PRC were under 4,000 MTOPS.
The aggregate of these computational resources is complemented by millions of non-export controlled low-end machines – about 4.5 million desktops, portable personal computers, personal computer servers, and workstations in 1998 alone. 90 percent of these machines are being used by the PRC Government, industry, and educational institutions. About 60 percent of these machines are being produced by PRC companies.
High Performance Computers
High Performance Computers (HPCs) are useful in a broad range of applications. These include pharmaceutical development, automobile crash modeling, aerospace engineering, petrochemical research, financial market and credit analysis, weather prediction, academic research, and national security applications.
A recent report by the Defense Department defines high performance computers as:
the mid-range of the speed scale. These computers are used for internet servers, Local Area Network (LAN) servers, affordable number crunchers, Computer Aided Design (CAD)/Computer Aided Manufacturing (CAM), publishing, billing, databases, data mining, banking, and much more. Presently these computers are in the speed range of 1500 — 40,000 Millions of Theoretical Operations Per Second (MTOPS).1
Current U.S. export controls define HPCs by establishing the threshold for license consideration at 2,000 or more MTOPS.
In the realm of national security, HPCs are valuable in the design, development, manufacturing, performance, and testing of weapons and weapons platforms. These systems include:
HPCs are also useful in the collection, processing, analysis, and dissemination of intelligence and in the encryption or decryption of communications.2
In addition, military applications such as target tracking and recognition, radar mapping, armor and anti-armor design, protective structures, aerodynamics, real-time modeling, and tactical weather prediction are substantially facilitated by the use of HPCs.3
While a broad array of potential applications for HPCs is known, the specific ways in which potential adversaries of the United States are using them is much harder to determine. For example, a 1998 study of the viability of U.S. export controls on HPCs stated:
It is difficult to acquire good information on the use of HPC[s] for national security-related applications by countries of national security concern. This is true whether one assumes foreign practice is the same as U.S. practice, or foreign practice involves different or more clever ways that might not have the same computing requirements.4
In short, there is limited information about how specific countries of national security concern, including the PRC, use HPCs.5
Another complicating factor in determining whether and how HPCs are being used by the PRC and others for national security applications is ambiguity as to the HPC performance minimally required for specific applications. Researchers are usually interested in improving their applications if they have access to more computing power. Therefore, the "bigger and faster" computers are, the better. Speed helps make optimum use of a researcher’s time.6 Many computer programs can be executed on less capable computer hardware, although there may be penalties in level of detail and turnaround time.7
The requirement to use the most powerful computers available may also be closely related to program economics.8 The use of less powerful computers leads to longer processing runs. This situation leaves expensive people and facilities idle, making the purchase of an expensive HPC necessary to employ all the resources available efficiently.9
There are many potential national security applications for which the PRC could use HPCs. The following figure10 shows that the U.S. defense community uses HPCs for national security applications over a full range of MTOPS performance levels. Although nearly 44 percent of the applications currently being run in the U.S. defense community are being run at performance levels below 7,000 MTOPS, many critical applications require processing power in excess of that threshold. The relative importance of the national security applications cannot be ascertained based on the MTOPS requirement.11 As newer computer systems with increased performance become available to the market, an increasing number of applications will appear in the higher MTOPS range (that is, above 30,000 MTOPS).12 These applications will be similar to current applications, but will require greater resolution or ability to address larger-sized problems than is possible on current systems.13
Millions of Theoretical Operations Number of Current U.S. Department of Defense
per Second (MTOPS) Range HPC Applications
Less than 2,000 61
2,000 to 7,000 24
7,000 to 10,000 8
10,000 to 20,000 30
20,000 to 30,000 47
Above 30,000 22
U.S. HPCs recently sold to PRC organizations are useful for a number of military purposes, including:
These PRC organizations are engaged in governmental, military, academic, and commercial work. In the absence of an end-use verification regime, the United States has no means of determining to what use a particular HPC is applied by such PRC entities.
Military Objectives Contribute to the PRC’s
Interest in High Performance Computers
PRC military objectives require superior battlefield management, including:
They also require superior weapons and platform design, testing, and maintenance. Satisfying these requirements can be facilitated by HPC capabilities.14
The PRC is seeking HPC software for:
The PRC is convinced that the United States has the most advanced HPC technology. Thus, it seeks to acquire as much of it as it can without jeopardizing PRC national security interests by, for example, becoming susceptible to computer viruses and information attacks.16
The specific ways the PRC is using HPCs for military applications is difficult to determine.17 During this investigation, reports regarding the PRC’s military objectives, information concerning the application of HPCs in support of national security objectives, and data concerning HPC sales to the PRC were analyzed.
The results of this analysis provide a basis for assessing the risk to U.S. national security and regional security interests that accrues from the PRC’s acquisition of HPCs. This assessment is summarized in the following paragraphs.
U.S. High Performance Computers Have the Greatest Potential Impact On the PRC’s Nuclear Weapons Capabilities
The Department of Energy judges that the PRC’s acquisition and application of HPCs to nuclear weapons development have the greatest potential impact on the PRC’s nuclear program. This is particularly true since the PRC has agreed to the ban on nuclear testing.18
Existing PRC Nuclear Weapons
The computing power required to simulate the performance of a specific nuclear weapon depends on the sophistication of the design, and the availability of nuclear and non-nuclear test data for the new and aging materials the weapon contains.19
For existing weapons with supporting test data, more powerful computing resources allow simulations that include more physical processes and more fundamental representations.20
One means of enhancing model fidelity — the extent to which the model accurately represents the real phenomena — is to represent all dimensions of the process being modeled.
The explosion of a nuclear weapon is a three-dimensional process that cannot be accurately represented in one or two dimensions. Augmenting model fidelity by shifting from two to three dimensions requires an increase in computer performance capacity to one million MTOPS.21
Results from higher-fidelity models allow scientists and decision-makers to develop a better estimation and understanding of the reliability and performance of the weapon.22
Another factor bearing on model fidelity and confidence in model results is the extent to which the model has been validated. Validation consists of running a simulation of a previously conducted test, and verifying that the computed results are close to the test results. The more the simulated situation differs from the actual test, the less confidence can be placed in the computed results.23
The fewer the tests that have been conducted, the more gaps there are in the understanding of nuclear weapons science.24
HPCs may help scientists gain insight and understanding by allowing many simulation runs to be conducted, changing one variable value at a time to create a range of solutions for comparison to test data. HPCs allow those calculations to be completed in an acceptable length of time.25
The following table illustrates HPC performance demand as a function of model complexity, test data, and weapons maturity. Row 1 of the table focuses on a full exploration of the weapons design category with data from tests of pristine and aged weapons. Row 2 of the table assumes the number of tests dedicated to each warhead class is between one and six. Row 3 assumes few proof-of-concept tests or zero nuclear tests conducted of the design after components have aged for ten years.
High Performance Computer Requirements for Various Levels of Testing and Nuclear Weapons Program Maturity26
|
Rudimentary Nuclear Weapons |
Intermediate Nuclear Weapons |
Advanced and Aging Nuclear Weapons | |
|
With Test Data |
200-400 MTOPS US, UK, France, Russia, PRC |
200-400 MTOPS US, UK, France, Russia |
400-10,000 MTOPS US, UK, France, Russia, PRC* |
|
With Some Test Data |
400-1,000 MTOPS No country |
1,000-4,000 MTOPS PRC, India, Pakistan |
4,000- 1,000,000 MTOPS PRC† |
|
Without Test Data |
400-4,000 MTOPS North Korea |
4,000-10,000 MTOPS Israel |
>1,000,000 MTOPS PRC† |
*If PRC has obtained U.S. or Russian nuclear test codes.
†The PRC is known to possess some test data for certain advanced nuclear weapons, but may be without test data for others.
As the table indicates, the PRC’s demand for HPCs covers a broad range of computing capability, and it is unclear where the PRC’s requirements fall within that broad range.
To date, the most powerful HPCs exported to the PRC from the U.S. — two in 1998 — have been at the 10,000 MTOPS level.
Even HPCs in the 2,000 to 10,000 MTOPS range are useful for nuclear weapons applications, although their precise utility is dependent on the amount of test data the PRC possesses.
New PRC Nuclear Weapons
The PRC’s nuclear weapons program has advanced rapidly, largely through the theft of U.S. nuclear weapons design information.
Originally, the PRC built large, heavy nuclear weapons for air or missile delivery. The PRC is now moving to new generation nuclear weapons, and has been significantly assisted by the theft of U.S. design data. These new nuclear weapons are smaller, lighter, and have higher yield-to-weight ratios.27 The Select Committee judges that the PRC has the infrastructure and ability to use the stolen U.S. design information to emulate elements of U.S. thermonuclear warheads for its next generation of thermonuclear warheads.
HPCs could be valuable to the PRC in connection with the production of these next generation nuclear weapons based on elements of U.S. design information, because they would enable scientists to examine many values for many uncertainties quickly.28
Similarly, HPCs could be useful in connection with maintaining the current PRC nuclear weapons stockpile for which test data exist, although the exact MTOPS range needed is uncertain. HPCs would permit analysis of any uncertainty with respect to the performance of these weapons.29
In addition, as military missions evolve and delivery platforms develop, the PRC may be forced to make modifications in tested designs to accommodate new size and weight goals. For example, a PRC focus on small-scale regional conflict would suggest the development of compact, low-yield nuclear devices. Evaluating the effects of these design changes would require sophisticated computer models run on HPCs. If the changes to the PLA’s nuclear weapons are significant, the need for modeling accuracy would require three-dimensional testing, possible only with computers that have a performance capability of a million MTOPS or more. For less extensive changes, including any changes required to weaponize new nuclear warhead designs that the PRC has already successfully tested, two-dimensional modeling may be sufficient. HPCs as low as 2,000 to 7,000 MTOPS are helpful in such applications, although the optimal MTOPS level required for such modeling is unclear.
Nuclear Weapons Stockpile Stewardship
Assuming compliance with the Comprehensive Test Ban Treaty, the designers of new or modified PRC nuclear devices will have to certify the performance of aging weapons by using a combination of treaty compliant experiments and computer simulations.30
Identifying, predicting and mitigating the effects of aging on nuclear weapons is computationally intensive, requiring three-dimensional modeling and simulation involving many uncertainties. For the PRC, the computing demands are even greater because of the limited amount of nuclear test data to support the modeling.
Thus, HPCs at high MTOPS levels would be particularly useful in helping explore many values for many variables quickly.31 As the United States is finding with its Stockpile Stewardship Program, maximum HPC performance in the range of millions of MTOPS is necessary for three-dimensional modeling of the aging of nuclear weapons.
For this reason, the Select Committee judges that the PRC is almost certain to use U.S. HPCs to perform nuclear weapons applications. Moreover, the PRC continues to seek HPCs and the related computer programs (known as codes) for these programs.
The U.S. national weapons laboratories are currently modernizing their test data or "legacy codes" based on data from the large number of U.S. tests. The Select Committee judges that if the PRC were to acquire nuclear test codes and data from the United States, then the PRC could access empirical data from the large number of U.S. tests that were conducted before the Comprehensive Test Ban Treaty.
The possession of stolen U.S. test data would greatly reduce the level of HPC performance required.
It is also likely that the PRC seeks access to the Los Alamos National Laboratory-based Dual Axis Radiographic Hydrodynamic Test Facility, for the reason that it uses powerful X-rays to analyze the effects of implosions during non-nuclear tests.
The PRC is also likely to seek information regarding the use of lasers for high energy density studies.
Transfer of HPC Technology Can Benefit PRC Intelligence Capabilities
The PRC is improving its capabilities in intelligence collection and unmanned aerial vehicles. The PRC is also a user of encryption technology in its government networks.32 HPCs are useful in the design and operation of intelligence collection platforms, including unmanned aerial vehicles, and are essential to running the computer codes that process intelligence data and perform encryption tasks.
Sensors for Surveillance, Target Detection, and Target Recognition
Radars, acoustic and non-acoustic sensors, and signal and image processing appear to be continuing targets for acquisition by the PRC.33
Based on U.S. experience, HPCs can be used to facilitate research and development of sensors for surveillance, target detection, and target recognition. Use of HPCs in this manner results in sensor systems that are more capable of detecting stealthy platforms, such as aircraft, missiles, and submarines.34
In the design phase, these applications can be computationally intensive, depending upon the level of realism required. For example, U.S. computational requirements range from 500 to over 40,000 MTOPS.35
Also, many of the resultant systems require HPCs and advanced software for their operation. For example, a deployed X-band phased-array radar for ballistic missile search, fire control, and kill assessment requires an HPC to control the radar, detect, identify, and track targets, and compute fire control solutions of multiple high-speed targets.36
In general, timely detection of targets using radar requires homogeneous, tightly coupled systems. The radar system functions by creating images of remote objects and processing the resulting images for review by humans or input into automated guidance or decision support systems. This operation is computationally intensive since large volumes of data must be filtered, enhanced, and interpreted, often in real time.37
In the United States, some radar processing applications — for example, the processing of data from synthetic aperture radars — require 32,000 to 115,000 MTOPS.38 Although less capable computers may be useful for these applications, they are not suitable for operational environments that require real-time detection of targets with weak radar signatures, or target discrimination in high target-density environments. Further, radar system performance requires high-quality target templates and empirical validation, in addition to HPC processing speed.39
Sensor Platforms for Aerial and Space-Based Reconnaissance
The PRC is interested in acquiring unmanned aerial vehicles (UAVs) that are used for day/night aerial reconnaissance, battlefield surveillance, target positioning, artillery spotting, border patrol, nuclear radiation sampling, and aerial photography.40
The HPC challenge is to provide a sufficient on-board-sensor data processing capability to allow wide-area searches at high resolution, while minimizing communications requirements.41
Satisfying such sensor data processing requirements could also be of value to the PRC’s efforts to improve space-based information gathering capabilities.42
Cryptology
Another potential application of HPCs by the PRC is cryptology — the design and breaking of encoded communications. This application demands fast processing, and the ability to handle large amounts of data. As a point of reference, the U.S. National Security Agency uses some of the highest performance computers available. However, significant cryptologic capabilities can be achieved through the use of widely available computer equipment, such as networked workstations or parallel processors.43
Transfer of High Performance Computer Technology To the PRC Could Contribute to the Manufacture of Weapons of Mass Destruction, Missiles, and Other Weapons
While there is little information regarding the specific ways that HPCs are being used in the PRC to achieve military objectives,44 open source reporting and stated PRC military modernization goals tend to support the belief that the PRC could be using HPCs in the design, development, and operation of missiles, anti-armor weapons, chemical and biological weapons, and information warfare technologies.45
Missiles
The PRC is developing advanced cruise missiles, anti-ship missiles, and conventional short-range ballistic missiles (SRBMs).
While the PRC could design, for example, a stealthy cruise missile without using HPCs, HPCs facilitate the design of such weapons, particularly in exploring guidance and stealth concepts.46 For instance, the Beijing Simulation Center is using hardware-in-the-loop testing in the development of homing guided missiles. Given that such testing involves near real-time processing, HPCs are particularly useful.47
The PRC is also developing new tanks, and new multiple-launch rocket systems. HPCs are useful for executing the detailed, physics-based simulations of weapons effects. Such simulations are useful in assessing the effectiveness and vulnerabilities of these new systems. The calculations are complex, and HPCs are required for efficient processing.48
Chemical and Biological Weapons
The PRC has mature chemical and biological weapons programs that have produced a variety of chemical and biological agents since the 1960s. Such weapons could serve deterrent, retaliatory, or offensive purposes.49
Computer-aided design and computer-aided manufacturing (CAD/CAM), a classic use of HPCs, would be useful in planning and designing the integration of chemical warfare agent development processes with chemical industries.50 This possibility is consistent with papers published by PRC scientists concerning chemical and manufacturing processes.51
The PRC can deliver chemical and biological agents with a variety of weapons systems, including missiles and artillery. Since the PRC can employ a variety of delivery means for such agents, key operational considerations for the PRC include how dispersion patterns vary as a function of delivery method and weather. This is a computationally demanding area in which HPCs are extremely useful.
The Select Committee concludes from evidence it has received that the PRC is interested in HPC modeling of dispersion patterns of chemical and biological weapons based on different weapons delivery systems and varying weather conditions.52 In addition, the PRC could be employing HPCs to model the negative effects on the opponent of casualties, and of cumbersome protective gear for a given dispersion pattern of chemical and biological weapons.53
Finally, the PRC may also be using HPCs to design chemical agent detection sensors and protective measures. Such applications can require computational power ranging from 2,000 to 30,000 MTOPS.54
Information Warfare
Several PRC scholars and leading military strategists indicate that the PRC has an ambitious, albeit nascent, offensive information warfare program. Currently, the PRC’s primary focus for information warfare is military conflict. Concluding that information is becoming a key determinant of military power and victory in war, the PRC has identified the development of information warfare capabilities as a key modernization goal of the PLA.
The PRC should . . . fully bring into play the guiding role of information warfare research in building the military and seek measures by which to launch vital strikes in future warfare, so as to damage the enemy’s intelligence gathering and transmission abilities, and weaken the enemy’s information warfare capacity.55
HPCs could prove valuable to the PRC in the evolution of this strategy by exploring U.S. information networks and their vulnerabilities, and the technologies that are associated with information warfare such as jammers, microwave weapons, and anti-satellite weapons.56
Transfer of High Performance Computer Technology To the PRC Could Support Attainment of Other PRC Military Objectives
The effectiveness of military operations depends heavily on support functions that include:
HPCs can be used to enhance all of these functions.
In military operations, size, weight, and power consumption limitations are all stressing requirements that may necessitate the use of customized or embedded HPCs, rather than commercially available systems.58
Command, Control, and Communications
Leading PRC military strategists and political/military scholars in the PRC have publicly recommended that the PLA give high priority to the development of improved automated command, control, and communications networks.59
The recommendations include:
Another PRC writer has stated that multi-dimensional interconnected networks on the ground, in the air (and outer space), and underwater — as well as terminals, modems, and software — are not only instruments, but also weapons.61
The PLA has begun research on the technologies necessary to develop an Integrated Battlefield Area Communications System.62 In addition, research is underway on related subjects such as real-time intelligent decision-making for fighter aircraft maneuver simulation systems.63
Full implementation of these goals will require exceptional computational power. However, this power can be efficiently provided by distributed computer systems.64 Battle management functions are also readily scalable, making them suitable for initial implementation on commercially available computer equipment.
Meteorology for Military Operations
Weather modeling and prediction is essential in military operations in that it effects force deployments, protection against chemical, biological, and nuclear environments, weapons effectiveness, and logistics.65
While a typical global weather model with 75-mile resolution can be executed on a workstation with performance in the 200 MTOPS range, typical tactical weather models with 30-mile resolution require computers rated in excess of 10,000 MTOPS. Calculation of weather forecasts in littoral areas to resolve complex air-ocean interactions is even more demanding.66
Cartography for Military Operations
Depending on the perceived requirements of military commanders, cartography requires high computational levels. For instance, processing topographic data in a timely manner to support military operations may require up to 24,000 MTOPS. For military planning purposes in which time is not a factor, cartographic applications can be accomplished at lower MTOPS levels — less than 4,600 MTOPS — and computer hardware can be selected based on cost rather than speed and memory capacity.67
Military Training Systems
Research underway at the PRC’s Harbin Institute of Technology indicates the PRC is focused on large-scale training systems.68 The computer performance requirements in this regard depend on the level of fidelity that is needed, the complexity of the training objectives, and the time that is available. For training objectives that require realism and representation of large-scale forces, HPC performance may exceed 10,000 MTOPS.69
National Security Implications of High Performance Computer Use by the PRC Military
The Select Committee judges that the PRC is attempting to achieve parity with U.S. systems and capabilities through its military modernization efforts. The PRC intends by this effort to increase its regional power projection capabilities and augment its ability to hold the neighboring countries of Taiwan, India, and Japan at risk.
The PRC’s use of HPCs for its military modernization poses risks to U.S. national security. Significant improvements in PRC information warfare and military operations may increase the threat to U.S. military systems and personnel in a way that cannot be easily countered.70 HPCs of varying capability could assist the PRC in this endeavor.71
Further, the PRC is likely to modernize its nuclear arsenal, with the help of HPCs. In this regard, it is believed that, if the PRC maintains its current path, it will still be a second-class nuclear power compared to the United States and Russia for the next several decades. However, if Washington and Moscow were to reduce their nuclear forces to about 1,000 warheads, as President Yeltsin has suggested, the PRC could conceivably expand its nuclear forces in an attempt to reach numerical parity.
The PRC’s continuing chemical and biological weapons programs, and improvement of weapons delivery platforms such as cruise missiles, may also be the beneficiaries of increased HPC capability. Continued development or use of chemical or biological weapons by the PRC could have serious strategic and tactical implications for the United States.72
If it is to fully exploit HPC hardware capabilities for military applications, the PRC requires improved system integration, quality production processes, and development of doctrine and tactics.73 The PRC also requires technologies that are interdependent with HPCs in military applications, such as telecommunications and microelectronics.
Control or monitoring of these HPC-related services and technologies may provide additional opportunities to influence the pace of the PRC’s attainment of its military modernization objectives.
U.S. Export Policy Has Gradually Relaxed Controls on High Performance Computers
In 1988, exporters of HPCs were required to obtain a Department of Commerce license to export computers with a performance level — called a Composite Theoretical Performance (CTP) — of 12.5 MTOPS or more to most destinations. A supercomputer was defined as any computer with a performance level of 195 MTOPS or greater.74
Foreign policy controls were imposed on supercomputers performing at 195 MTOPS and higher in May 1992, based on a bilateral arrangement with Japan, the other major supercomputer-exporting country.75
As required by the Export Enhancement Act of 1992, the Trade Promotion Coordinating Committee submitted to Congress a report entitled "Toward a National Strategy" in September 1993.76 That report presented a strategic plan that included as one key element changing the standard for a supercomputer from 195 MTOPS to 2,000 MTOPS.77
In February 1994, the Department of Commerce raised the licensing threshold for the export of supercomputers to most destinations from 195 MTOPS to 1,500 MTOPS or higher. At the same time, the United States announced that it had reached agreement with Japan, the other partner in the "supercomputer regime," regarding the new supercomputer definition of 1,500 MTOPS. The United States also announced that it would continue to seek Japan’s agreement to further increase the supercomputer threshold to 2,000 MTOPS.78
In April 1994, the Department of Commerce established a new General License "GLX," which would allow certain shipments of any items, including computers up to 1,000 MTOPS that formerly required an individual validated license, to civil end users and nonproliferation end uses in formerly proscribed destinations, including the PRC. The purpose of the new general license was to reduce paperwork and licensing delays for exporters, while focusing controls on exports of "direct strategic concern." The Department of Commerce stated that it established the "GLX" designation to bridge the transition between the termination of COCOM in March 1994 and the establishment of a successor regime.79
In January 1995, the Department of Commerce again revised certain supercomputer requirements. Specifically, Commerce noted that it would conduct annual reviews of the supercomputer definition, threshold levels, safeguards, supercomputer country groupings, and supercomputer licensing requirements. The reviews would examine HPC controls in light of national security and proliferation concerns, technical advancements, and changes in market conditions, and would consider recommendations to revise the controls. The regulations included the following country requirements:
Some Reviews That Contributed to High Performance Computer Policy Changes in 1996 Have Been Criticized
On January 25, 1996, after the first periodic review, the Department of Commerce published revised controls for computers in the Export Administration Regulations and identified four computer country groups for export purposes. In announcing the January 1996 revision, the Executive branch stated that one goal of the changes was to permit the government to calibrate control levels and licensing conditions to the national security or proliferation risk posed at a specific destination.81
The Stanford Study
A key element of the 1995 Executive branch review of HPC export controls was a Stanford University study that was commissioned jointly by the Commerce and Defense Departments.82 Among other things, the study was tasked to assess the availability of HPCs in selected countries, and the capabilities of those countries to use HPCs for military and other defense applications.83 The study, released in November 1995, concluded:
The study also concluded that it would be too expensive for the U.S. Government and industry to maintain the effective control of computing systems with performance levels below 7,000 MTOPS. Further, the study stated that attempts to control computer exports below this level would become increasingly ineffectual, would harm the credibility of export controls, and would unreasonably burden a vital sector of the computer industry. The study also raised concerns about the ability of the U.S. Government to control HPC exports in the future, in light of advances in computing technology and its dispersal worldwide.85
However, the Stanford study had several methodological limitations. It lacked empirical evidence or analysis to support its conclusion that HPCs were "uncontrollable" given both worldwide availability and insufficient resources to control them. Neither the study nor the U.S. Government made estimates of these resources. Also, the study did not assess the capabilities of countries of concern to use HPCs for designated military and proliferation applications, even though that was required by the tasking.86
Seymour Goodman, one of the authors of the 1995 Stanford study, acknowledged that U.S. Government data was inadequate to make this assessment and the study recommended that better data be gathered.87 Furthermore, the study noted that data used from the High Performance Computing Modernization Office was not optimal for the study’s purposes, although it stated that the data was sufficient to "conjecture" that the majority of national security applications were already possible at uncontrollable levels. Also, the study stated that time constraints did not allow a comprehensive review of defense applications.88
In addition to the Stanford study, Executive branch officials have said that they also relied on other analytical products as part of the HPC review process.89 These included:
Some officials also referred to two 1995 Commerce Department studies on the worldwide "supercomputer" market and technology trends. These documents supported the conclusion that foreign availability of HPCs, especially in countries of proliferation concern, was limited in 1995, but that technology trends would make HPC technology more readily available throughout the world in the future.94 As a result, it appeared that denying HPC access to proliferating countries in the next century would become increasingly difficult, and perhaps impossible.
Another factor that may have figured in the decision to relax HPC export controls is that the National Security Agency (NSA) — which had been quite active in the past in HPC controls, including reviewing Commerce license applications to the Commerce Department for exports of HPCs — changed its approach. Around 1993, the NSA began to ease its involvement in computer export controls. By 1995, NSA had moved away from its activities in the supercomputer area, and had backed out of the high performance computer export control debate entirely.
The stated justification for this change in policy was concern for the health of the U.S. computer industry and the industry’s need for exports.
Defense Department Review of Military Applications for HPCs
Pentagon officials advised the General Accounting Office that there was no document that summarized the results of the Department of Defense review of military applications for HPCs.95 One Defense Department official stated that these results were incorporated into the Stanford study.96 An August 24, 1995 Defense Technology Security Administration (DTSA) memorandum summarized some general points of a Defense Department "Supercomputing Study" that reviewed military applications.97 The DTSA memorandum concluded:
Institute of Defense Analyses Technical Assessment
An IDA technical assessment reported that a consensus of computing experts, supported by available data, believed that supercomputing restrictions for systems above 10,000 MTOPS, but below about 20,000 MTOPS, could be circumvented to some extent by aggregating lower performance processors. However, the IDA assessment stated that it was difficult to go beyond this level as of 1996, except for a small set of "embarrassingly parallel problems" — that is, problems that could easily be broken up into parts that could be solved simultaneously.99
The assessment predicted that, by 1996, users should be able to interconnect systems with a total of 40,000 to 80,000 MTOPS. Such a configuration could be programmed, according to IDA, "to yield computational capabilities approximating that of a single 20,000 CTP computer for a given problem or constrained set of problems." A user may achieve this by investing from six months to a year’s worth of effort, although the resulting system would be neither user-friendly nor economically competitive in the world market.100
The IDA assessment also stated that the security risk would depend on whether there are militarily critical problems that demand high performance computing capability between 10,000 and 20,000 MTOPS, and that cannot be attacked for some technical reason by aggregation. If such problems exist, IDA advised, the issue would become how much benefit to U.S. national security it is to delay or degrade a foreign entity’s ability to achieve certain results for a given class of problems. The IDA study concluded that a user faced with limited computing power would simply run the program for a longer period of time or run it with coarser granularity.101 (Granularity of an application refers to the amount of computation relative to the amount of movement of data between processors.102 When this relationship becomes a processing bottleneck in the interconnect between processors, problems that are more easily broken up into parts — that is, "coarsely grained" — are those that can be run effectively.103)
The IDA assessment and its sponsors, Dr. Joseph Golden, Director of Multinational Technology Programs in the Office of the Deputy Undersecretary of Defense for International and Commercial Programs, and Norman Jorstad, Director of IDA’s Technology Identification and Analysis Center, provided only minimal support and documentation (four articles) for the study’s conclusions.104
As IDA officials subsequently explained to General Accounting Office, IDA had assembled a group of specialists from the U.S. Government and the computer industry who discussed the issues and produced the report following a series of meetings. While the specialists might have assembled documentation, IDA retained none of it.105
A Defense Technology Security Administration official commented in July 1998 that the agency had concerns about the study.106
Defense Department Proliferation Criteria
In the 1995 effort to develop the country tier system, Defense Department officials assessed countries for the HPC export control review process on the basis of six criteria and assigned each country to a particular HPC country tier.107 Part of the information used in this process was a ranking of each country in the world by the level of risk associated with that country’s proliferation record. The PRC was ranked at the highest level of risk.
Former Deputy Assistant Secretary of Defense for Counterproliferation Policy, Mitchel Wallerstein, explained to the General Accounting Office that the Defense Department did not conduct a threat assessment regarding HPCs because it was not tasked do so.108 Wallerstein later said that he had consulted with a counterpart on the Joint Staff about the risk associated with the levels of HPC being considered for export, and that, while the Joint Staff had concerns, the risk was considered reasonable.109
The six assessment criteria used by the Defense Department to create HPC country tiers were:
Details of the 1996 High Performance Computer Export Control Policy Changes
The export control policy announced in October 1995 and implemented in January 1996 removed license requirements for most HPC exports with performance levels up to 2,000 MTOPS.111
The policy also organized countries into four "computer tiers." Tier 1 represents the lowest level of concern to U.S. security interests, and each subsequent tier represents a higher level.112
The revised HPC policy was applied as follows:
The U.S. Government continues to implement the Enhanced Proliferation Control Initiative, which seeks to block exports of computers of any level in cases involving exports to end uses or end users of proliferation concern, or risks of diversion to proliferation activities.114 Criminal as well as civil penalties apply to violators of the Initiative.115
Export Administration Act Provisions and Export Administration Regulations Currently Applicable to High Performance Computers
Specific provisions of the Export Administration Act of 1979, as amended, and the Export Administration Regulations apply to HPCs. In addition, Export Administration Regulations that regulate dual-use exports generally apply to HPCs.
The Commerce Department’s Bureau of Export Administration maintains the Commerce Control List that includes items (commodities, software, and technology) subject to the authority of the Bureau. HPC technology is included on the Commerce Control List under Category 4, "Computers." HPCs specifically fall under 4A003 (which includes "Digital computers," "electronic assemblies," and "related equipment, and specially designed components") and D001 ("Software specially designed or modified for the ‘development’, ‘production’ or ‘use’ of equipment or ‘software’ controlled by" various other export control categories).116
The Export Administration Regulations identify six bases for controlling HPC technology, in order of restrictiveness. Those requiring licenses for a larger number of countries or items are listed first:
The Export Administration Regulations state the terms of the Composite Theoretical Performance license exception and the country tier structure. They also detail the new requirements on notification, post-shipment verifications for Tier 3 countries mandated by the Fiscal 1998 National Defense Authorization Act, and other restrictions and reporting requirements.118
The Export Administration Regulations contain special provisions for exports, re-exports, and certain intra-country transfers of HPCs, including software and technology. License requirements reflected in this section are based on particular destinations, end users, and end uses. These license requirements supplement those that apply for other control reasons, such as nuclear nonproliferation.119
License applications for HPC technology covered by this section are also to be reviewed for nuclear nonproliferation licensing policy. The Commerce Department may also require end-use certifications issued by the government of the importing country and safeguard conditions on the license.120
The Export Administration Regulations state security conditions and safeguard plans for the export, re-export, or in-country transfer of HPCs that the Bureau of Export Administration may impose to certain destinations. Up to 36 safeguard conditions are available.121 These include the following:
The Export Administration Regulations contain prohibitions against exports, re-exports, and selected transfers to certain end users and end uses. They state that the exporter may not export or re-export any item without a license to any destination, other than those specified in the regulations, if at the time of the export, the exporter knows the item will be used directly or indirectly in proscribed activities.123
These activities include nuclear, missile, chemical, and biological end uses.124 The Export Administration Regulations define "knowledge" of a circumstance not only as positive knowledge that the circumstance exists or is substantially certain to occur, but also an awareness of a high probability of its existence or future occurrence.125 Such awareness is inferred from evidence of the conscious disregard of facts known to a person, and is also inferred from a person’s willful avoidance of facts.126
The Export Administration Act provides that the Secretary of Commerce and his designees may conduct, outside the United States, pre-license investigations and post-shipment verifications of items licensed for export.127
The Second Stanford Study
Two of the three authors of the 1995 Stanford study were again engaged and funded by the Departments of Commerce and Defense to prepare a second paper as a contribution to the periodic review of HPC export controls.128
This paper, released in April 1998, concluded that rapid advances in computer technology were continuing. However, it also suggested that a proposed change in licensing procedure — to review each HPC at its highest attainable level, rather than its configuration at the time of the export — would remove the concern that HPCs were being upgraded without the knowledge of exporters or the U.S. Government.129
As of the date of this report, no further Executive branch action has been reported or notified to Congress concerning further revisions to export controls on HPCs.
Arms Export Control Act Provisions and International Traffic In Arms Regulations Currently Applicable to Computers
The Arms Export Control Act and International Traffic In Arms Regulations treat certain computers differently than the dual-use computers that are regulated by the Export Administration Act and Export Administration Regulations.130
The United States Munitions List, which is included in the International Traffic in Arms Regulations, controls computers that have been modified for rugged conditions and "Tempested" — made ready for secure use — specifically for military systems.131 It also controls software specifically designed for military uses and technical data, which is often paper converted to software.132
The State Department, which has license authority over Munitions List items, restricts the export of computers designed for military uses and does not distinguish among computers based on MTOPS or other performance measures.133
Concerns Over High Performance Computer Exporters’ Ability to Review End-Users in the PRC Prompted the Requirement for Prior Notification
The January 1996 revisions to the Export Administration Regulations governing HPCs made several other important changes. Most importantly, they made exporters responsible for determining whether an export license is required, based on the MTOPS level of the computer, and for screening end users and end uses for military or proliferation concerns.134
Thus, U.S. companies that wish to export HPCs are authorized to determine their own eligibility for a license exception.135
Prior to this change, only U.S. HPC exports to Japan were allowed without an individual license. At that time, a violation of the Export Administration Regulations could be identified by an export of an HPC that occurred without a license.
Since the change, in order to prove a violation of the regulations, the Commerce Department must demonstrate that an exporter improperly used the Composite Theoretical Performance license exception and knew or had reason to know that the intended end user would be engaged in military or proliferation activities.136
Also, the revised Export Administration Regulations required that exporters keep records and report to the Commerce Department on exports of computers with performance levels at or above 2,000 MTOPS. In addition to existing record-keeping requirements, the regulations added requirements for the date of the shipment, the name and address of the end user and of each intermediate consignee, and the end use of each exported computer. Although these records have been reported to the Commerce Department on a quarterly basis for the past two years, some companies have reported inconsistent and incomplete data for resellers or distributors as end users.137
Since U.S. HPCs obtained by countries of proliferation concern could be used in weapons-related activities, the Congress enacted a provision in the Fiscal 1998 National Defense Authorization Act138 that required exporters to notify the Commerce Department of all proposed HPC sales over 2,000 MTOPS to Tier 3 countries. The Act gives the U.S. Government an opportunity to assess these exports within 10 days and determine the need for a license. Following such notification, the Departments of Commerce, State, Defense, and Energy, and the U.S. Arms Control and Disarmament Agency, can review a proposed HPC sale and object to its proceeding without an export license. The Commerce Department announced regulations implementing the law on February 3, 1998.139
A November 1998 Defense Department study, however, identified potential problems with the 10-day notification procedure. The study noted that the Defense Department provides comments on export notices referred to it regarding those end users for which the Defense Department has information. The study also noted that:
The operating assumption is that, if there is no information on the end-user, then the end-user is assumed to be legitimate. This is probably true in most cases; however, there is no means to verify that high performance computers are not making their way to end-users of concern to the United States.140
Furthermore, the Defense Department study expressed concern that foreign buyers might circumvent current Export Administration Regulations provisions requiring attestation to the buyer’s knowledge that the export will have no military or proliferation end user or end use.141 By designating a company in the United States to act on its behalf, the foreign company could have its U.S. designee submit the HPC notification to the Commerce Department; the U.S. designee and not the foreign buyer would then be responsible for all compliance with notification procedures.142 The U.S. designee would be responsible only for shipping the item and would not take title of the item.143
Under the Export Administration Regulations, the U.S. designee could complete the notification to its knowledge, which might be useless if the U.S. designee is in fact ignorant of the actual end use. The Defense Department study noted the obvious problems with this system.
The study also observed that the 10-day notification period was insufficient to ensure that U.S. designees and foreign buyers are providing accurate and complete information.144
Finally, the Defense Department study warned that foreign buyers of U.S. computer technology might circumvent the notification procedure by notifying the Commerce Department that they are purchasing a system that is not above the 7,000 MTOPS threshold, but later upgrading the system with processors that are below the 2,000 MTOPS level. There would be no requirement to notify the Commerce Department of the acquisition of the lower than 2,000 MTOPS upgrades to the previously-notified system.145
The U.S. Government Has Conducted Only One End-Use Check for High Performance Computers in the PRC
The Fiscal 1998 National Defense Authorization Act now requires the Commerce Department to perform post-shipment verifications on all HPC exports of HPCs to Tier 3 countries with performance levels over 2,000 MTOPS.146
Post-shipment verifications are important for detecting and deterring physical diversions of HPCs, but they do not always verify the end use of HPCs.147
The PRC traditionally has not allowed the United States to conduct post-shipment verifications, based on claims of national sovereignty, despite U.S. Government efforts since the early 1980s.148 This obduracy has had little consequence for the PRC, since HPC exports have continued to be approved and, in fact, have increased in recent years.
In June 1998, the PRC agreed with the United States to cooperate and allow post-shipment verifications for all exports, including HPCs.149 PRC conditions on the implementation of post-shipment verifications for HPCs, however, render the agreement useless.150 Specifically:
The Select Committee has reviewed the terms of the U.S.-PRC agreement and found them wholly inadequate. The Clinton administration has, however, advised the Select Committee that the PRC would object to making the terms of the agreement public. As a result, the Clinton administration has determined that no further description of the agreement may be included in this report.
According to Iain S. Baird, Deputy Assistant Secretary of Commerce for Export Administration within the Bureau of Export Administration, post-shipment verifications are conducted by the PRC’s Ministry of Foreign Trade and Economic Cooperation for U.S. computers having over 2,000 MTOPS that are exported to the PRC. He says such verifications are done in the presence of the U.S. commercial attaché.151
Commerce reported on November 17, 1998, that no post-shipment verifications would be performed on HPCs that were exported to the PRC from November 18, 1997 through June 25, 1998 because the PRC/U.S. agreement applies only prospectively from June 26.
Since June 26, the Commerce Department reported, only one post-shipment verification has been completed and one was pending as of November 12, 1998. Commerce also stated that "Post shipment verifications were not done on most of the others [HPCs] because the transactions do not conform to our arrangement with the PRC for end use checks."152
Thus, post-shipment verifications will not be done on any HPCs exported to the PRC prior to the agreement, nor on any HPCs shipped that are exported in the future under the Composite Theoretical Performance license exception (that is, those between 2,000 and 7,000 MTOPS) to civilian end users.
According to Commerce Department Under Secretary for Export Enforcement William Reinsch, a pending regulatory change will instruct HPC exporters to seek end-use certificates from the PRC Government. Where PRC end-use certificates are obtained, this regulation purportedly would allow more post-shipment verifications to be requested consistent with the PRC-U.S. agreement.153
Reinsch stated that the PRC has indicated that it would be willing to issue end-use certificates. However, the PRC office in question reportedly has a staff of five, which would severely limit the number of post-shipment verifications it could implement.154
According to a September 1998 report from the General Accounting Office, U.S. Government officials agreed that the manner in which post-shipment verifications for computers traditionally have been conducted has limited their value because they establish only the physical presence of an HPC, not its actual use. In any event, according to national weapons laboratory officials within the Energy Department, it is easy to conceal how a computer is being used.155
Even when U.S. Government officials perform the post-shipment verification, the verifying officials have received no specific computer training and are capable of doing little more than verifying the computer’s location. It is possible to verify an HPC’s use by reviewing internal computer data, but this is costly and intrusive, and requires sophisticated computer analysis.156
The General Accounting Office report also noted that the U.S. Government makes limited efforts to monitor exporter and end-user compliance with explicit conditions that are often attached to HPC export licenses for sensitive end users. The U.S. Government relies largely on the HPC exporters to monitor end use, and may require them or the end users to safeguard the exports by limiting access to the computers or inspecting computer logs and outputs.157
The end user may also be required to agree to on-site inspections, even on short notice, by the U.S. Government or exporter. These inspections would include review of the programs and software that are being used on the computer, or remote electronic monitoring of the computer.158
Commerce officials stated to GAO that they may have reviewed computer logs in the past, but do not do so anymore, and that they have not conducted any short notice visits. They also acknowledged that they currently do not do any remote monitoring of HPC use anywhere and that, ultimately, monitoring compliance with safeguards plans and their conditions is the HPC exporter’s responsibility.159
Some U.S. High Performance Computer Exports to the PRC Have Violated U.S. Restrictions
During the 1990s, there have been several cases of export control violations involving computer technology shipments to the PRC. One ongoing case concerns the diversion of a Sun Microsystems HPC from Hong Kong to the PRC.160
On December 26, 1996, a Hong Kong reseller for Sun Microsystems, Automated Systems Ltd., sold an HPC to the PRC Scientific Institute, a technical institute under the Chinese Academy of Sciences — a State laboratory specializing in parallel and distributed processing. At some point after the sale but before delivery, the computer was sold to Changsha Science and Technology Institute in Changsha, Hunan Province. The machine was delivered directly to that Institute in March 1997.161
Automated Systems of Hong Kong claimed to Sun officials in June 1997 that it had understood that the Changsha Institute was "an educational institute in Wuhan Province providing technological studies under the Ministry of Education." The end use there, according to Automated Systems, was to be for "education and research studies in the college and sometimes for application development for outside projects." Sun was recommended to contact the end user, the Changsha Institute, for more specific end-use information.162
The HPC sale came to the attention of the Deputy Assistant Secretary for Export Enforcement, Frank Deliberti. He queried the U.S. Embassy in Beijing about the Changsha Institute. Deliberti gave the information he obtained to Sun Microsystems, which then initiated efforts to have its computer returned.163
During the same period, the Foreign Commercial Officer at the U.S. Embassy in Beijing consulted his contacts at the PRC’s Ministry of Foreign Trade and Economic Cooperation. The Ministry denied that the Changsha Institute was affiliated with the PRC military.164
Subsequently, the Ministry called the FCO to inform him that the actual buyer of the computer was an entity called the Yuanwang Corporation, and that Sun Microsystems had been aware of this corporation’s PRC military ties. Reportedly, Yuanwang is an entity of the Commission on Science, Technology, and Industry for National Defense (COSTIND). So far as the PRC’s Ministry of Foreign Trade and Economic Cooperation reportedly could determine, the end-use statements that had been provided to Sun through Automated Systems of Hong Kong were totally fictitious. The Changsha Science and Technology Institute, according to the Ministry, did not exist.165
The official position of the Ministry of Foreign Trade and Economic Cooperation was that the PRC Government would not help to obtain the return of the computer. The role of the PRC Government, the Ministry asserted, had been merely to help two private parties rectify a misunderstanding. In any event, the computer was returned to the United States on November 6, 1997.166 The Commerce Department investigation reportedly is continuing.167
A number of other violations of U.S. laws and regulations concerning computers exported to the PRC have been investigated by the Commerce Department:
New World Transtechnology
On December 20, 1996, New World Transtechnology of Galveston, Texas, pled guilty to charges that it violated the export control laws and engaged in false statements by illegally exporting controlled computers to a nuclear equipment factory in the PRC in August 1992. The company was also charged with attempting to illegally export an additional computer to the PRC through Hong Kong in October 1992. The company was sentenced to pay a $10,000 criminal fine and a $600 special assessment fee.168
Compaq Computer Corporation
On April 18, 1997, the Commerce Department imposed a $55,000 civil penalty on Compaq Computer Corporation of Houston, Texas, for alleged violations of the Export Administration Regulations. The Commerce Department alleged that, on three separate occasions between September 17, 1992 and June 11, 1993, Compaq exported computer equipment from the United States to several countries, including the PRC, without obtaining required export licenses. Compaq agreed to pay the civil penalty to settle the allegations.169
Digital Creations
On June 12, 1997, Digital Creations Corporation of Closter, New Jersey, was sentenced to pay an $800,000 criminal fine for violating the Export Administration Act and Regulations in connection with exports of computers to the PRC. Digital had previously pled guilty in December 1994 to charges that it had violated the Export Administration Regulations by illegally exporting a Digital Equipment Corporation computer to the PRC without obtaining the required export license.170
Lansing Technologies Corporation
On June 17, 1997, Lansing Technologies Corporation, of Flushing, New York, pled guilty to charges that it violated the Export Administration Regulations in 1992 by exporting a Digital Equipment Corporation computer vector processor and a data acquisition control system to the PRC without obtaining the required export licenses from the Commerce Department.171
Other serious violations of HPC export control laws and regulations have occurred in recent years, but these concerned Russia. On July 31, 1998, for example, the Department of Justice announced that IBM East Europe/Asia Ltd. entered a guilty plea. IBM received the maximum allowable fine of $8.5 million for 17 counts of violating U.S. export laws through the sale of HPCs to a Russian nuclear weapons laboratory known as Arzamus-16. In another example, an ongoing U.S. Government investigation of Silicon Graphics Incorporated/Convex is examining whether a violation of law occurred in a sale of HPCs to another Russian nuclear weapons laboratory, Chelyabinsk-70.172
High Performance Computers at U.S. National Weapons Laboratories Are Targets for PRC Espionage
No other place in the world exceeds the computational power found within the U.S. national weapons laboratories. For this reason, both the computational power and the data it can generate have been the focus of the PRC’s and other countries’ intelligence collection efforts.
The desire for access to this computing power and data, in turn, is one of the reasons so many foreign nationals want to visit the laboratories.
According to David Nokes, the network administrator at Los Alamos National Laboratory, all operating systems have vulnerabilities that can be exploited by a knowledgeable, valid user.173 Nokes also says that there are a few solutions to issues of HPC network security. These include:
U.S. National Weapons Laboratories Have Failed to Obtain Required Export Licenses for Foreign High Performance Computer Use
When foreign nationals use the U.S. national weapons laboratories’ HPCs, their activities should generally be considered "deemed exports." The "deemed export" rule [15 CFR 734.2 (b) (ii)] covers those situations in which an export-controlled technology or software-source code information is released to a visiting foreign national, for which a license would have been required. In such situations, an "export" is "deemed" to have occurred.
The Select Committee is concerned that HPC system managers in the U.S. national weapons laboratories lack an essential understanding of the deemed export rule. This lack of understanding was substantiated by interviews with representatives from the Department of Commerce who had no recollection of ever having seen an application for a deemed export from any of the U.S. national weapons laboratories.
When PRC nationals visit and use the HPCs at a U.S. national weapons laboratory, their access should be limited to the same computing capabilities to which the PRC itself is restricted, especially for military uses.175 The Select Committee discovered, however, that the laboratories do not even measure the computational power of their HPCs in MTOPS. Moreover, many of the laboratories have difficulty in converting to MTOPS from the units they use to measure the power of an HPC.
The Department of Commerce could not recall a laboratory ever having sought guidance on how to compute an HPC’s MTOPS rating. Significantly, the Select Committee discovered that a rather modest HPC (by Department of Energy standards) in a U.S. National Laboratory used by foreign nationals had a substantially higher MTOPS rating than the controlled threshold. No licenses, however, had ever been obtained.
The "deemed export" rule also applies in those instances in which a PRC national or entity accesses an HPC remotely via the Internet.
In the absence of an effective audit system, which monitors the codes being run by the PRC user, the U.S. national weapons laboratories cannot verify that they are in compliance with the law, or that PLA or PRC intelligence is not using the HPCs for the design or testing of nuclear or other weapons.
PRC Students Have U.S. Citizen-Like Access To High Performance Computers at the National Weapons Laboratories
The U. S. national weapons laboratories rely upon nuclear weapons test simulation software and computers provided by the Accelerated Strategic Computer Initiative (ASCI). Five major U.S. universities support ASCI through the Academic Strategic Alliances Program (ASAP).
As a result, hundreds of research students and staff at these universities have access to the HPCs used by the national weapons laboratories for U.S. nuclear weapons research and testing. As many as 50 percent of these research students and staff are foreign nationals, some of whom may have foreign intelligence affiliations.
Holders of Immigration and Naturalization Service "green cards" — PRC nationals who have declared their intent to remain permanently in the U.S. — are treated as U.S. citizens for export control purposes. They are then given U.S. citizen-like HPC access, free to return to the PRC once their objectives are fulfilled.
In November 1998, the Secretary of Energy issued an Action Plan that includes a task force to review HPC usage by foreign nationals and provide a report to the Secretary within six months. The Department of Energy is currently preparing an implementation plan to address counterintelligence issues identified in a July 1998 report, entitled "Mapping the Future of the Department of Energy’s Counterintelligence Program," including HPC usage by foreign nationals.
Many Types of Computer Technology Have Been Made Available to the PRC That Could Facilitate Running Programs of National Security Importance
One of the bases for the 1996 increase in export control thresholds was that individual PCs were widely available on the open market in the United States, but not able to be exported to the potentially huge PRC market.176 What was an HPC in 1993 (those capable of 195 or more MTOPS) was no longer even considered necessary to control for weapons proliferation concerns.177
By 1997, PCs and workstations assembled in the PRC captured approximately 60 percent of the PRC’s domestic market.178 All of these locally-assembled computers used imported parts — over 70 percent contained United States-produced Pentium microprocessors.179
Three of the largest manufacturers in the PRC were affiliates of IBM, Hewlett Packard, and Compaq, with a combined market share of approximately 21 percent.180 A large share (but probably not more than 20 percent) of the PC assembly in the PRC was done by small, independent assembly shops.181
The largest individual producer of PCs and workstations in the PRC is the Legend enterprise, a spin-off of the Chinese Academy of Sciences.182 This domestic computer assembly industry dovetails well with Beijing’s overall plans for economic modernization. Beijing reportedly desires an independent PRC source of most high-technology items to avoid reliance on foreign providers for these goods.
To participate more fully in the PRC market, United States firms have been pressured by the PRC government to relinquish technological advantage for short-term market opportunities. The PRC requires that foreign firms be granted access to the PRC market only in exchange for transferring technology that would enable the state-run enterprises to eventually capture the home market and begin to compete internationally.
However, the PRC’s strategy of coercing technology from foreign firms has not enabled state-run industries to close the technology gap with more developed nations. In the context of establishing domestic production of computers for sale in the PRC, this PRC "technology coercion" policy appears to have worked.183 The PRC now has a growing industrial base of small computer assemblers. For the most part, these companies are not State-run. The technology that was "coerced" from U.S. computer manufacturers as a cost of entering the PRC market apparently better serves the expansion needs of small, relatively independent enterprises and not the intended needs of central planners in Beijing.
90 percent of PRC consumers of PCs and workstations are business, government, and educational entities, with individual purchases accounting for only 10 percent of the PRC’s PC market.184 To illustrate the size of the individual purchaser segment of the PRC’s market, it is estimated that only 5 million individuals out of the PRC’s 1.2 billion have the expendable funds required to purchase a low-end PC in the PRC.185
Despite the limited number of individual purchasers, the actual size of the PRC PC and workstation market was 2.18 million units in 1996; 3 million units in 1997; and 4.5 million units in 1998. It is anticipated the PRC PC and workstation market will grow at the rate of 1.5 million to 2 million units per year through the year 2000. According to figures provided by the Asia Technology Information Project, an independent research foundation, non-PRC manufacturers of PCs and workstations, including U.S. manufacturers, could expect to partake of a portion of the almost 2 million units expected to be imported for sale in the PRC in 1998.186
The PRC Has a Limited Capability to Produce High Performance Computers
The PRC has demonstrated the capability to produce an HPC using U.S.-origin microprocessors over the current threshold of 7,000 MTOPS. The PRC "unveiled" a 10,000 MTOPS HPC — the Galaxy III — in 1997 based on Western microprocessors.
But PRC HPC application software lags farther behind world levels than its HPC systems. Also, despite the existence of a few PRC-produced HPCs based on Western components, the PRC cannot cost effectively mass-produce HPCs currently. There really is no domestic HPC industry in the PRC today.
While it is difficult to ascertain the full measure of HPC resources that have been made available to the PRC from all sources, available data indicates that U.S. HPCs dominate the market in the PRC.187
Although the PRC has a large market for workstations and high-end servers, there is a smaller market for parallel computers which is entirely dominated by non-PRC companies such as IBM, Silicon Graphics/Cray, and the Japanese NEC. However, there continues to be significant market resistance to Japanese HPC products in Asia, especially as U.S. products are beginning to have significant market penetration.188
U.S. High Performance Computer Exports To the PRC Are Increasing Dramatically
A review of Commerce Department information regarding the total of HPC license applications that were received for the time frame January 1, 1992 to September 23, 1997, revealed the following:
The approximate total value of the HPCs exported, of whatever description, to both Hong Kong and the PRC, for the six-year period ending September 23, 1997, was only $86 million.190
The nine-month period between January 1998 and September 1998, however, saw U.S. exporters notify the Commerce Department of their intention to export 434 HPCs (in the 2,000 to 7,000 MTOPS range) to the PRC (total value $96,882,799).191 Nine times the number of HPCs were exported in one-ninth the time.192
During approximately the same time frame (calendar year 1998) it is estimated that 9,680,000 individual PCs and workstations were sold in the PRC. The market share that U.S. exporters could reasonably expect to benefit from was approximately 3,872,000 units, worth approximately $1.8 billion.193
Apparently, the proximate cause of U.S. computer manufacturers aggressively lobbying for the raising and maintaining of export thresholds above the PC level was to capture this $1.8 billion per year market share.
The United States dominates the PRC’s HPC market, but U.S. exports clearly do not dominate the PRC’s personal computer and workstation market.194 The difference between the 460-unit, $100 million HPC market described above, stretched over a six-year period, and the yearly three million-unit PC and workstation market, with a value of $1.8 billion, is dramatic.
The performance levels of U.S. HPCs reported to be exported to the PRC over the past year continued to be predominantly in lower-end machines, as shown in the following table. For example, 77 percent of U.S. HPCs (a total of 388 machines) have performance levels below 4,000 MTOPS.
Number of U.S. HPCs Reported to Be Exported to the PRC Under License Exception, by MTOPS195 September 1997 - September 1998
MTOPS Total
2,000 to 2,999 302
3,000 to 3,999 86
4,000 to 4,999 71
5,000 to 5,999 28
6,000 to 6,999 15
> 7,000 0
Total 502
The PRC Is Obtaining Software From U.S. and Domestic Sources
In June 1997, it was estimated that 96 percent of software programs sold in the PRC were pirated versions of commercially available U.S. programs. These programs were designed for use on PCs and workstations, and are not considered useful for the very sophisticated programming done on HPCs.
Some major U.S. software producers have begun contracting with PRC programming firms. These PRC software firms are comprised of recently-graduated PRC university students. They are attempting to write programs in Chinese to capitalize on a huge domestic market.196
Two factors mitigate against the success of the PRC developing its domestic programming industry.
The first factor is that street-level "software pirates" sell dozens of U.S. computer programs at a time on one CD-ROM for a small fee (reportedly $20). In other words, one can meet most or all of one’s programming needs in the PRC for a nominal fee. It is anticipated that it will be difficult, if not impossible, for a domestic software industry to recoup the start up costs associated with just one software program, let alone the dozens needed to compete with the street level dealers.
The second factor is that these pirated U.S.-produced, English language programs are more mature, widespread, and robust than PRC programs.197 It is axiomatic that any new product will have "bugs in the system." It is considered unlikely that new, unproven, and possibly weak software programs will effectively compete with cheap, proven, and robust software that is widely available at such nominal fees. It is conceivable that the PRC will abandon instituting a domestic programming industry altogether.198
Potential Methods of Improving End-Use Verification
According to a 1996 RAND study, there are non-intrusive and intrusive approaches to assessing the manner in which a buyer is actually applying dual-use technologies. Among the non-intrusive methods are:
Among the intrusive methods are:
Tagging
Tagging is achieved by attaching an active system to the item that is to be exported, rather than just a passive tag for identification during an inspection. The active system would both monitor the object tagged and communicate that information back to the United States. The RAND study noted that in practice, this means the objects to be tagged must be physically large systems, such as a machine-tool cell, or a major component of some larger system, such as a turbine engine in a helicopter.200
According to the RAND study, the tag should be capable of at least communicating information about the item’s physical location. Some sensors may provide other kinds of information, as well. The information could be communicated to a satellite or over a data link. Early versions of such devices were already in use in 1996 to monitor nuclear materials and technologies.201
These "smart" tags exploit the potential of several technologies, according to the RAND study. They combine encryption, the Global Positioning System, and emerging global wireless communications systems, such as Iridium or Orbcomm. These technologies would allow the tags to report back on the status and location of the tagged object. In principle, such tags could report the position of an object at any given time in order to verify limitations on their location. Such tags could also report on the activities of a "smart" system to which they are attached. For example, a machine-tool cell could report whether the machine had been used to make parts resembling aircraft components.202
Such tags could have many applications in a cooperative regime. Their application and use in a prohibited environment would be more difficult and consequential.203
The RAND study cautioned that all sellers of a particular technology must participate in the tagging and that this would probably also require cooperation of the buyers. Otherwise, buyers would gravitate to untagged items, if they were available. Attempts to conceal system location or deviate from a pattern of cooperation would be considered evidence of a potential failure of performance by the buyer. The study concluded that tagging may become an important oversight method for controlling technology transfers, but that it should never become the sole means of oversight.204
Technical Safeguards
In 1994 several types of technical safeguards were in advanced development in the United States. The technologies required for these safeguards were expected to enter testing within the next two years. They included:
Although these technical safeguards seem feasible, none had been proved to be inexpensive, sensitive enough to detect most illegal activity, and difficult to circumvent by determined adversaries. The auditing tools under development showed great promise, however. Authorities were pessimistic about the likelihood that technical high-performance computer safeguards would be widely adopted and able to succeed in the near future.
Other Possibilities
Officials of the Mitre Corporation made several suggestions to strengthen U.S. national security in the context of HPC export controls. These included:
Technical Afterword
HIGH PERFORMANCE COMPUTERS
Changing High Performance Computer Technology Is Making Export Control More Difficult
New designs in HPCs and systems of computers, as well as availability of more advanced and less costly processors, software, and peripheral equipment, is rendering the challenge of applying export controls to HPCs more difficult.
For certain types of computer designs, the ability to add processors or boards could increase the machine’s performance beyond authorized levels. In addition, advances in computer processor communications technology have facilitated the clustering of personal computers and workstations into effective parallel computers.
The usefulness of clustered computers is application-dependent. Some U.S. Government and computer industry experts have concluded that for many problems, networks of workstations could not compete with appropriately designed high performance computers.1 Most traditional HPCs achieve far greater efficiency than parallel machines, due to their use of custom-made components.
Foreign access to high performance computers through networks is possible because of inadequate security measures.
Vector Architectures
Vector architecture relies on custom-designed processors to move a complex problem through computer processing units in sequential stages. This type of machine is designed to handle arithmetic operations efficiently on elements of arrays, called vectors.2
Vector systems are especially useful in high-performance scientific computing.3 Vector systems, also called "pipeline" architectures, work like an assembly line. They work best with many similar tasks that can be broken down into steps.
The memory interface in vector machines is custom-made, and subject to export controls.
Vector machines are useful for cryptography, modeling fluids, and in the design of weapons. In particular, vector systems are suited to problems in which data at one point influence other variables in the problem, a common situation in national security applications.4
It is more straightforward for a programmer to use a vector system than a system comprised of parallel processors (discussed below), since it is easier to obtain maximum performance with one or a few high-power processors than with a collection of many lower capability processors.5
Since one of the main concerns with any HPC system is the rate of speed with which data can be retrieved from memory, another advantage is that a vector machine has a very fast memory.6
Still further advantages of vector systems are that they feature high memory bandwidth and low memory latency — that is, very large amounts of data can travel to and from memory very efficiently. A related advantage is that vector systems have the ability to seek multiple memory locations at the same time. This translates into very fast computational speed.
A disadvantage of a vector machine is that vector system software is not really portable. It cannot be readily transported to other vector machines.7
The main disadvantage of vector systems, however, is their high cost. Significant improvements in software and hardware allow the purchase of a parallel processing system for $40,000, as opposed to $1 million for a comparable vector computer.8
At the Defense Department’s High Performance Computer Management Office, vector systems are being phased out in favor of parallel processing systems. Out of a total of 40 HPCs in the High Performance Computer Management Office inventory, fewer than 10 are now vector systems.9
Parallel Processing: The Connection of Computers Into a Powerful Central Resource
A parallel processing computer is a collection of processors that are connected through a communications network.10 The type of processor, the network configuration, and the operating system that coordinates the activities distinguish parallel processing systems.
Many national security applications involve problems that can be separated into independent variables, and it is for these types of problems that parallel processing is best suited.11
The fastest parallel machines are all based on commodity processors — that is, processors that are commercially available on the market.12 This approach has been applied to virtually every area of theoretical and applied physics.13
Massively Parallel Processors
A massively parallel processor is a collection of computers, or central processing units, linked together.14 Each computer that is part of the whole massively parallel processor has its own memory, input/output system, and central processing unit.15 Massively parallel processors now use commodity processors, and can utilize commodity interconnects to communicate between the individual computers that make up the system.16 Some massively parallel processors use custom-made, very fast interconnect switches that are not commodities and are subject to export control.17
An advantage of a massively parallel processor is that an unlimited quantity of processors can be incorporated into the design of the machine. In a massively parallel processor, the more processors, the greater the computing speed of the machine.18
Because each processor is equipped with its own memory, massively parallel processors have much more memory than traditional supercomputers. The extra memory, in turn, suits these machines to data-intensive applications, such as imaging or comparing observational data with the predictions of models.19
A disadvantage of massively parallel processors is that memory latency is a bigger problem because the processors have to share the available memory. Another disadvantage is that each one of the computers that is part of the system has to be instructed what to do individually.20 This phenomenon requires specialized, extremely proficient programmers to create efficient communications between the individual computers.
The commercial availability of inexpensive, powerful microprocessors has given massively parallel processors a boost in their competition with vector machines for the supercomputer market. IBM, for example, more than doubled the number of its computers in the Top 500 list (discussed below) between November 1997 and June 1998 by introducing the SP2, which strings together up to 512 of the company’s RSI6000 workstation microprocessors.21
If optimum speed is desired, this massively parallel configuration is the best of all HPC designs.22 The fastest high performance computer now available is the ASCI Blue Pacific.23 That machine is part of the Department of Energy’s Accelerated Strategic Computing Initiative (ASCI) program and is located at Lawrence Livermore National Laboratory. Developed in conjunction with IBM, it is a 5,856-processor machine, boasting a top speed of 3.8 teraflops24 (Tflops) peak speed with 2.6 terabytes (Tbytes) of memory.25 In the next phase of the ASCI initiative, IBM will deliver a 10-Tflops machine to the Department of Energy in mid-2000.26
Symmetrical Multiprocessor Systems
Symmetrical multiprocessor systems use multiple commodity central processing units (CPUs) that are tightly coupled via shared memory. The number of processors can be as low as two and as many as about 128.27
Symmetrical multiprocessor systems treat their multiple CPUs as one very fast CPU.28 The CPUs in a symmetrical multiprocessor system are arranged on a single motherboard and share the same memory, input/output devices, operating system, and communications path.
Although symmetrical multiprocessor systems use multiple CPUs, they still perform sequential processing,29 and allow multiple concurrent processes to be executed in parallel within different processors.30
An advantage of symmetrical multiprocessor systems is that the programming required to control the CPUs is simplified because of the sharing of common components.31
Another major advantage is cost. A Silicon Graphics symmetrical multiprocessor system, for example, with 18 microprocessors, each rated at 300 megaflops (MFLOPS)32 or more, and a peak speed of more than 5 gigaflops (GFLOPS), costs about $1 million whereas a Cray C90 costs about $30 million.33
Even though the Silicon Graphics machine is about a third as fast as the Cray machine, it is still very popular with consumers of these types of machines. The University of Illinois Supercomputing Center reportedly likes the price, flexibility, and future promise of symmetrical multiprocessor systems so much that it plans to use them exclusively within two years. Its older Crays were "cut up for scrap" at the beginning of this year, and its massively parallel computers will be phased out by 1997.34
One disadvantage of a symmetrical multiprocessor system is that all the CPUs on a single board share the resources of that board. This sharing limits the number of CPUs that can be placed on a single board.35
Although the programming model that a symmetrical multiprocessor system provides has proved to be user-friendly, the programmer must exercise care to produce efficient and correct parallel programs. To limit latency in individual jobs, most software requires enhancement — for example, employing special programming techniques to prevent components of the computer program from competing for system resources — thereby increasing inefficiency.
For this reason, symmetrical multiprocessor systems are not good platforms for high-performance real-time applications.36
In a symmetrical multiprocessor system design, as is true with a massively parallel processor system, the number of CPUs determines how fast a machine potentially will operate. This fact causes a problem for export controls because it is possible to add CPUs to the boards of a symmetrical multiprocessor system, or boards to a massively parallel processor system, and push the machine over export control thresholds after the original export-licensed purchase.37
Clusters of Commercial Off-the-Shelf Computers and Networks
Recent advances in the process of computer-to-computer communication, or networking, allow computers to be linked together, or "clustered." Networking has allowed the clustering of personal computers and workstations into well-balanced effective parallel computers, with much higher computing capabilities than any one of the clustered computers.38
Four thresholds have been crossed in connecting commercial-off-the-shelf components to create parallel computers:
According to officials at the Lawrence Livermore National Laboratory, networking represents only a 10 percent additional cost over the cost of the computing hardware for large systems. Thus, up to approximately 50,000 MTOPS, the computing capability available to any country today is limited only by the amount of money that is available to be spent on commercial-off-the-shelf networking.40
A typical commercial-off-the-shelf networking technology contains five essential elements. They are all inexpensive and widely available. The three hardware elements are switches (approximate cost: $2,000), cables (approximate cost: $100), and interface cards (approximate cost: $1,500). The two software elements are low-level network drivers for common operating systems, and industry standard communication libraries. The hardware and software technology necessary to successfully cluster commercial-off-the-shelf CPUs into effective parallel computers are well developed and disseminated in open, international collaborations worldwide.41
The concept of clustering commercial-off-the-shelf computers has been a subject of open academic study for over a decade. Today, the Beowulf Consortium acts as a focal point for information on clustering technology and has links to many projects. One Beowulf project is the Avalon computer at Los Alamos National Laboratory. Avalon can operate at 37,905 MTOPS42 and was built in four days in April 1998 entirely from commodity personal computer technology (70 DEC Alpha CPUs) for $150,000.
Although commercial-off-the-shelf networking technology has only recently become effective, it has been adopted rapidly. There currently are at least seven competing high-performance network technologies (over 100 megabytes per second or higher): Myrinet, HIPPI, FiberChannel, Gigabit Ethernet, SCI, ATM, and VIA. One network vendor reported over 150 installations in the United States and 17 foreign countries including Australia, Brazil, Canada, the Netherlands, England, France, India, Israel, Italy, Japan, the Republic of Korea, and the PRC.43
Gigabit Ethernet is of particular interest because it is being developed by a cooperative, worldwide industry effort called the Gigabit Ethernet Alliance. 74 companies have pledged to develop products for the open standard — that is, the source software is available openly to software developers. Foreign companies are alliance members and also participate as members of the steering committee and the certification process for compliance. Gigabit Ethernet is projected to be a $3 billion market by the year 2000, which at today’s prices translates into approximately 300,000 network switches per year.44
On October 15, 1997, a group of experts met to discuss computer performance metrics for export control purposes. The computer and high-tech industries were represented by Hewlett-Packard, Silicon Graphics/Cray Research, IBM, Digital Equipment Corporation, Intel, Sun Microsystems, the Center for Computing Sciences, the Institute for Defense Analyses, and Centerpoint Ventures. The U.S. Government was represented by the National Institute of Standards and Technology, the Naval Research Laboratory, the Defense Advanced Research Projects Agency, the National Security Agency, Lawrence Livermore National Laboratory, the Defense Technology Security Administration, and the Department of Commerce Bureau of Export Administration.45
The consensus of the discussion was that commercial-off-the-shelf networking is not so significant a threat to replace HPCs as might at first appear to be the case:
Networks of workstations using [commercial-off-the-shelf] networking technology differ from supercomputers. Some problems will run easily and effectively on such networks, while other classes of problems important to national security concerns will not run effectively without a major software redesign effort. For many problems no amount of software redesign will allow networks of workstations to compete with appropriately designed high performance computers.
Even if a "rogue state" assembled such a large network of workstations by legitimately acquiring large numbers of commodity processors, the actual effort to produce the software necessary to realize the full potential of such an aggregate system would take several years. During this time, the state of the art of computational technology would have increased by approximately an order of magnitude.
After considerable discussion, most of the participants were in agreement that there was a fundamental difference between a system designed by a single vendor that was built as an aggregate of many commodity processors and included the software to enable these processors to cooperatively work on solving single problems of national concern, and a large collection of commodity processors not subject to export control that are externally networked together.46
According to one expert, many universities have clustered systems, as they are easy to establish. For $70,000, a 12-node system with two Pentium II processors at 300 megahertz (MHz) each would produce a system with 7,200 GFLOPS.. However, the system must be properly structured to perform well, and performance will vary depending on the application, the programmer’s ability, and the connection of the machines. An integrated system from Silicon Graphics/Cray will achieve between 10-20 percent of peak performance at best.47
An example of a powerful commercial-off-the-shelf network can be found at the Illinois Supercomputing Center. Four eight-processor and two 16-processor machines from Silicon Graphics are connected in a cluster with a peak speed of nearly 20 GFLOPS.48
According to one expert, it does not require any special expertise to network workstations using commercial-off-the-shelf technology. The software engineering techniques are being taught to undergraduates as part of standard courses in advanced computing, but anyone with programming knowledge should be able to create a network as well.49
The parallel supercomputers of today have peak speeds of over 100 billion floating point operations per second (100 GFLOPS). This is roughly 100 times the peak speed of a Cray YMP class machine, which was the standard for high-performance computing of just five years ago.50
However, it is difficult to achieve a high percentage of this peak performance on a parallel machine.
Whereas a tuned code running on a Cray might reach 80-90 percent of peak speed, codes running on parallel computers typically execute at only 10-20 percent of peak.51 There are two reasons for this:
Parallel machines, on the other hand, are generally built from much simpler building blocks. For example, they may use the same processors as are used in stand-alone computer workstations. Individually, these processors are not nearly so sophisticated or so efficient as the vector processors. Thus, it is not possible to achieve so high a percentage of peak speed.53
Some parallel machines contain custom processors (TMC CM-5 vector units) or custom modifications of off-the-shelf processors (Cray T-3D modified DEC alpha chips). Even in those cases, however, the percent of peak achievable on a single node is still on the order of 50 percent or less. In parallel computer design, there is constant tension between the need to use commodity parts as the computational building blocks in order to achieve economies of scale, and the desire to achieve ever-higher percentages of peak performance through the implementation of custom hardware.54
Virtually all parallel computers today are "distributed memory" computers. This means that the random access memory (RAM) is spread though the machine, typically 32 megabytes at each node. When a calculation is performed on a parallel machine, access is frequently needed to pieces of data on different nodes.
It may be possible to overlap this communication with another computation in a different part of the program in order not to delay the entire program while waiting for the communication, but this is not always the case. Since the timing clock continues while the communication is taking place, even though no calculational work is being performed, the measured performance of the code goes down and a lower percentage of peak performance is recorded.55
Domain Decomposition
"Domain decomposition" involves partitioning the data to be processed by a parallel program across the machine’s processors.56
In distributed memory architectures, each processor has direct access only to the portion of main memory that is physically located on its node. In order to access other memory on the machine, it must communicate with the node on which that memory is located and send explicit requests to that node for data.57 Figuring out the optimal domain decomposition for a problem is one of the most basic and important tasks in parallel computing, since it determines the balance between communication and computation in a program and, ultimately, how fast that program will run.58
Memory access constitutes an inherent bottleneck in shared-memory systems.59
Highly Parallel Technology
Microprocessor-based supercomputing has brought about a major change in accessibility and affordability. Massively parallel processors continue to account for more than half of all installed supercomputers worldwide, but there is a move toward shared memory, including the use of more symmetrical multiprocessor systems and of distributed-shared memory. There is also a tendency to promote scalability through the clustering of shared memory machines because of the increased efficiency of message passing this offers. The task of data parallel programming has been helped by standardization efforts such as Message Passing Interface and High-Performance Fortran.60
Highly parallel technology is becoming popular for the following reasons. First, affordable parallel systems now out-perform the best conventional supercomputers. Cost is, of course, a strong factor, and the performance per dollar of parallel systems is particularly favorable.61 The reliability of these systems has greatly improved. Both third-party scientific and engineering applications, as well as business applications, are now appearing. Thus, commercial customers, not just research labs, are acquiring parallel systems.62
Twice a year the "Top 500 list," a compendium of the 500 most powerful computer systems, is published.63 On the previous page is an example of the numbers and types of systems in the biannual list of the top 500 fastest computers. As this chart points out, massively parallel processors and symmetrical multiprocessor systems are on the rise, while vector systems are losing ground.64
Microprocessor Technology
While vector and massively parallel computers have been contending for the supercomputing market, an important new factor has become the availability of extremely powerful commodity microprocessors, the mass-produced chips at the heart of computer workstations.
Ten years ago, workstation microprocessors were far slower than the processors in supercomputers. The fastest microprocessor in 1988, for example, was rated at one million floating point operations per second (MFLOPS) while Cray’s processors were rated at 200 MFLOPS.65 A floating-point operation is the equivalent of multiplying two 15-digit numbers. Today, Cray’s processors have improved by a factor of ten, to two gigaflops in the brand-new T90; but the fastest microprocessor runs at 600 MFLOPS, an improvement by a factor of 600.
Commercial off-the-shelf microprocessor power is available for a fraction of the cost of a traditional vector processor. Unlike vector processors, which consist of complex collections of chips and are only fabricated by the hundreds each year, commercial off-the-shelf microprocessors are designed for mass production based on two decades of experience making integrated circuits. Research and development costs for each commercial off-the-shelf microprocessor are spread over hundreds of thousands of chips.66
Microprocessors, also known as CPUs, are integrated circuits. They can be divided into broad categories of logic family technologies. The selection of a certain logic technology in the design of an integrated circuit is made after determining an application and weighing the advantages of each type of logic family. Among these are:
Realizing the differences between logic technologies gives a perspective to understanding where CPU technology is headed, and the reasons that the market is driving one technology faster than another. As the following chart illustrates, commercial off-the-shelf, inexpensive CPUs are coming to dominate the high performance computing world.68
Interconnect Technology
In multiprocessor systems, actual performance is strongly influenced by the quality of the "interconnect" that moves data among processors and memory subsystems.69
Traditionally, interconnects could be grouped into two categories: proprietary high-performance interconnects that were used within the products of individual vendors, and industry standard interconnects that were more readily available on the market, such as local area networks.70 The two categories featured different capabilities, measured in bandwidth and latency.
Recently, a new class of interconnect has emerged: clustering interconnects. These offer much higher bandwidth and lower latency than local area networks. Their shortcomings are comparable to proprietary high-performance interconnects, including lower bandwidth, higher latency, and greater performance degradation in large configurations or immature system software environments.71
Message Passing Interface
Message Passing Interface (MPI) is a program containing a set of sub-routines that provide a method of communication that enables various components of a parallel computer system to act in concert. The communications protocol that MPI uses is the same utilized by the Internet. According to Dr. Jeff Hollingsworth of the University of Maryland Computer Science Department, an example of how each of the different software applications interact with the hardware would be as follows::72
Application (Code)
t
MPI
t
TCP/IP
t
Linux
t
Windows NT (Operating system)
t
Hardware
Some software, says Hollingsworth, is sold in a version that is compatible with MPI. One example is automobile crash simulation software. This software, which is essentially code to simulate a physical system in three dimensions, is adaptable to other scientific applications such as fluid dynamics, according to Hollingsworth.73
Hollingsworth states that software that is not already "MPI ready" can be modified into code that can be run in an MPI, or parallel, environment. Modifying this software to enable it to run in an MPI environment can be very difficult, or quite easy, says Hollingsworth, depending on "data decomposition." 74
The ease of converting software that is not "MPI ready" into an "MPI ready" version is dependent on the expertise of the software engineers and scientists working on the problem. For a single application and a single computer program, the level of expertise required to convert a computer program in this way is attainable in graduate level, and some undergraduate level, college courses, according to Hollingsworth.75
It has not been possible to determine which, if any, commercially available software is both MPI ready and applicable to defense-related scientific work.
Endnotes
1 Background Paper on High Performance Computers to Countries of Concern, Defense Department, November 19, 1998.
2 Ibid.
3 Memorandum for the Record by Paul Koenig, Defense Technical Security Administration, August 24, 1995.
4 "High-Performance Computing, National Security Applications, and Export Control Policy at the Close of the 20th Century," Seymour Goodman, Peter Wolcott, Patrick Homer, May 1998.
5 "Information on the Decision to Revise High Performance Computer Controls," GAO/NSIAD-98-196.
6 Memorandum for the Record by Paul Koenig, Defense Technical Security Administration, August 24, 1995.
7 Interview of Dr. David Kahaner, October 19, 1998.
8 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the 1990s," Seymour Goodman, Peter Wolcott, Grey Burkhart, Center for International Security and Arms Control, Stanford University, November 1995.
9 Ibid.
10 "High-Performance Computing, National Security Applications, and Export Control Policy at the Close of the 20th Century," Seymour Goodman, et. al.
11 Ibid.
12 Ibid.
13 Ibid.
14 Commander in Chief, U.S. Pacific Command Memorandum to the Joint Chiefs of Staff, Ser: 444-98, November 9, 1998.
15 "PR China: System Simulation Activities," Asian Technology Information Program, December 24, 1992.
16 "Chinese Prove To Be Attentive Students of Information Warfare," Jane’s Intelligence Review, October 1997.
17 "High-Performance Computing, National Security Applications, and Export Control Policy at the Close of the 20th Century," Seymour Goodman, et. al.; "Information on the Decision to Revise High Performance Computer Controls," GAO/NSIAD-98-196.
18 Department of Energy, Office of Intelligence, Technical Intelligence Note on Computers, Nuclear Weapons, and U.S. Security, May 6, 1998.
19 Ibid.
20 Ibid.
21 "Export Controls and China," briefing prepared for the House Committee on Commerce by Dr. Thomas L. Cook, Los Alamos National Laboratory, September 17, 1998.
22 Testimony of Notra Trulock.
23 "High-Performance Computing, National Security Applications, and Export Control Policy at the Close of the 20th Century," Seymour Goodman, et. al.
24 "Export Controls and China," briefing prepared for the House Committee on Commerce by Dr. Thomas L. Cook, Los Alamos National Laboratory, September 17, 1998.
25 Testimony of Dr. Thomas L. Cook, Los Alamos National Laboratory, December 16, 1998.
26 "Export Controls and China," briefing prepared for the House Committee on Commerce by Dr. Thomas L. Cook, Los Alamos National Laboratory, September 17, 1998.
27 Commander in Chief, U.S. Pacific Command Memorandum to the Joint Chiefs of Staff, Ser: 444-98, November 9, 1998.
28 "Export Controls and China," briefing prepared for the House Committee on Commerce by Dr. Thomas L. Cook, Los Alamos National Laboratory, September 17, 1998.
29 Department of Energy, Office of Intelligence, Technical Intelligence Note on Computers, Nuclear Weapons, and U.S. Security, May 6, 1998.
30 Department of Energy, Office of Intelligence, Technical Intelligence Note on Computers, Nuclear Weapons, and U.S. Security, May 6, 1998, Attachment.
31 "Export Controls and China," briefing prepared for the House Committee on Commerce by Dr. Thomas L. Cook, Los Alamos National Laboratory, September 17, 1998.
32 "Key Projects in China’s Computerization," Asian Technology Information Program report 98.048.
33 "PR China: System Simulation Activities," Asian Technology Information Program, December 24, 1992.
34 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the 1990s," Seymour Goodman, et. al.
35 "High-Performance Computing, National Security Applications, and Export Control Policy at the Close of the 20th Century," Seymour Goodman, et. al.
36 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the 1990s," Seymour Goodman, et. al.
37 "High-Performance Computing, National Security Applications, and Export Control Policy at the Close of the 20th Century," Seymour Goodman, et. al.
38 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the 1990s," Seymour Goodman, et. al.
39 Ibid.
40 "Information Warfare Grips China," Jane’s Intelligence Review.
41 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the l990s," Seymour Goodman, et. al.
42 "Information Warfare Grips China," Jane’s Intelligence Review.
43 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the l990s," Seymour Goodman, et. al.
44 "High-Performance Computing, National Security Applications, and Export Control Policy at the Close of the 20th Century," Seymour Goodman, et. al.
45 "The Militarily Critical Technologies List Part I: Weapons Systems Technologies," June 1996, Department of Defense, p. 8-7, Background Paper on High Performance Computers to Countries of Concern, Defense Intelligence Agency, November 19, 1998, cover letter to the Honorable Floyd Spence.
46 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the 1990s," Seymour Goodman, et. al.
47 Advanced Computing Technologies in China, March 24, 1998, Asian Technology Information Project report #ATIP98.022, Book #1, Selected Reports of Asian Technology Information Project.
48 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the 1990s," Seymour Goodman, et. al.
49 Commander in Chief, U.S. Pacific Command Memorandum to the Joint Chiefs of Staff, Ser: 444-98, November 9, 1998.
50 "The Militarily Critical Technologies List Part I: Weapons Systems Technologies," June 1996, Department of Defense, Commander in Chief, U.S. Pacific Command Memorandum to the Joint Chiefs of Staff, Ser: 444-98, November 9, 1998.
51 Advanced Computing Technologies in China, March 24, 1998, Asian Technology Information Project report #ATIP98.022, Book #1, Selected Reports of Asian Technology Information Project.
52 "High-Performance Computing, National Security Applications, and Export Control Policy at the Close of the 20th Century," Seymour Goodman, et. al.
53 Ibid.
54 Ibid.
55 "Chinese Prove To Be Attentive Students of Information Warfare," Jane’s Intelligence Review, October 1997.
56 "Quicklook Report for Military Trends Workshop," Strategic Assessment Center, Science Applications International Corporation, June 8, 1998.
57 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the 1990s," Seymour Goodman, et. al.; "Information Warfare Grips China," Jane’s Intelligence Review.
58 Memorandum for the Record, Joint Investigative Staff Visit to the Mitre Corporation, Bedford, MA, October 6, 1988.
59 "Chinese Prove To Be Attentive Students of Information Warfare," Jane’s Intelligence Review, October 1997; "Information Warfare Grips China," Jane’s Intelligence Review.
60 "Chinese Prove To Be Attentive Students of Information Warfare," Jane’s Intelligence Review, October 1997.
61 Ibid.
62 "Information Warfare Grips China," Jane’s Intelligence Review.
63 Advanced Computing Technologies in China, March 24, 1998, Asian Technology Information Project report #ATIP98.022, Book #1, Selected Reports of Asian Technology Information Project.
64 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the 1990s," Seymour Goodman, et. al.
65 Ibid.
66 Ibid.
67 Ibid.
68 Advanced Computing Technologies in China, March 24, 1998, Asian Technology Information Project report #ATIP98.022, Book #1, Selected Reports of Asian Technology Information Project.
69 "High-Performance Computing, National Security Applications, and Export Control Policy at the Close of the 20th Century," Seymour Goodman, et. al.
70 "Quicklook Report for Military Trends Workshop," Strategic Assessment Center, Science Applications International Corporation, June 8, 1998.
71 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the 1990s," Seymour Goodman, et. al.
72 "Quicklook Report for Military Trends Workshop," Strategic Assessment Center, Science Applications International Corporation, June 8, 1998.
73 Commander in Chief, U.S. Pacific Command Memorandum to the Joint Chiefs of Staff, Ser: 444-98, November 9, 1998.
74 Export Administration Annual Report Fiscal Year 1991.
75 "Export Control on Supercomputers," Federal Register, May 18, 1992, Vol. 57, No. 96.
76 "Toward a National Export Strategy," Trade Promotion Coordinating Committee report, September 30, 1993, Chapter 5.
77 Export Administration Annual Report 1995 and 1996 on Foreign Policy Export Controls.
78 Federal Register, February 24, 1994.
79 Federal Register, April 4, 1994.
80 Code of Federal Regulations, p. 349-356.
81 "Export Controls: Information on the Decision to Revise High Performance Computer Controls," GAO/NSIAD-98-196, September 16, 1998.
82 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the 1990s," Seymour Goodman, et. al.
83 "Export Controls: Information on the Decision to Revise High Performance Computer Controls," GAO/NSIAD-98-196, September 16, 1998.
84 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the 1990s," Seymour Goodman, et. al.
85 "Export Controls: Information on the Decision to Revise High Performance Computer Controls," GAO/NSIAD-98-196, September 16, 1998.
86 Ibid.
87 Ibid.
88 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the 1990s," Seymour Goodman, et. al.
89 U.S. General Accounting Office staff interview of Paul Koenig et. al., November 24, 1997, and February 19, 1998.
90 Defense internal memorandum for the record from Paul Koening on "Supercomputing Study," August 24, 1995, and U.S. General Accounting Office staff interview of Paul Koenig, et.al., November 24, 1997.
91 "High Performance Computing Technical Assessment, Summary Report Findings," Institute for Defense Analyses, Technology Identification & Analyses Center, Prepared for: Principal Deputy Assistant Secretary of Defense (Dual-Use Technology Policy & International Programs), August 1995.
92 "Export Controls: Information on the Decision to Revise High Performance Computer Controls," GAO/NSIAD-98-196, September 16, 1998.
93 U.S. General Accounting Office staff interview of Oksana Nesterczuk, July 21, 1998.
94 "Supercomputer Technology Status and Trends Over the Next Two Years," June 2, 1995; "Global Supercomputer Industry and Market Assessment," Department of Commerce, Office of Strategic Industries and Economic Security, Economic Analysis Division, June 2, 1995.
95 U.S. General Accounting Office staff interview of Defense Technology Security Administration officials, November 23, 1997.
96 U.S. General Accounting Office staff interview of Defense Technology Security Administration officials, February 19, 1998.
97 Defense internal memorandum for the record from Paul Koening on "Supercomputing Study," August 24, 1995.
98 Ibid.
99 "High Performance Computing Technical Assessment, Summary Report Findings," Institute for Defense Analyses, Technology Identification & Analyses Center, Prepared for: Principal Deputy Assistant Secretary of Defense (Dual-Use Technology Policy & International Programs), August 1995.
100 Ibid.
101 Ibid.
102 Bandwidth (through-put) and latency (transmission delay) are crucial parameters of the interconnect between processors. The lower the bandwidth, the higher the latency, and the less scalable the interconnect, the more of a bottleneck the interconnect becomes. The more the interconnect is a bottleneck, the more "coarsely grained" an application must be to run effectively on the system. See "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the 1990s," Seymour Goodman, et. al.
103 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the 1990s," Seymour Goodman, et. al.
104 U.S. General Accounting Office staff interview of Institute for Defense Analyses and Office of the Secretary of Defense officials and written response.
105 Ibid.
106 U.S. General Accounting Office staff interview of Defense Technology Security Administration officials, July 21, 1998.
107 "Export Controls: Information on the Decision to Revise High Performance Computer Controls," GAO/NSIAD-98-196, September 16, 1998.
108 Ibid.
109 Interview of Mitchel Wallerstein, November 25, 1998.
110 "Export Controls: Information on the Decision to Revise High Performance Computer Controls," GAO/NSIAD-98-196, September 16, 1998.
111 Export Administration Regulations, January 25, 1996.
112 Ibid.
113 Ibid.
114 Ibid.
115 Ibid.
116 Export Administration Regulations, April 1998, Supplement No. 1 to Part 774.
117 Ibid.
118 Export Administration Regulations, April 1998, Supplement No. 1 to Part 774, Part 740.7.
119 Export Administration Regulations, April 1998, Supplement No. 1 to Part 774, Part 740.7, "Computers (CTP)."
120 Export Administration Regulations, April 1998, Supplement No. 1 to Part 742.12, Part 742.12, "High Performance Computers.’’
121 Ibid.
122 Ibid.
123 Export Administration Regulations, April 1998, Part 744, "Control Policy: End-User and End-Use Based."
124 Ibid.
125 Ibid.
126 Export Administration Regulations, April 1998, Part 772.
127 Export Administration Regulations, April 1998, Legal Authority, Section 12, "Enforcement," April 1998.
128 "High-Performance Computing, National Security Applications, and Export Control Policy at the Close of the 20th Century," Seymour Goodman, et. al.
129 Ibid.
130 Interview of Will Lowell, November 19, 1998; International Traffic in Arms Regulations, Section 120.
131 Ibid.
132 Ibid.
133 Ibid.
134 End-use screening is the process exporters follow to evaluate whether a transaction involves an unacceptable risk of use in, or diversion to, a proliferator or military end user. "Export Controls: Information on the Decision to Revise High Performance Computer Controls," GAO/NSIAD-98-196, September 16, 1998.
135 "Export Controls: Information on the Decision to Revise High Performance Computer Controls," GAO/NSIAD-98-196, September 16, 1998.
136 Export Administration Regulations, April 1998, Part 744, "Control Policy: End-User and End-Use Based."
137 "Export Controls: Information on the Decision to Revise High Performance Computer Controls," GAO/NSIAD-98-196, September 16, 1998.
138 Public Law 105-85.
139 "Export Controls: Information on the Decision to Revise High Performance Computer Controls," GAO/NSIAD-98-196, September 16, 1998.
140 Department of Defense Study, November 19, 1998, Enclosure 7, "Expected Developments."
141 Ibid.
142 Ibid.
143 Ibid.
144 Ibid.
145 Ibid.
146 PSVs are on-site visits, generally by U.S. government officials, to locations where goods are shipped for the purpose of ensuring that they have not been diverted to other locations. ("Export Controls: Information on the Decision to Revise High Performance Computer Controls," GAO/NSIAD-98-196, September 16, 1998.)
147 "Export Controls: Information on the Decision to Revise High Performance Computer Controls," GAO/NSIAD-98-196, September 16, 1998.
148 Ibid.
149 Arrangement between the United States and China for end-use visits.
150 Ibid..
151 Interview of Iain S. Baird, November 17, 1998.
152 Response to Committee questions from William Reinsch, James Lewis, and Mark Menefee, November 17, 1998.
153 Interview of William Reinsch, November 19, 1998.
154 Ibid.
155 "Export Controls: Information on the Decision to Revise High Performance Computer Controls," GAO/NSIAD-98-196, September 16, 1998.
156 Ibid.
157 Ibid.
158 Ibid.
159 Ibid.
160 U.S. General Accounting Office Staff Summary of Sun Microsystems HPC diversion, April 1998.
161 Ibid.
162 Ibid.
163 Ibid.
164 Ibid.
165 Ibid.
166 Ibid.
167 Ibid.
168 Export Administration Annual Report, Export Enforcement, March 1998.
169 Ibid.
170 Ibid.
171 Ibid.
172 Export Controls: Information on the Decision to Revise High Performance Computer Controls, GAO/NSIAD-98-196, September 16, 1998.
173 Memorandum for the Record, "Report on National Lab Briefings of September 28 and 29, 1998," Joint Staff, October 2, 1998.
174 Ibid.
175 The military uses are the design or development of nuclear, biological or chemical weapons; the design or development of weapons of mass destruction; the design or development of missile or rocket systems; and any cryptoanalytic or cryptographic purpose.
176 "High-Performance Computing, National Security Applications, and Export Control Policy at the Close of the 20th Century," Seymour Goodman, et. al.; Advanced Computing Technologies in China, March 24, 1998, Asian Technology Information Project report #ATIP98.022, Book #1, Selected Reports of Asian Technology Information Project.
177 Ibid.
178 "Computers in China/Korea/Hong Kong, ATIP, 97.048," Asian Technology Information Project Selected Reports, Book #1.
179 "U.S. Technology Transfers to the People’s Republic of China," by Kathleen Walsh, DFI International, dated December, 1997. This report was produced at the request of the Commerce Department’s Bureau of Export Administration. It was the first in a series of studies into technology transfers to the PRC commissioned by BXA.
180 "Computers in China/Korea/Hong Kong, ATIP, 97.048," page 10, Asian Technology Information Project Selected Reports, Book #1.
181 Ibid.
182 "U.S. Technology Transfers to the People’s Republic of China," by Kathleen Walsh, DFI International, dated December, 1997. This report was produced at the request of the Commerce Department’s Bureau of Export Administration. It was the first in a series of studies into technology transfers to the PRC commissioned by BXA.
183 "Computers in China/Korea/Hong Kong, ATIP, 97.048," Asian Technology Information Project Selected Reports, Book #1.
184 Ibid.
185 Ibid.
186 Advanced Computing Technologies in China, March 24, 1998, Asian Technology Information Project report #ATIP98.022, Selected Reports of Asian Technology Information Project, Book #1.
187 Ibid.
188 "High Performance Computing (HPC) in Japan (1)," May 16, 1997, Asian Technology Information Project report # ATIP97.044, Selected Reports of Asian Technology Information Project, Book #1.
189 Commerce Department, Bureau of Export Administration’s Export Control Automated Support System (ECASS).
190 Ibid.
191 These notifications included three machines for Hong Kong, a Tier 2 country, that did not need to be reported under the 1998 notification regulations. Their total value was about $256,000. Commerce Department, Bureau of Export Administration’s Export Control Automated Support System (ECASS).
192 Commerce Department, Bureau of Export Administration’s Export Control Automated Support System (ECASS).
193 Advanced Computing Technologies in China, March 24, 1998, Asian Technology Information Project report #ATIP98.022, Book #1, Selected Reports of Asian Technology Information Project.
194 "Computers in China/Korea/Hong Kong, ATIP, 97.048," page 10, Asian Technology Information Project Selected Reports, Book #1.
195 Three HPC exports for Hong Kong were included in the data for China.
196 "Computers in China/Korea/Hong Kong, ATIP, 97.048," Asian Technology Information Project Selected Reports, Book #1.
197 Ibid.
198 Michael Evans, a computer programmer for the Federal Bureau of Investigation with 13 years private sector programming experience, and 3 years criminal investigative experience regarding computer crimes, Memorandum For the Record dated, December 15, 1998.
199 "Arms Proliferation Policy, Support to the Presidential Advisory Board," RAND, 1996.
200 Ibid.
201 Ibid.
202 Ibid.
203 Ibid.
204 Ibid.
205 Interview of Bob Nesbit, October 6, 1998.
Technical Afterword Endnotes
1 "The Round Table on Computer Performance Metrics for Export Control: Discussions and Results," Alfred E. Brenner, Task leader, and Norman R. Howes. Document of the Institute For Defense Analyses, December 1997. IDA Document D-2116.
2 "NHSE Review 1996 Volume," 1996, Aad J. van der Steen and Jack J. Dongarra.
3 "Practical Parallel Computing (Physics 500)," Lecture Series, University of New Mexico, Fall 1995, Dr. Michael Warren.
4 Memorandum for the Record, Joint Staff Interview of Dr. David Kahaner, October 19, 1998.
5 Ibid.
6 Memorandum for the Record, Joint Staff Interview of Dr. Jeff Hollingsworth, November 5, 1998.
7 Interview of Tom Dunn by Joint Staff, October 29, 1998.
8 Ibid.
9 Ibid.
10 "Practical Parallel Computing (Physics 500)," Lecture Series, University of New Mexico, Fall 1995, Dr. Michael Warren.
11 Memorandum for the Record, Joint Staff Interview of Dr. David Kahaner, October 19, 1998.
12 Interview of Dr. Jack Dongarra, by U.S. General Accounting Office, February 6, 1998.
13 "Practical Parallel Computing (Physics 500)," Lecture Series, University of New Mexico, Fall 1995, Dr. Michael Warren.
14 "Off-the-Shelf Chips Conquer The Heights of Computing," Science, September 8, 1995, American Association for the Advancement of Science, Robert Pool.
15 Defense Intelligence Agency Study, "Distribution of U.S. and Allied High Performance Computers . . . ," November 19, 1998.
16 Interview of Dr. Jack Dongarra, by U.S. General Accounting Office, February 6, 1998.
17 Memorandum for the Record, Joint Staff, November 2, 1998 DoD HPC briefing.
18 Defense Intelligence Agency Study, "Distribution of U.S. and Allied High Performance Computers . . . ," November 19, 1998.
19 "Off-the-Shelf Chips Conquer The Heights of Computing", Science, September 8, 1995, American Association for the Advancement of Science, Robert Pool.
20 Defense Intelligence Agency Study, "Distribution of U.S. and Allied High Performance Computers . . . ," November 19, 1998.
21 "Off-the-Shelf Chips Conquer The Heights of Computing", Science, September 8, 1995, American Association for the Advancement of Science, Robert Pool.
22 Defense Intelligence Agency Study, "Distribution of U.S. and Allied High Performance Computers . . . ," November 19, 1998.
23 "ASCI Blue Pacific can bring IBM on first position in TOP500", Primeur Analysis, October 28, 1998, Ad Emmen.
24 "Teraflop" is a term used to describe a trillion floating point operations a second. "Terabyte refers to a trillion bytes of data.
25 "ASCI Blue Pacific can bring IBM on first position in "TOP500," Primeur Analysis, October 28, 1998, Ad Emmen.
26 Ibid.
27 "Lawrence Livermore National Laboratory Presentations on China, Nuclear Weapons and High Performance Computing" September 15, 1998.
28 Defense Intelligence Agency Study, "Distribution of U.S. and Allied High Performance Computers . . . ," November 19, 1998.
29 Ibid.
30 "Fitting Architecture to Application: Choosing between SMP and RACE," Mercury Computer Systems, Inc., June 17, 1996.
31 Defense Intelligence Agency Study, "Distribution of U.S. and Allied High Performance Computers . . . ," November 19, 1998.
32 An approximation is that the composite theoretical performance rating in MTOPS is about 75 percent the value in MFLOPS. Defense Intelligence Agency Study, "Distribution of U.S. and Allied High Performance Computers . . . ," November 19, 1998.
33 "Off-the-Shelf Chips Conquer The Heights of Computing", Science, September 8, 1995, American Association for the Advancement of Science, Robert Pool.
34 Ibid.
35 Defense Intelligence Agency Study, "Distribution of U.S. and Allied High Performance Computers . . . ," November 19, 1998.
36 "Fitting Architecture to Application: Choosing between SMP and RACE", Mercury Computer Systems, Inc., June 17, 1996.
37 Defense Intelligence Agency Study, "Distribution of U.S. and Allied High Performance Computers . . . ," November 19, 1998.
38 "Lawrence Livermore National Laboratory Presentations on China, Nuclear Weapons and High Performance Computing," September 15, 1998.
39 Ibid.
40 Ibid.
41 Ibid.
42 Ibid.
43 Ibid.
44 Ibid..
45 "The Round Table on Computer Performance Metrics for Export Control: Discussions and Results," Institute For Defense Analyses, December 1997.
46 Ibid.
47 Interview of Dr. Jack Dongarra by U.S. General Accounting Office, February 6, 1998.
48 "Off-the-Shelf Chips Conquer The Heights of Computing," Science, September 8, 1995, American Association for the Advancement of Science, Robert Pool.
49 Interview of Dr. Jack Dongarra, by U.S. General Accounting Office, February 6, 1998.
50 "Practical Parallel Computing (Physics 500)," Lecture Series, University of New Mexico, Fall 1995, Dr. Michael Warren.
51 Ibid.
52 Ibid.
53 Ibid.
54 Ibid.
55 Ibid.
56 Ibid.
57 Ibid.
58 Http://www.phys.unm.edu
59 "Building on the Basics: An Examination of High-Performance Computing Export Control Policy in the 1990s," Seymour Goodman, et. al.
60 "High-Performance Computing Technologies: Where We Are Today," Jack Dongarra, University of Tennessee and Oak Ridge National Laboratory.
61 Ibid.
62 "Off-the-Shelf Chips Conquer The Heights of Computing," Science, September 8, 1995, American Association for the Advancement of Science, Robert Pool.
63 University of Mannheim and Netlib (a repository maintained by Bell Laboratories, Oak Ridge National Laboratory, and the University of Tennessee to serve the computer community) publish statistics on HPCs to, among other things, provide a better understanding of the HPC market.
64 "Top500 Supercomputer List at Mannheim University and Netlib," "Slides about Top500," November 1998, Netlib.
65 "Off-the-Shelf Chips Conquer The Heights of Computing," Science, September 8, 1995, American Association for the Advancement of Science, Robert Pool.
66 Ibid.
67 "Logic Design for Array-Based Circuits," Dr. Donnamaie E. White, September 11, 1996.
68 "Top500 Supercomputer List at Mannheim University and Netlib," "Slides about Top500," November, 1998, Netlib.
69 "High-Performance Computing, National Security Applications, and Export Control Policy at the Close of the 20th Century," Seymour Goodman, et. al.
70 Ibid.
71 Ibid.
72 Interview of Dr. Jeff Hollingsworth by Joint Staff, November 5, 1998.
73 Ibid.
74 Ibid.
75 Ibid.