NSF logo [logo: ITRD Assessment of High-End Computing Research and Development in Japan [logo: DOE] Nasa logo ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Final report The final report on high-end computing in Japan is available in PDF format. ( ~ 11.7 MB ) Individual chapters are also available. * Cover, Frontmatter and Preface * Executive Summary * Ch. 1 - Introduction * Ch. 2 - The Earth Simulator * Ch. 3 - Policy Considerations That Influence HEC Development in Japan * Ch. 4 - Scientific Applications of High-End Computing I * Ch. 5 - Scientific Applications of High-End Computing II * Ch. 6 - Architecture Overview * Ch. 7 - Software for High-End Computing * Ch. 8 - Grid Computing in Japan * Appendices * A. Biographies * B. Site Reports * C. Top500 update * D. U.S. Workshop Summary * E. Glossary Workshop Viewgraphs presented at the May 25, 2004 workshop are available below * Context of the HEC Study by Dr. R. D. Shelton * Executive Summary by Al Trivelpiece * WTEC Panel on High End Computing in Japan by Al Trivelpiece * The Earth Simulator by Jack Dongarra * Scientific Applications by Rupak Biswas * Scientific Applications II by Peter Paul * Architecture by Jack Dongarra * Software & Grids by Kathy Yelick * High End Computing Revitalization Task Force by Dave Nelson Scope While the Earth Simulator itself is fairly well known, the main task for this study is to determine what comes after the Earth Simulator in Japan. The WTEC panelists will address this task by gathering information and analyses from a variety of sources: (a) their own existing knowledge of and insights into U.S. and Japanese HEC R&D activities; (b) a review of the published literature and any "gray literature" reports that can be found on the subject; (c) correspondence with U.S. and Japanese colleagues, as part of a careful process of preparation for the Japan site visits; (d) a week of site visits in Japan, following several months of careful preparations for those visits; (e) a workshop at which the panelists present their preliminary findings and receive comments on those findings from sponsors, U.S. colleagues, and invited Japanese site visit hosts and, (a) reviews of the panel's draft site reports and draft final report by the sponsors and Japanese hosts. After collecting all this information, the WTEC expert panelists add their own analyses and report their findings back to the U.S. government sponsors in the most timely manner possible. The details of the study scope will be agreed on at the study kickoff meeting, in consultation with both panelists and sponsors. A preliminary list of issues to be covered might include the following: Technical Issues: * Review of likely future Japanese machines, and of long-term Japanese research activities in the field such as the machine Fujitsu is building for the Japanese National Aerospace Laboratory (NAL). * What innovations in architecture, hardware, and software systems are under development for the next generation of Japanese high end computing systems? NEC's R&D and special architectural approaches in supercomputers played a key role to the success of the Earth Simulator (ES). What are the real, perhaps under-publicized, technical innovations at NEC and other Japanese supercomputer vendors? What are their current research themes? What are the technical barriers that NEC and other Japanese vendors feel must be overcome to achieve the next level of performance? * Overall comparative assessment of Japanese high end computing research, development, and deployment programs relative to those in the U.S. * Efforts in the software area, including open source operating systems and programming environments (e.g., Linux) and models of software support for architectures aimed at achieving high memory-to-CPU bandwidth. What is the Japanese model for developing and maintaining software capabilities? Topics to be covered in this area could include programming languages and environments, compilers, debuggers, libraries, and run-time tools. * Conception, design, and implementation of the Earth Simulator. The research and development plan for ES was openly publicized and well known to the international community. Our fact-finding mission will focus on how the ES team executed the plan, what technical hurdles were encountered, and how they were overcome. What was the strategy behind decisions that were made concerning Internet access, data storage availability, etc.? * Implications of the Earth Simulator and likely follow-on machines for other competing architectural designs and machines in Japan (and the U.S.), including, for example, cluster systems. With the NEC success, are companies like Hitachi and Fujitsu likely following suit, or are they moving in other directions? * How do the Japanese evaluate total cost of ownership of these high end computing systems, in terms of how efficiently they are using scientists' time, dollars per code execution, etc.? What are the appropriate benchmarks for comparing these machines? * Impacts of the requirements of applications areas like climate research, nanotechnology, atomic energy, and biotechnology on the future of high end computing. * Current and planned applications of the Earth Simulator. The Earth Simulator was originally announced as a tool for research in Earth sciences, and is now being used for that purpose. It is especially suited for simulating complex linked systems, for example, linked global atmosphere and ocean models. Recent government programs, however, have gone beyond the application in Earth sciences. How would ES open up and possibly change the research outlook for other computational sciences, such as biology, genomics, medicine, nanotechnology, or even human or artificial intelligence? How are the Japanese industries making the best use of the ES for research and commercial applications? What has been the payoff so far in terms of science and engineering advancements enabled by the Earth Simulator? * Experiences of the users - do they support investment in more powerful parallel vector supercomputers? What have been the experiences of U.S. or other foreign scientists in using the Earth Simulator? Non-technical Issues * R&D funding for the Earth Simulator project and other Japanese supercomputer projects in progress (e.g., the Petaflop Protein Explorer launched last year and the Fujitsu machine for NAL) in general, compared to the United States. How strong is Japanese government support for the Earth Simulator and other related programs? * Overall government strategy for supercomputing R&D and applications development - does Japan have an industrial policy for developing and maintaining its domestic supercomputing hardware or software development capabilities, and if so, what is it? Is there a Japanese road map for supercomputer development? * Opportunities for collaboration between the U.S. and Japan. * Management of the Earth Simulator's development and other high performance computing projects, especially the interaction among government, industry, and academe. How do interactions and collaborations among these groups involved in the Earth Simulator project compare to other major Japanese government projects involving shared funding and use of such resources? * Implications of the Earth Simulator and other advanced Japanese machines in future supercomputing R&D, manufacture, and world markets. * Roles of industry and academia in research, technology development, and commercialization of high end computing. * Long-range research, educational, and infrastructure issues that need to be addressed to promote better U.S. progress in the field, and to regain U.S. world leadership. Panelists Biographies [photo: Al Trivelpiece] Al Trivelpiece (panel chair) * Former director of the Oak Ridge National Laboratory * 14 Wade hampton Trail Henderson, NV 89052 * (702) 492 1602 * awt511 at cox.net Rupak Biswas Rupak Biswas * NASA Ames Research Center Mail Stop T27A-1 Moffett Field, CA 94035 * E-mail: rupak.biswas at nasa.gov * Office: 650-604-4411 * Fax: 650-604-3957 Peter Paul Peter Paul * Deputy Director for Science and Technology, Brookhaven National Laboratory * Brookhaven National Laboratory Upton, NY 11973-5000 * 631-344-8623 * ppaul at bnl.gov [photo: Jack Dongarra] Jack Dongarra * Director, Innovative Computing Laboratory * Director, Center for Information Technology Research * Computer Science Department 1122 Volunteer Blvd University of Tennessee Knoxville, TN 37996-3450 * E-mail: dongarra at cs.utk.edu * Office: 865-974-8295 * Home: 865-481-8295 * Fax: 865-974-8296 Kathetine Yelick Katherine Yelick * Professor * 777 Soda Hall Computer Science Division University of California at Berkeley Berkeley, CA 94720-1776 * Phone: (510)642-8900 * Fax: (510)642-5775 * yelick at cs.berkeley.edu Advisor to the Panel [photo: Dan Reed] Dan Reed * Former Director, National Center for Supercomputing Applications, University of Illinois * Former Director, NSF PACI National Computational Science Alliance * Dept of CS Sitterson Hall Campus Box 3175 Univ NC Chapel Hill 27599 * Email: reed at unc.edu ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________