International Council for Science - World Data Center System

Chapter 1. INTRODUCTION

Scientific data gathering has a long history. Information about solar and auroral activity in past millennia was chronicled by the Chinese and other peoples. In the Western world, systematic geophysical measurements extend back for centuries, but mechanisms for data distribution and exchange are more recent. In the 18th and 19th Centuries, data were exchanged from the early geomagnetic and seismic observatories largely through publication of annual station books. Oceanographic and geological data were recorded in expedition reports. Our knowledge of the geomagnetic field, plate tectonics and ocean currents owe much to these records, even though there were no convenient ways to copy the originals.

The first large-scale international scientific enterprises were the International Polar Years of 1882-1883 and 1932-1933, and eventually led to the International Geophysical Year of 1957-1958. Planning of the IGY was coordinated by CSAGI, the Special Committee for the IGY set up by the International Council of Scientific Unions. CSAGI established the World Data Center system to serve the IGY, and developed data management plans for each IGY scientific discipline. The data specifications were published in a series of Guides to Data Exchange, originally issued in 1957 and updated in 1963, 1973, 1979 and 1987. The IGY planners were remarkably prescient: the 1955 recommendation mentioned that Data Centers should be prepared to handle data in machine-readable form, which at that time meant punched cards and punched tape.

National IGY Committees were invited to establish and operate World Data Centers at national expense, abiding by the CSAGI rules. The U.S.A. and U.S.S.R. established complex Centers, known respectively as WDCA and WDCB, to serve most IGY disciplines. In many disciplines there was a third or even a fourth Center, known as WDCC1 if in Western Europe and WDCC2 if in Asia or Australia (the European Centers being known simply as WDC-C if there was no corresponding WDC-C2). Multiple Centers were deemed advisable to guard against catastrophic loss of data, and for the convenience of data providers and users.

Because of its success, the WDC system was made permanent and used for post-IGY data. New programs evolved, based on the IGY structure as a general framework, such as the International Quiet Sun Year of 1964-1965, the International Magnetospheric Study of 1976-1979, the Solar Maximum Year of 1979-1981 and the Middle Atmosphere Program of 1982-1985. Most of the sponsoring national bodies agreed to continue the WDCs to serve these programs, and the data collections have remained accessible to users.

Since the IGY, the gathering and exchange of data has been transformed by technological advances, such as the replacement of analog with digital instruments, the networking of digital instruments to simplify collection and exchange of data, and unstaffed automatic observatories. Personal computers and compact disc readers are ubiquitous. Many WDCs publish collections of digital data sets on compact discs for easy distribution. Digital communication networks make it possible to transfer large data files by electronic mail, reducing much of the routine work of WDC staff, who are now largely engaged in developing new data compilations and new tools for data display and analysis. The environmental WDC disciplines make use of map-based data and information on natural features and human activities that differ in character from the numerical data sets of the older disciplines and require new analysis techniques.

Over the years the tally of WDCs has changed because of scientific, technical and economic factors. Solar-terrestrial physics developed into a unified discipline embracing many IGY subjects. Some specialized topics have declined in importance, and the WDC system has expanded into the environmental and Earth resource fields. A comprehensive set of WDC-D was established in China in 1988. WDCA in the U.S.A. has expanded; many of its discipline Centers are collocated with national data Centers. WDCB in Russia is now operated by three different organizations. Some of the C, C1 and C2 Centers in Europe and Asia have moved or have closed (especially those that depended on the expertise of a particular research group), but new Centers have opened. All Centers now have computer facilities and most use electronic networks to meet requests, exchange catalog information and transfer data.

Today the WDC system is healthy and viable. Most Centers are maintaining their funding, though not without struggle. Data acquisition, storage and distribution are expensiveWDCs cost money, but they are cost-effective in transferring data to users, and their operational costs represent a tiny fraction of worldwide scientific activity. The ICSU Panel on World Data Centers hopes that this Guide will provide a useful overview of the system. Users should read Chapter 2, especially section F, Using the WDC System, in conjunction with the information on individual WDCs in Chapters 3.