Providing data for the GBDC

In order to create indices and summary plots from ground-based data, the GBDC will need to be supplied with data from a whole of range of disparate sources. The only way in which this will be feasible is if as much of the data transfer as possible is done automatically.

This page gives a brief description of how we intend such data transfers to be carried out.

Preliminaries

Arrange with the GBDC (gbdc_support@ukssdc.ac.uk) to set up a data retrieval system for your data. This will involve the creation of a mail alias to which either data or a message triggering an automatic retrieval program can be sent. The mail address to which messages from projectx will be sent is of the form
projectx@wdcc1a.bnsc.rl.ac.uk
For example, the automatic retrieval of data from IMAGE will be triggered by a message sent to
image@wdcc1a.bnsc.rl.ac.uk
We will always need to be told the data format(s) of any files to be transferred.
If there is any software available for manipulating your data, such software may be useful to us. We would need to liaise with you to decide what software might be appropriate for us to have.

Data Transfer

Once the arrangements described above have been made, we are ready for data to be transferred. The precise mechanism for transferring data is negotiable, provided it is such that it can be easily automated. We have already implemented two methods which should cover most scenarios, and we would obviously prefer it if one of these could be used.

Direct Email

If the data files to be transferred are small (<5K) we can arrange for you to mail them directly to the project alias on our machine.

Anonymous FTP triggered by Email

For larger files, or for a large number of relatively small files, we would prefer anonymous ftp to be used. A mail message with a list of files ready to be transferred will trigger an automatic retrieval program. This mail message should be of the form:
reply person@place ftp binary ftp://machine1.name/pub/data/filename1.dat ftp://machine1.name/pub/data/filename2.dat ftp://machine2.name/pub/data/filename3.dat

The line
reply person@place
is an optional line for acknowledgements other than to the sender of the original message.

The line
ftp binary
indicates the ftp transfer mode. The keyword can be either 'binary' or 'ascii'. If the keyword is omitted, then the default of the machine providing the data will be used.

Lines of the form
ftp://machine1.name/pub/data/filename1.dat
are URLs of files to be retrieved by anonymous ftp. Note that files may be on arbitrary machines and in arbitrary directories, but must have unique names. We suggest that filenames identify both the project and the time period, e.g. image_950307. If a suffix is used it could be to identify distinct stations or file formats.

For example, to trigger the retrieval of several files of IMAGE magnetometer data, one could create a file trigger.txt containing
reply hakkinen@fmi.fi ftp ascii ftp://sumppu.fmi.fi/pub/data/image/cluster/image_950101.iaga ftp://sumppu.fmi.fi/pub/data/image/cluster/image_950102.iaga ftp://sumppu.fmi.fi/pub/data/image/cluster/image_950103.iaga ftp://sumppu.fmi.fi/pub/data/image/cluster/image_950104.iaga

Then executing
mail image@wdcc1a.bnsc.rl.ac.uk < trigger.txt
(assuming this is done under UNIX) will trigger the automatic retrieval by anonymous ftp of the listed files; an acknowledgement message indicating the success or failure of the ftp will be sent to hakkinen@fmi.fi

If you need to get in touch with us about data transfer arrangments, please send email to gbdc_support@ukssdc.ac.uk.

GBDC Home | Data

12-FEB-2001 Richard Stamper