Please note: The archive is operated as part of the service Archivierung and thus not part of the cluster system.
The archive can be used to store results and simulation data permanently. Each account has to be registered for archive use, before using it. This can be done on the BIAS-website after logging in with your user name and password. After clicking on the link entitled Ihren Benutzernamen für die Nutzung des Archivsystems zulassen it takes roughly an hour, before the archive can be used.
Archival storage in the archive system of Leibniz Universität Hannover is controlled by a quota mechanism. There is a quota on the amount of files as well as storage space. Please see the website of the archive service for further details at http://www.luis.uni-hannover.de/archivierung.html.
In order to transfer data into the archive of Leibniz Universität Hannover, it is recommended to use the cluster’s dedicated transfer node, see section.
The archive can be reached at
archiv.luis.uni-hannover.de using the
username@transfer:~$ lftp <username>@archiv.luis.uni-hannover.de
After entering your cluster user name’s password the
lftp prompt appears.
Now you can use the
ls command to list your directory contents at the archive. At the same time this is to test an established connection to the archive.
lftp <username>@archiv.luis.uni-hannover.de:~> ls
At your first login to the archive system with your account the directory is empty. The
ls command will not return any listing. You can terminate the connection with
lftp <username>@archiv.luis.uni-hannover.de:~> exit <username>@transfer:~$
On the cluster system’s transfer node change to the directory where the data to be copied are located.
username@transfer:~$ cd $BIGWORK/my_data_dir username@transfer:/bigwork/username/my_data_dir$
After logging in using lftp the
put command is used.
username@transfer:/bigwork/username/my_data_dir$ lftp <username>@archiv.luis.uni-hannover.de:~> lftp <username>@archiv.luis.uni-hannover.de:~> put myfile.tar.gz
myfile.tar.gz is located inside the directory we previously changed to in this example. After using
put to transfer the file it is also available on the archive. The
TAB key works for completing file and directory names in lftp as well.
Saving multiple small files in the archive is not desired, because at least one copy of the data are kept on magnetic tape. Therefore a constant stream of data is desirable which can be achieved by some large files. It is recommended to use
zip to combine small files into one bigger file. This can also optimize your quota.
In order to transfer multiple (large) files at once, the
mput command can be used. This is short for multiple put. The
mput command understands the wildcard
* as it is used in bash.
lftp <username>@archiv.luis.uni-hannover.de:~> mput mydata*.tar.gz
In order to get fetch files from the archive, the
get command can be used.
lftp <username>@archiv.luis.uni-hannover.de:~> get myfile.tar.gz
This command puts the file at the location the
lftp command was issued from which transferred the file into the archive. For fetching more than one file the
mget command can be used (multiple get). Fetching the file may take some time until transfer starts. This time is needed by the storage robot to find the respective magnetic tape and wind the tape to the position the file is located at.
Listing the current directories’ contents can be achieved by the command
!ls. An exclamation mark executes the command on the machine lftp was started on. On the contrary listing the current local directory can be done with
lpwd at the lftp prompt.
It is possible to create directories in the archive using the
lftp <username>@archiv.luis.uni-hannover.de:~> mkdir myDir
Changing directories works in the usual way using
lftp <username>@archiv.luis.uni-hannover.de:~> cd myDir
And back up one directory.
lftp <username>@archiv.luis.uni-hannover.de:~> cd ..
A local directory can be changed using the
lcd command, short for local cd.
lftp <username>@archiv.luis.uni-hannover.de:~> lcd /bigwork/<username>/datadir
man lftp. Navigate using the arrow keys and exit with 'q'