====== Transferring data into the archive ======
----
<typo fc:#FF0000>Please note</typo>: The archive is operated as part of the service [[https://www.luis.uni-hannover.de/archivierung.html|Archivierung]] and thus not part of the cluster system.

The archive can be used to store results and simulation data permanently. Each account has to be registered for
archive use, before using it. This can be done on the [[https://bias.luis.uni-hannover.de/|BIAS-website]] after logging in with your user name and
password. After clicking on the link entitled //Ihren Benutzernamen für die Nutzung des Archivsystems zulassen//
it takes roughly an hour, before the archive can be used.

===== Quota =====

Archival storage in the archive system of Leibniz Universität Hannover is controlled by a quota mechanism.
There is a quota on the amount of files as well as storage space. Please see the website of the archive service
for further details at http://www.luis.uni-hannover.de/archivierung.html.

===== Transferring data into the archive =====

In order to transfer data into the archive of Leibniz Universität Hannover, it is recommended to use the cluster’s dedicated transfer node, see [[guide:connecting_and_data_transfer#file_transfer_using_linux_or_mac_os|section]].

===== Login with lftp =====

The archive can be reached at ''archiv.luis.uni-hannover.de''  using the ''lftp''  command.
<code>
username@transfer:~$ lftp <username>@archiv.luis.uni-hannover.de
</code>

After entering your cluster user name’s password the ''lftp''  prompt appears.

<code>
lftp <username>@archiv.luis.uni-hannover.de:~>
</code>

Now you can use the ''ls''  command to list your directory contents at the archive. At the same time this is to test an established connection to the archive.

<code>
 lftp <username>@archiv.luis.uni-hannover.de:~> ls
</code>

At your first login to the archive system with your account the directory is empty. The ''ls''  command will not return any listing. You can terminate the connection with ''exit''.

<code>
lftp <username>@archiv.luis.uni-hannover.de:~> exit
<username>@transfer:~$
</code>

Aliases for ''exit''  are ''quit''  and ''bye''.

===== Copying files into the archive =====

On the cluster system’s transfer node change to the directory where the data to be copied are located.

<code>
username@transfer:~$ cd $BIGWORK/my_data_dir
username@transfer:/bigwork/username/my_data_dir$
</code>

After logging in using lftp the ''put''  command is used.

<code>
username@transfer:/bigwork/username/my_data_dir$ lftp
<username>@archiv.luis.uni-hannover.de:~>
lftp <username>@archiv.luis.uni-hannover.de:~> put myfile.tar.gz
</code>

The file ''myfile.tar.gz''  is located inside the directory we previously changed to in this example. After using ''put''  to transfer the file it is also available on the archive. The ''TAB''  key works for completing file and directory names in lftp as well.

Saving multiple small files in the archive is not desired, because at least one copy of the data are kept on magnetic tape. Therefore a constant stream of data is desirable which can be achieved by some large files. It is recommended to use ''tar''  or ''zip''  to combine small files into one bigger file. This can also optimize your quota.

In order to transfer multiple (large) files at once, the ''mput''  command can be used. This is short for //multiple put//. The ''mput''  command understands the wildcard ''*''  as it is used in bash.

<code>
lftp <username>@archiv.luis.uni-hannover.de:~> mput mydata*.tar.gz
</code>

===== Fetching files from the archive =====

In order to get fetch files from the archive, the ''get''  command can be used.

<code>
lftp <username>@archiv.luis.uni-hannover.de:~> get myfile.tar.gz
</code>

This command puts the file at the location the ''lftp''  command was issued from which transferred the file into the archive. For fetching more than one file the ''mget''  command can be used (//multiple get//). Fetching the file may take some time until transfer starts. This time is needed by the storage robot to find the respective magnetic tape and wind the tape to the position the file is located at.

===== Some useful commands =====

Listing the current directories’ contents can be achieved by the command ''!ls''. An exclamation mark executes the command on the machine lftp was started on. On the contrary listing the current //local//  directory can be done with ''lpwd''  at the lftp prompt.

It is possible to create directories in the archive using the ''mkdir''  command.

<code>
lftp <username>@archiv.luis.uni-hannover.de:~> mkdir myDir
</code>

Changing directories works in the usual way using ''cd''.

<code>
lftp <username>@archiv.luis.uni-hannover.de:~> cd myDir
</code>

And back up one directory.

<code>
lftp <username>@archiv.luis.uni-hannover.de:~> cd ..
</code>

A local directory can be changed using the ''lcd''  command, short for //local cd//.

<code>
lftp <username>@archiv.luis.uni-hannover.de:~> lcd /bigwork/<username>/datadir
</code>

===== Further reading =====

  * man page lftp, '' man lftp''. Navigate using the arrow keys and exit with 'q'
  * Service [[http://www.luis.uni-hannover.de/archivierung.html|Archivierung]]