Transferring Files

This section explains how to transfer files between your local computer and GridUnesp.

Transferring to GridUnesp

Linux/Unix/macOS (command line)

Transfer file to GridUnesp
scp local_file.dat your_username@access.grid.unesp.br:/home/your_username/

Example:

scp experiment_data.tar.gz john@access.grid.unesp.br:/home/john/

You will be prompted to enter your password.

Windows (WinSCP)

Use WinSCP as configured in Accessing the Cluster. Simply drag files from the local window (left) to the remote window (right).

Warning

Fail2Ban Limitation:

Multiple transfers via scp in a short interval may block your access for 15 minutes. To transfer many files:

  1. Compress everything into a single file:

    tar -czf complete_data.tar.gz folder_with_data/
scp complete_data.tar.gz user@access.grid.unesp.br:~/

  2. Use rsync (more efficient for directories):

    rsync -avz data_folder/ user@access.grid.unesp.br:~/data_folder/

Transferring from GridUnesp

Linux/Unix/macOS (command line)

Transfer file from GridUnesp
scp your_username@access.grid.unesp.br:/home/your_username/remote_file.dat .

The dot (.) at the end indicates the current directory on your local computer.

Example:

scp john@access.grid.unesp.br:/home/john/results.tar.gz ./

Windows (WinSCP)

Select the file on the remote side (right) and drag it to the desired folder on the local side (left).

Data Care

Danger

GridUnesp does NOT have an automatic backup system.

  • Data lost due to hardware failure cannot be recovered

  • You are solely responsible for the integrity of your data

Recommendations:

  1. Transfer important results to your local computer as soon as possible

  2. Keep only what is necessary on the cluster for current/future processing

  3. Regularly clean up temporary files and old results

  4. Make external backups of critical data

Tip

To check which files are taking up the most space:

du -sh ~/* | sort -hr | head -20

See Data Maintenance for more details.

Command Summary

Useful commands for file transfer

Command

Description

scp local.dat user@access.grid.unesp.br:~/

Send local file to GridUnesp

scp user@access.grid.unesp.br:~/remote.dat .

Download file from GridUnesp

rsync -avz folder/ user@access.grid.unesp.br:~/folder/

Synchronize directory (upload)

rsync -avz user@access.grid.unesp.br:~/folder/ ./folder/

Synchronize directory (download)

tar -czf dados.tar.gz folder/

Compress directory before transferring

See also