.. _transferring_files:

==================
Transferring Files
==================

.. contents:: In this section:
   :local:
   :depth: 2

This section explains how to transfer files between your local computer and GridUnesp.

.. _transferring_to_gridunesp:

Transferring to GridUnesp
=========================

Linux/Unix/macOS (command line)
-------------------------------

.. code-block:: bash
   :caption: Transfer file to GridUnesp

   scp local_file.dat your_username@access.grid.unesp.br:/home/your_username/

**Example:**

.. code-block:: bash

   scp experiment_data.tar.gz john@access.grid.unesp.br:/home/john/

You will be prompted to enter your password.

Windows (WinSCP)
----------------

Use WinSCP as configured in :ref:`accessing_the_cluster`. Simply drag files from the local window (left) to the remote window (right).

.. warning::
   **Fail2Ban Limitation:**
   
   Multiple transfers via ``scp`` in a short interval may block your access for 15 minutes. To transfer many files:
   
   1. **Compress** everything into a single file:

      .. code-block:: bash

         tar -czf complete_data.tar.gz folder_with_data/
         scp complete_data.tar.gz user@access.grid.unesp.br:~/

   2. Use **rsync** (more efficient for directories):

      .. code-block:: bash

         rsync -avz data_folder/ user@access.grid.unesp.br:~/data_folder/

.. _transferring_from_gridunesp:

Transferring from GridUnesp
===========================

Linux/Unix/macOS (command line)
-------------------------------

.. code-block:: bash
   :caption: Transfer file from GridUnesp

   scp your_username@access.grid.unesp.br:/home/your_username/remote_file.dat .

The dot (.) at the end indicates the current directory on your local computer.

**Example:**

.. code-block:: bash

   scp john@access.grid.unesp.br:/home/john/results.tar.gz ./

Windows (WinSCP)
----------------

Select the file on the remote side (right) and drag it to the desired folder on the local side (left).

Data Care
=========

.. danger::
   **GridUnesp does NOT have an automatic backup system.**

   - Data lost due to hardware failure **cannot be recovered**
   - You are solely responsible for the integrity of your data

**Recommendations:**

1. **Transfer important results** to your local computer as soon as possible
2. **Keep only what is necessary** on the cluster for current/future processing
3. **Regularly clean up** temporary files and old results
4. **Make external backups** of critical data

.. tip::
   To check which files are taking up the most space:

   .. code-block:: bash

      du -sh ~/* | sort -hr | head -20

   See :ref:`data_maintenance` for more details.

Command Summary
===============

.. list-table:: Useful commands for file transfer
   :header-rows: 1
   :widths: 65 35

   * - Command
     - Description
   * - ``scp local.dat user@access.grid.unesp.br:~/``
     - Send local file to GridUnesp
   * - ``scp user@access.grid.unesp.br:~/remote.dat .``
     - Download file from GridUnesp
   * - ``rsync -avz folder/ user@access.grid.unesp.br:~/folder/``
     - Synchronize directory (upload)
   * - ``rsync -avz user@access.grid.unesp.br:~/folder/ ./folder/``
     - Synchronize directory (download)
   * - ``tar -czf dados.tar.gz folder/``
     - Compress directory before transferring

.. seealso::

   - :ref:`data_maintenance` - Space management
   - :ref:`best_practices` - Usage recommendations
   - :ref:`troubleshooting` - Transfer problems