Frequently Asked Questions

In this section:

Accessing GridUnesp
File Transfer
- 10) How do I transfer files to my account?
- 11) How do I transfer files from GridUnesp to my computer?
Installing Programs
- 12) I don’t have administrator permission. How do I install software?
- 13) How do I install a specific piece of software?
Job Processing
Job Monitoring
Processing Failures
Job Optimization
Best Practices
Other Questions

This section compiles answers to the most common questions from GridUnesp users. Questions are organized by category for easy reference.

Tip

Use your browser’s search function (Ctrl+F) to quickly locate a specific term.

Accessing GridUnesp 

1) How do I get help using GridUnesp?

Answer

Contact the GridUnesp specialist team by sending a message to support.ncc@unesp.br.

For problems with jobs, include in your message:

Location of the submission script (full path: /home/$USER/path/to/script.sh)
Location of the input files
Log files (slurm-JOBID.out)
Command used to submit the job
The job number(s) in question

2) How do I log in to GridUnesp?

Answer

Linux/Mac:

ssh username@access.grid.unesp.br

Windows:

Use PuTTY or WinSCP. See Accessing the Cluster for detailed instructions.

3) I forgot my password. How do I reset it?

Answer

Go to the page:

https://www.ncc.unesp.br/password-new/

Enter your username or registered e-mail. A password reset token will be sent to your e-mail.

Recommendations for a strong password:

Minimum of 8 characters
Uppercase and lowercase letters
Numbers and symbols (!, $, %, @, #, …)
Do not use common words

Tip

Use a random password generator: https://www.lastpass.com/features/password-generator

4) “The authenticity of host… can’t be established” — What does this mean?

Answer

This is the first time you are accessing the server. SSH is asking whether you recognize the server as trusted.

Check whether the fingerprint matches one of those below and type yes:

RSA:     SHA256:X3iCb13fWj7u2Tvp/MCCpn0brfSNS5Ie6ehm6lsvPSQ
ECDSA:   SHA256:WVFokXOLnuH9+2e7xxRU2gp7XJqxwuE6H8bbUMqTCXo
ED25519: SHA256:+HEvFMmo0EA6ipPMJSaj5+Q6IabebRN+nRD0nxdYrKQ

5) “WARNING: POSSIBLE DNS SPOOFING DETECTED!” — What should I do?

Answer

This message appears when the server key has changed (for example, after maintenance). Remove the old entry:

Linux:

ssh-keygen -f "/home/your_local_username/.ssh/known_hosts" -R "access.grid.unesp.br"

Mac:

ssh-keygen -f "/Users/your_local_username/.ssh/known_hosts" -R "access.grid.unesp.br"

Then try connecting again.

6) Why can’t I log in?

Answer

Possible causes:

Incorrect password: Reset it at https://www.ncc.unesp.br/password-new/
Blocked by Fail2Ban: After multiple failed attempts, repeated scp, or multiple logins, the system blocks access for 15 minutes. Wait and try again.
Network issues: Check your internet connection.

If the problem persists, send the output of the command below to support.ncc@unesp.br:

ssh -vvv username@access.grid.unesp.br

7) I was connected and the terminal froze. Why?

Answer

Common causes:

Internet failure: Temporary connection drop
Inactivity timeout: Idle connections may be terminated
Blocked by Fail2Ban: If you were performing multiple operations

Simply reconnect.

Tip

For unstable connections, consider using screen or tmux. These are terminal multiplexers and session managers that allow programs to keep running in the background even if the connection drops.

8) I don’t know what to do after logging in. Can you help?

Answer

GridUnesp is a high-performance computing platform. The typical workflow is:

Transfer your data and programs to the cluster (Transferring to GridUnesp)
Prepare submission scripts (Running Simulations)
Submit jobs for execution on the compute nodes
Monitor execution
Retrieve results

See the Quick Start Guide and User Manual for detailed guidance.

9) Can I access GridUnesp via a browser?

Answer

No. Access to GridUnesp is exclusively via SSH through the terminal. There is no web interface for running commands.

File Transfer 

10) How do I transfer files to my account?

Answer

Linux/Mac (command line):

scp local_file.dat username@access.grid.unesp.br:/home/username/

Windows:

Use WinSCP or FileZilla. See Transferring Files for instructions.

11) How do I transfer files from GridUnesp to my computer?

Answer

Linux/Mac:

scp username@access.grid.unesp.br:/home/username/remote_file.dat .

The dot (.) indicates the current directory.

Windows:

Use WinSCP or FileZilla to download files.

Installing Programs 

12) I don’t have administrator permission. How do I install software?

Answer

You can install applications in your own /home/ directory. Many programs allow you to specify an alternative installation directory.

Options:

Install locally with --prefix=$HOME/local
Install inside an image or download one using Apptainer
Use Conda/Anaconda to manage environments
Request a global installation from the GridUnesp team (support.ncc@unesp.br)

See Installing Applications for details.

13) How do I install a specific piece of software?

Answer

Check whether the software is already available as a module:
```
module avail partial_name
```
If not available, try a local installation:
- Consult the software documentation for installation instructions
- Use ./configure --prefix=$HOME/local (for software with a configure script)
- Use pip install --user (for Python)
- Use R CMD INSTALL -l $HOME/R/libs (for R)
If the installation is complex, request help from the support team.

Job Processing 

14) Can I process jobs in /home?

Answer

No. The access server must not be used for processing jobs. It is only for:

Accessing your account
Preparing scripts
Submitting jobs (sbatch)
Lightweight pre/post-processing tasks

Jobs must be run on the compute nodes via SLURM.

15) How do I run jobs on GridUnesp?

Answer

Create a submission script with SBATCH directives
Submit it with the command:
```
sbatch submission_script.sh
```
Monitor it with:
```
squeue -u $USER
```

See Running Simulations for detailed examples.

16) How do I create a submission script?

Answer

A basic script looks like this:

example.sh

#!/bin/bash
#SBATCH -J job_name
#SBATCH -n 1
#SBATCH -t 01:00:00

export INPUT="input.dat"
export OUTPUT="output.dat"

module load module_name
job-nanny ./my_program

See Improving the Submission Script for all available options.

Job Monitoring 

17) How long can my job run?

Answer

The maximum time is 30 days, depending on the chosen partition:

short: 24 hours
medium: 7 days
long: 30 days
gpu: 24 hours on a GPU server

Use the #SBATCH -t directive to specify the time.

See Queue System (Detailed) for examples.

18) How many partitions are there?

Answer

GridUnesp has 4 partitions:

short: Jobs up to 24 hours
medium: Jobs up to 7 days
long: Jobs up to 30 days
gpu: 24 hours on a GPU server

Example: #SBATCH -t 2-00 (2 days) will cause SLURM to allocate the job to the medium partition.

The partition is selected automatically based on the requested time. For requests up to 24 hours, SLURM may allocate either the short or gpu partition.

The gpu partition can be explicitly requested with: #SBATCH --partition=gpu. See the GPU Usage section.

19) How do I know my job has started running?

Answer

Use the command:

squeue -u $USER

R (Running): Job is executing
PD (Pending): Job is waiting for resources
CG (Completing): Job is finishing
F (Failed): Job failed

20) Why does my job take a long time to start?

Answer

Possible causes:

High demand: Many jobs in the queue
Resources requested: Jobs requesting many resources may wait longer
Priority: Based on the usage history of the user/group

To view the queue:

squeue -r | wc -l              # Total jobs
squeue -r --states=PD | wc -l  # Pending jobs
squeue -o "%.18i %.9Q %.8j %.8u %.10V %.6D %R" --sort=-p,i --states=PD  # Priority

21) Is there an estimated start time for my job?

Answer

No. The wait time depends on non-deterministic factors:

Duration of currently running jobs
Failures that free up resources early
Arrival of new jobs with different priorities

22) Why do other users have many jobs running while I don’t?

Answer

GridUnesp uses a Fair Share policy:

Users with less recent usage have higher priority
Jobs requesting fewer resources may “skip the queue”
The system seeks balance among all users
The queue configuration aims to avoid idle resources

See Priority Policy for details.

Processing Failures 

23) My job was cancelled due to TIMEOUT. What does that mean?

Answer

The job reached the requested time limit and was terminated by the system.

Solution: Increase the time in the #SBATCH -t directive or optimize your code to run faster.

24) My job was cancelled due to missing INPUT. What does that mean?

Answer

The job-nanny script requires the INPUT variable to be defined. Add it to your script. Example:

export INPUT="input_file.dat input_directory/"

25) My job was cancelled due to missing OUTPUT. What does that mean?

Answer

The job-nanny script requires the OUTPUT variable to be defined. Add it to your script. Example:

export OUTPUT="output_file.dat output_directory/"

26) Why did my job fail?

Answer

Common causes:

Missing input files
Bug in the program code
Missing required modules
Insufficient memory
Timeout

Check the log file:

cat slurm-JOBID.out

If you cannot find the cause, send the log to the support team.

Job Optimization 

27) How can I speed up my job?

Answer

Possible strategies:

Increase available memory (#SBATCH --mem=16G)
Use more CPUs (#SBATCH -c 8)
Parallelize with OpenMP or MPI
Optimize the code (algorithms, compilation)
Use GPUs when applicable

See Optimizing Performance.

28) How many CPUs can I request?

Answer

Each CPU node has 28 cores with Hyper-Threading, for a total of 56 threads. You can request up to 52 CPUs per node with -c 52. The remaining 4 CPUs are reserved for the exclusive use of the operating system.

The GPU node has 48 cores with Hyper-Threading, for a total of 96 threads. You can request up to 88 CPUs per node with -c 88. Again, the remaining 8 CPUs are reserved for the exclusive use of the operating system.

Cluster Resource Allocation (Core Specialization)
Node Type	Total Capacity	Reserved for OS	Available for Jobs
Regular Nodes (`node[001-056]`)	28 Cores / 56 CPUs	2 Cores / 4 CPUs	26 Cores / 52 CPUs
GPU Node (`gpunode001`)	48 Cores / 96 CPUs	4 Cores / 8 CPUs	44 Cores / 88 CPUs

Available memory is 128 GB per CPU node (approximately 2 GB per core), but it is not recommended to request all the memory because there should be room for operating system tasks. In turn, the GPU node has 1.5 TB of RAM.

29) Does GridUnesp have GPUs?

Answer

Yes. GridUnesp has one GPU node (gpunode001) equipped with:

4 NVIDIA L40S GPUs (48 GB each)
48 AMD EPYC CPU cores
1.5 TB of RAM

See GPU Usage for usage details.

30) How do I use multiple CPUs in a job?

Answer

Request the CPUs with the -c directive:

#!/bin/bash
#SBATCH -c 16

Make sure your program is configured to use multiple threads (OpenMP, pthreads).

See Shared Memory section.

31) How do I run jobs in parallel?

Answer

Two main approaches:

Shared memory (OpenMP):

#!/bin/bash
#SBATCH -c 8
export OMP_NUM_THREADS=8
./omp_program

Distributed memory (MPI):

#!/bin/bash
#SBATCH -n 4
module load openmpi/4.0.1
mpirun -n 4 ./mpi_program

See Shared Memory and Distributed Memory.

32) How do I use MPI?

Answer

script_mpi.sh

#!/bin/bash
#SBATCH -N 2      # Requesting 2 nodes
#SBATCH -n 4      # Requesting 4 processes distributed across the 2 nodes

module load openmpi/4.0.1
# or module load intel/mpi/2017

mpirun -np $SLURM_NTASKS ./mpi_program

See section Distributed Memory.

33) How do I specify the number of nodes?

Answer

Use the -N directive:

#SBATCH -N 2           # Exactly 2 nodes
#SBATCH -N 2-4         # Minimum 2, maximum 4 nodes

Combine with -n to distribute processes across nodes.

See section Distributed Memory.

34) Can I submit multiple jobs at once?

Answer

Yes. Use a job array:

job_array.sh

#!/bin/bash
#SBATCH --array=1-10
#SBATCH -n 1

export INPUT="input_${SLURM_ARRAY_TASK_ID}.dat"
export OUTPUT="output_${SLURM_ARRAY_TASK_ID}.dat"

job-nanny ./my_program

See Job Array.

35) Why is my job faster on 1 node than on multiple nodes?

Answer

Possible causes:

Communication overhead: MPI introduces network latency
Shared I/O: Concurrent access to storage
Granularity: Problem too small to justify parallelization

Jobs using multiple nodes write to /store/ (shared), while single-node jobs use local /tmp/, which is faster.

36) Does GridUnesp use InfiniBand or Ethernet?

Answer

Currently, GridUnesp uses 40 Gb/s Ethernet. InfiniBand was used previously, but its maintenance became infeasible.

Best Practices 

37) My files are very large. Is that a problem?

Answer

Single-node jobs: use /tmp/ (~180 GB local). Suitable for files up to a few tens of GB.
Multi-node jobs: use /store/ automatically (large shared space).

To force use of /store/ in single-node jobs with large files:

export SHARED_FS="true"
# or
export LARGE_FILES="true"

38) How do I use /store/ instead of /tmp/?

Answer

Add to the submission script:

export SHARED_FS="true"
# or
export LARGE_FILES="true"

Or request multiple nodes (-N 2) to automatically force the use of /store/.

39) How do I use /tmp/ instead of /store/?

Answer

For multi-node jobs, it is not possible to force use of /tmp/. For single-node jobs, /tmp/ is already the default. To ensure it:

export SHARED_FS="false"
# or
export LARGE_FILES="false"

40) Can I leave files stored on GridUnesp?

Answer

As a rule, no. The cluster should not be used for long-term storage. Keep only files needed for current or near-future processing.

Reasons:

Limited and shared space
No automatic backup
Risk of data loss

41) How do I delete files from my account?

Answer

# Remove a file
rm file.dat

# Remove an empty directory
rmdir directory/

# Remove a directory with its contents
rm -rf directory/

Warning

The rm command is destructive and cannot be undone. Use with care.

42) How can I contribute to the smooth running of the cluster?

Answer

Follow the Best Practices
Respect the Usage Policy
Clean up temporary files regularly
Do not run heavy jobs on the access server (access.grid.unesp.br)
Report problems to the support team
Include acknowledgements in publications as indicated in the Publications section