.. _priority_policy: =============== Priority Policy =============== .. contents:: In this section: :local: :depth: 2 GridUnesp uses a priority system based on **Fair Share** to ensure balanced use of computational resources among all users and research groups. Fair Share Concept ================== Fair Share works based on the following principles: - Each **research group** is entitled to a fair portion of the resources - Each **user** within the group is entitled to a fair portion - Priority decreases as the user/group uses more resources - Priority is gradually "reset" over time (14-day half-life) **Simplified formula:** - Group: ``1/N`` of the resources, where N = total number of groups - User: ``1/M`` of the group's resources, where M = number of users in the group Viewing Fair Share ================== sshare Command -------------- The ``sshare`` command shows the current Fair Share status of all users: .. code-block:: bash sshare -a **Example output:** .. code-block:: text Account User RawShares NormShares RawUsage EffectvUsage FairShare ------------ --------- ---------- ----------- ----------- ------------- ---------- startrek 10 0.008065 40079089 0.008656 startrek kirk 1 0.333333 1761930 0.043961 0.220126 startrek picard 1 0.333333 38237843 0.954060 0.219227 startrek spock 1 0.333333 79315 0.001979 0.221024 hogwarts 10 0.008065 8028212 0.001734 hogwarts potter 1 0.250000 0 0.000000 0.278527 hogwarts hermione 1 0.250000 8028212 1.000000 0.275831 hogwarts voldemort 1 0.250000 0 0.000000 0.278527 hogwarts malfoy 1 0.250000 0 0.000000 0.278527 **Interpretation of the columns:** - **Account:** Group name - **User:** User name - **RawShares:** Assigned weights (the higher, the more resources) - **NormShares:** Normalized share - **RawUsage:** Recent cumulative usage - **EffectvUsage:** Effective usage (0-1) - **FairShare:** Current priority (**the higher, the better**) Filtering the View ------------------ To see only your group: .. code-block:: bash sshare -a | grep group_name To see only your user: .. code-block:: bash sshare -a | grep $USER How Priority Affects Jobs ========================= The priority calculated by Fair Share directly influences the execution order of jobs in the queue. Listing Jobs in Priority Order ------------------------------ .. code-block:: bash :caption: View pending jobs sorted by priority squeue -o "%.18i %.9Q %.8j %.8u %.10V %.6D %R" --sort=-p,i --states=PD **Example output:** .. code-block:: text JOBID PRIORITY NAME USER SUBMIT_TIM NODES NODELIST(REASON) 2847404 296816 force1.s yoda 2022-03-31 1 (Priority) 2838783_12 222821 dark1.sh vader 2022-03-15 1 (Priority) 2838783_14 222821 dark1.sh vader 2022-03-15 1 (Priority) 2844502 185817 crucio1. valdemo+ 2022-03-28 1 (Priority) 2848555 50480 vulcano. spock 2022-04-01 4 (Priority) **Notes:** - The **PRIORITY** column shows the numeric priority value - Jobs with higher priority (larger number) run first - Priority is dynamic and changes as: - New jobs are submitted - Running jobs finish - Time passes (14-day half-life) Factors That Influence Priority =============================== 1. **Recent cluster usage** - The more you used in the last 14 days, the lower your priority - Every 14 days, the "counter" is gradually reset 2. **Number of users in the group** - Groups with many users have smaller slices per person - Encourages groups to manage usage internally 3. **Job size** - Jobs that request many resources may have their priority adjusted - The system seeks a balance between large and small jobs 4. **Wait time** - Jobs that wait a long time gradually gain priority - Prevents starvation (jobs that never run) - However, jobs that require a short processing time have a greater chance of obtaining resources more quickly. Tips to Improve Your Priority ============================= ✅ **Do:** 1. **Request only what you need** .. code-block:: bash #SBATCH -t 02:00:00 # If your job runs in 1h30, don't request 24h #SBATCH --mem=8G # If it uses 6G, don't request 64G 2. **Distribute usage over time** - Avoid submitting hundreds of jobs at once - Spread submissions over days/weeks 3. **Use job arrays for many small jobs** .. code-block:: bash #SBATCH --array=1-1000 # 1000 small jobs #SBATCH --ntasks=1 #SBATCH --time=01:00:00 4. **Monitor your Fair Share regularly** .. code-block:: bash sshare -U $USER ❌ **Avoid:** - **Monopolizing resources** with very large and long jobs - **Requesting resources far above what is needed** - **Leaving jobs running** that could have already finished Understanding the 14-Day Half-Life ================================== The system uses an exponential decay with a 14-day half-life: - **Day 0:** Maximum usage (minimum priority) - **Day 14:** The "weight" of the usage drops by half - **Day 28:** The "weight" drops to 1/4 of the original value - **And so on...** This means that: - If you used many resources today, your priority tends toward the initial value as the days pass - The system gradually "forgets" old usage - Everyone has a chance to recover priority after periods of inactivity Practical Example ================= Let's follow a user over time: **Week 1:** João submits few jobs: high priority (0.9) .. code-block:: bash sshare -U joao # FairShare = 0.901 **Week 2:** João submits many jobs: priority drops .. code-block:: bash sshare -U joao # FairShare = 0.452 **Week 3:** João keeps using a lot: very low priority .. code-block:: bash sshare -U joao # FairShare = 0.187 **Week 4:** João reduces usage: priority starts to recover .. code-block:: bash sshare -U joao # FairShare = 0.223 **Weeks 5-6:** With moderate usage: priority tends toward the initial value .. code-block:: bash sshare -u joao # FairShare = 0.854 Command Summary =============== .. list-table:: Commands for priority management :header-rows: 1 :widths: 50 50 * - Command - Description * - ``sshare -a`` - View Fair Share of all users * - ``sshare -U $USER`` - View only your Fair Share * - ``sshare -a | grep group`` - View your group's Fair Share * - ``squeue -U $USER --sort=-p`` - View your jobs sorted by decreasing priority (``-p``) * - ``squeue -o "%.18i %.9Q %.8j %.8u %.10V %.6D %R" --sort=-p,i --states=PD`` - List pending jobs in order of decreasing priority (``-p``) .. seealso:: - :ref:`running_simulations` - How to submit jobs - :ref:`optimizing_performance` - Tips for optimizing resources - :ref:`best_practices` - General recommendations