.. _history: ======= History ======= .. contents:: In this section: :local: :depth: 2 This section presents the evolution of GridUnesp year by year, from its conception to the present day. The video below summarises the history of GridUnesp since its creation. .. raw:: html
2008 - Infrastructure Announcement ==================================== UNESP announces the largest high-performance computing infrastructure in Latin America. **Total investment:** R$ 8 million - **Ministry of Science and Technology:** R$ 4.4 million (via Finep/Fundunesp) - **UNESP:** R$ 3.6 million **Acquisition:** Competitive bidding process won by Sun Microsystems of Brazil. **Original hardware specifications:** - Processors: Quad Core Intel Xeon 5400 - Processing cores: 2,048 - Capacity: 33.3 teraflops (trillions of calculations per second) **Composition:** - **Central cluster:** 256 processing nodes on the UNESP campus in São Paulo (Barra Funda) - **Six distributed clusters:** 16 nodes each, on the campuses of: - Araraquara - Bauru - Botucatu - Ilha Solteira - Rio Claro - São José do Rio Preto **Connectivity:** - High-speed connection to the North American Internet2 - MetroSampa network linking institutions in the São Paulo metropolitan area - ANSP/RNP/Florida International University network between São Paulo and Miami - FAPESP's KyaTera network connecting São Paulo to the interior clusters **Benefits for the scientific community:** - Remote access for large-scale computational operations - Support for areas requiring processing and analysis of large data volumes - Reduction in research computation time - Performance leadership with low energy consumption - International partnerships (Open Science Grid – USA, European and Asian grids) 2009 - Start of Operations =========================== GridUnesp begins operation, representing a Brazilian milestone in: - High-performance computing - Long-distance optical networks for scientific research **Equivalent capacity at the time:** - 10,000 personal computers - Storage equivalent to an 18-storey tower of stacked DVDs - Communication speed 5,000 times faster than home broadband **Training:** First course offered at the Center for Scientific Computing of UNESP (NCC/Unesp), covering: - In-person format with online broadcast - Basic operation of GridUnesp - Compilers and development tools - Parallelism and optimisation techniques 2010 - Expansion and Outreach ============================== **MasterClass Event:** Reception of high-school students during an international event, including: - Presentation on the GridUnesp infrastructure - Visit to the cluster at the Datacenter **Year-end metrics:** - 26 registered scientific projects - Areas covered: Biophysics, Biology, Materials Science, Computing, Physics, Geosciences, Chemistry - 90 registered members - 10 million processing hours 2011 - Accelerated Growth ========================== - New call for project registration - **12 new scientific projects** - **41 new users** - More than **20 million** processing hours 2012 - International Recognition ================================== **Higgs Boson Discovery:** Scientists connected to GridUnesp take part in the discovery at CERN. **CERN Visit:** A delegation of six physicists from the European Organization for Nuclear Research visits GridUnesp, related to Brazil's bid for associate membership. 2013 - Consolidation and New Services ====================================== **HPC Administrators Workshop:** - Support for technical teams of research groups - Training based on GridUnesp's operational experience - Creation of a professional network for knowledge exchange **Intel Software Conference:** - Parallel programming focused on Intel Xeon Phi - Hands-on activities **ANSPGrid CA:** Deployment of the `ANSPGrid CA `_ Certification Authority at NCC/Unesp: - Issuance of X.509 grid certificates for researchers - Public-key cryptography for the São Paulo scientific community - Compliance with International Grid Trust Federation (IGTF) guidelines 2014 - Accessibility ===================== Creation of a web portal to facilitate researchers' access: - Intuitive interface for a complete workflow - Developed with free and open-source code - Accessible via browser **Press visit:** Trainee reporters from the newspaper *Folha de S. Paulo* visit the infrastructure during the Science and Health Training Programme. 2015 - Expressive Milestones ============================== GridUnesp reaches: - More than **60** registered scientific projects - More than **300** registered users - More than **140 million** processing hours 2016 - Upgrade and Scientific Achievements =========================================== **Gravitational wave discovery:** LIGO (Laser Interferometer Gravitational-wave Observatory) announces the detection of gravitational waves. GridUnesp was: - The second largest contributor to processing, behind only Fermilab - 7.5 million hours processed (12.3% of the total in OSG) **Acquisition of new equipment (upgrade):** - **Funding:** R$ 2.2 million from Finep - **Supplier:** Huawei **New hardware specifications:** *Processing:* - 56 servers (1,568 cores) - 2 Intel E5-2680 v4 processors (2.4 GHz) per server - 28 cores per server - 4 GB per core - 40 GB/s network - 60 TFlops (theoretical) *Management:* - 4 servers with 12 cores each - 40 GbE network *Storage:* - 288 TB raw (216 TB effective) - NL-SAS 4 TB disks with Lustre technology - 4 control servers *Internal network:* - 40 GbE interconnecting processing nodes - 2 switches with 32 ports of 40 GbE each **Upgrade gains:** - Solution for saturated infrastructure - 15-fold reduction in energy consumption **New documentation:** Release of the reformulated and updated user documentation. 2017 - Upgrade Completion ========================== **Final capacity:** ================ ========= ============ Feature Before After upgrade ---------------- --------- ------------ Processing 23 TFlops 77 TFlops Storage 136 TB 288 TB CPU Cores 2,048 3,104 ================ ========= ============ 2018 - Expansion and Statistics ================================= **ANSPGrid CA:** - Certificates now issued to research institutes across all of Brazil - Certificate Policy updated (version 2.0) **Statistics:** - More than 80 registered scientific projects - 500 registered users - 170 million processing hours - More than 200 publications using GridUnesp resources 2019 - Science Outreach ========================= **Special MasterClass event:** Celebration of the International Day of Women and Girls in Science: - Around 100 girls from public and private schools - Presentation of Physics topics (Standard Model) - Visit to the cluster at the Data Center 2020 - Pandemic and Accelerated Demand ======================================== **Operations:** - Team working in remote mode - Remote maintenance of almost all systems **Demand:** - More than 100 registered projects (19 in 2020 alone) - More than 650 registered users (75 in 2020) - Around 13 million processing hours in the year 2021 - Remote Continuity ========================== - Maintenance of remote working regime - **67** new registered users - More than **11 million** processing hours **ANSPGrid CA:** - Root certificate renewed for a further 10 years - Certificate Policy updated (version 3.0) 2022 - Return and Consolidation ================================= - Return to in-person work - Rare idle periods for the cluster - More than 1,000 daily jobs in the partition (≈50% running simultaneously) - Release of user documentation version 2.0 2023-2024 - Continuous Expansion ================================== **New equipment:** - New 400 TB storage system with the installation of a more modern and reliable system - GPU nodes to meet demands from machine learning and artificial intelligence **Community:** - More than 140 registered projects - More than 850 registered users - Continuous scientific output across multiple areas of knowledge 2025 - New Milestones ====================== - Surpasses 1,000 registered users - Intensive GPU use: more than 50,000 jobs in one year - More than 160 scientific applications and corresponding versions installed since 2017 .. seealso:: - :ref:`statistics` - Updated usage data - :ref:`infrastructure` - Current cluster configuration - :ref:`publications` - Scientific output using GridUnesp - :ref:`covid19` - Impact of the pandemic on operations