FRANKFURT, Germany; BERKELEY, Calif.; and KNOXVILLE, Tenn.— The 59th edition of the TOP500 revealed the Frontier system to be the first true exascale machine with an HPL score of 1.102 Exaflop/s.

The No. 1 spot is now held by the Frontier system at Oak Ridge National Laboratory (ORNL) in the US. Based on the latest HPE Cray EX235a architecture and equipped with AMD EPYC 64C 2GHz processors, the system has 8,730,112 total cores, a power efficiency rating of 52.23 gigaflops/watt, and relies on gigabit ethernet for data transfer.

However, a recent development to the Frontier system has allowed the machine to surpass the 1 exaflop barrier. With an exact HPL score of 1.102 Exaflop/s, Frontier is not only the most powerful supercomputer to ever exist – it’s also the first true exascale machine.

The top position was previously held for two years straight by the Fugaku system at the RIKEN Center for Computational Science (R-CCS) in Kobe, Japan. Sticking with its previous HPL benchmark score of 442 PFlop/s, Fugaku has now dropped to No. 2. Considering the fact that Fugaku’s theoretical peak is above the 1 exaflop barrier, there’s cause to also call this system an exascale machine as well. However, Frontier is the only system able to demonstrate this on the HPL benchmark test.

Another change within the TOP10 is the introduction of the LUMI system at EUROHPC/CSC in Finland. Now occupying the No. 3 spot, this new system has 1,110,144 cores and has a HPL benchmark of nearly 152 PFlop/s. LUMI is also noteworthy in that it is the largest system in Europe.

Finally, another change within the TOP10 occurred at the No. 10 spot with the new addition of the Adastra system at GENCI-CINES in France. It achieved an HPL benchmark score of 46.1 Pflop/s and is the second most powerful machine in Europe, behind LUMI.

Here is a summary of the system at the Top10:

• Frontier is the new No. 1 system in the TOP500. This HPE Cray EX system is the first US system with a peak performance exceeding one ExaFlop/s. It is currently being integrated and tested at the ORNL in Tennessee, USA, where it will be operated by the Department of Energy (DOE). It currently has achieved 1.102 Exaflop/s using 8,730,112 cores. The new HPE Cray EX architecture combines 3rd Gen AMD EPYC™ CPUs optimized for HPC and AI with AMD Instinct™ 250X accelerators and Slingshot-11 interconnect.
• Fugaku, now the No. 2 system, is installed at the RIKEN Center for Computational Science (R-CCS) in Kobe, Japan. It has 7,630,848 cores which allowed it to achieve an HPL benchmark score of 442 Pflop/s. This puts it 3x ahead of the No. 3 system in the list. 
• The new LUMI system, another HPE Cray EX system installed at EuroHPC center at CSC in Finland, is the new No. 3 with a performance of 151.9 Pflop/s just ahead of No 4. The European High-Performance Computing Joint Undertaking (EuroHPC JU) is pooling European resources to develop top-of-the-range Exascale supercomputers for processing big data. One of the pan-European pre-Exascale supercomputers, LUMI, is in CSC's data center in Kajaani, Finland.
• Summit, an IBM-built system at ORNL in Tennessee, USA, is now listed at the No. 4 spot worldwide with a performance of 148.8 Pflop/s on the HPL benchmark which is used to rank the TOP500 list. Summit has 4,356 nodes, each housing two Power9 CPUs with 22 cores and six NVIDIA Tesla V100 GPUs, each with 80 streaming multiprocessors (SM). The nodes are linked together with a Mellanox dual-rail EDR InfiniBand network.
• Sierra, a system at the Lawrence Livermore National Laboratory, CA, USA, is at No. 5. Its architecture is very similar to the #4 systems Summit. It is built with 4,320 nodes with two Power9 CPUs and four NVIDIA Tesla V100 GPUs. Sierra achieved 94.6 Pflop/s.
• Sunway TaihuLight is a system developed by China's National Research Center of Parallel Computer Engineering & Technology (NRCPC) and installed at the National Supercomputing Center in Wuxi, China's Jiangsu province, is listed at the No. 6 position with 93 Pflop/s.
• Perlmutter at No. 7 is based on the HPE Cray "Shasta" platform, and a heterogeneous system with AMD EPYC based nodes and 1536 NVIDIA A100 accelerated nodes. Perlmutter achieved 64.6 Pflop/s
• Now at No. 8, Selene is an NVIDIA DGX A100 SuperPOD installed inhouse at NVIDIA in the USA. The system is based on an AMD EPYC processor with NVIDIA A100 for acceleration and a Mellanox HDR InfiniBand as network and achieved 63.4 Pflop/s.
• Tianhe-2A (Milky Way-2A), a system developed by China's National University of Defense Technology (NUDT) and deployed at the National Supercomputer Center in Guangzhou, China is now listed as the No. 9 system with 61.4 Pflop/s.
• The Adastra system installed at GENCI-CINES is new to the list at No. 10. It is the third new HPE Cray EX system and the second fastest system in Europe. It achieved 46.1 Pflop/s.

 

Other TOP500 Highlights

Once more, AMD processors seem to be a preferred technology for HPC systems. Frontier utilizes 3rd Gen AMD EPYC CPUs optimized for HPC and AI. The No. 3 LUMI system also relied on AMD 3rd Gen EPYC processors, while No. 7 Permutter used AMD EPYC 7763 processors and No. 8 Selene used AMD EPYC 7742 processors.

Another point reminiscent of most recent TOP500 lists is the fact that China and the United States are the two countries with the most entries on the list. While China stagnated at 173 systems, the United States dropped from 150 systems to 126. Still, these two countries make up for nearly two-thirds of the supercomputers on the TOP500.

While there was some change in terms of system interconnects, the same trends as last list continue to hold. Ethernet still won out with 226 machines, but this was a drop from 240 machines on the last list. Infiniband accounted for 196 on the current list, which increased from 180 previously. Omnipath stayed consistent with 40 machines on the list, while custom interconnects dropped from 34 connections in the last list to 32 connections on the current one. Like the last list, there were only 6 systems with proprietary networks.

GREEN500 Results

The system to claim the No. 1 spot for the GREEN500 is the Frontier Test & Development System (TDS) at ORNL in the US. With 120,832 total cores and an HPL benchmark of 19.2 PFlop/s, the Frontier TDS machine is basically just one rack identical to the actual Frontier system. Therefore, it makes sense that it is outmatched by Frontier’s 7,733,248 cores and HPL benchmark of 1.102 Exaflop/s. However, Frontier TDS has some impressive efficiency capabilities. With a power efficiency of 62.8 gigaflops/watt, Frontier TDS is the clear leader of the GREEN500.

Continuing with this trend, and perhaps even redefining it, is the original Frontier system at ORNL in the US. This machine earned the highest spot on the TOP500 list and was still able to take the No. 2 spot on the GREEN500. This system is able to produce a whopping 1.102 Exaflop/s HPL benchmark score while keeping its power efficiency at 55.23 gigaflops/watt. Considering this machine was able to stay competitive on the GREEN500 while becoming the first exascale system shows how energy efficiency is becoming a top priority for HPC facilities.

The No. 3 spot was taken by the LUMI system, which is quite an accomplishment for the newcomer. Despite being the largest system in Europe, LUMI has an impressive power efficiency rating of 51.63 gigaflops/watt.

In fact, LUMI is part of a wider trend in HPC that is proving that power doesn’t have to be sacrificed in the name of efficiency. The No. 4 spot was held by Adastra, an HPE Cray EX system at GENCI-CINES that took the No. 10 spot on the TOP500. It’s also the second fastest in Europe and has an efficiency rating of 50.03 gigaflops/watt.

Last year’s winner of the GREEN500 has moved down to the No. 5 spot, as the MN-3 system from Preferred Networks in Japan received a power efficiency rating of 40.90 gigaflops/watt. This is an improvement over the system’s previous score of 39.38 gigaflops/watt.

HPCG Results

The TOP500 list has incorporated the High-Performance Conjugate Gradient (HPCG) Benchmark results, which provide an alternative metric for assessing supercomputer performance and is meant to complement the HPL measurement.
At the moment, we do not have HPCG data on Frontier. Therefore, Fugaku is the winner here with 16.0 HPCG-petaflops. Once again, Summit retained its No. 2 spot with 2.93 HPCG-petaflops, while LUMI took the No. 3 spot from Sierra with 1.94 HPCG-petaflops.

HPL-AI Results

The HPL-AI benchmark seeks to highlight the convergence of HPC, and artificial intelligence (AI) workloads based on machine learning and deep learning by solving a system of linear equations using novel, mixed-precision algorithms that exploit modern hardware.

The Frontier system has demonstrated an HPL-AI benchmark of 6.86 exaflops, which more than triples Fugaku’s previous winner score of around 2 exaflops. Fugaku was the reigning champ for some time, but it would appear that Frontier has come along to shake up the competition.