Effective Bandwidth (b_eff) Benchmark

The algorithm of b_eff (version 3.6 + bugfix 3.6.0.1)

Definition of the effective bandwidth b_eff:

Details of the algoritm:

Summary:

Background

Output of the b_eff Benchmark

• b_eff.prot a detailed protocol
• b_eff.short a short version of the protocol
• b_eff.sum a short overview
• b_eff.plot a file with data for plotting
• b_eff.gps a script for gnuplot
• b_eff.tex a LaTeX source file to create a benchmark report

Sourcecode

Benchmarking

• which compilation command was used,
• which MPI implementation, version, ... was used,
• do you have setup a special environment, e.g. UNIX environment, variables for compiling or running the benchmark,
• with which command and/or batch script have you started the benchmark,
• b_eff.c writes its results on stdout; please attach this output as a gzip'ed attachment.

First Results

Distributed Memory Systems

On a Cray T3E-900 with 512+32 processors and 128 MByte/processor

Used commands: module switch mpt mpt.1.3.0.2 
               cc -o b_eff -DMEMORY_PER_PROCESSOR=128 b_eff.c 
               export MPI_BUFFER_MAX=4099 
               mpirun -np size ./b_eff result_3.2_t3e_size
MPI release:   mpt.1.3.0.2
Execution time < 225 sec 

On a Hitachi SR 8000 with 128 processors on 16 nodes and 1 GByte/processor

Used commands: mpicc -O4 -pvec +Op -noparallel -o b_eff -DMEMORY_PER_PROCESSOR=1024 b_eff.c  (MPI-MPP)
               mpicc -O4 -pvec +Op -parallel -o b_eff -DMEMORY_PER_PROCESSOR=1024 b_eff.c    (MPI + Compas)
               mpiexec -p multi -N #nodes -n #processes  ./b_eff
MPI release:   P-1811-1113, HI-UX/MPP 
               MPIR_RANK_NO_ROUND=yes
               JOBTYPE=E8S
               was automatically set according to the HLRS defaults with MPI on SR8000
Execution time < 130 sec 

On a Hitachi SR 8000 with 24 processors on 3 nodes and 1 GByte/processor

Used commands: mpicc -o b_eff -DMEMORY_PER_PROCESSOR=1024 b_eff.c -lm
               hpstatus
               limit datasize 500000 
 
explicit allocation of contiguous ranks on 3 nodes: 
               mpiexec -p NODE0 -N 1 -n size/3 ./b_eff result_3.3_SR8000_1GB_003nodes_sizePEs_c \
                     : -p NODE1 -N 1 -n size/3 ./b_eff \
                     : -p NODE2 -N 1 -n size/3 ./b_eff
         contiguous ranks on 2 nodes: 
               mpiexec -p NODE0 -N 1 -n size/2 ./b_eff result_3.3_SR8000_1GB_002nodes_sizePEs_c \
                     : -p NODE1 -N 1 -n size/2 ./b_eff
         contiguous ranks on 1 node: 
               mpiexec -p NODE0 -N 1 -n size/1 ./b_eff result_3.3_SR8000_1GB_001nodes_sizePEs_c 

         special additional options
          64: option -lp64, used on mpicc
          SS: environment variable JOBTYPE=SS, set while executing mpiexec 
 
 
default round-robin allocation:  
               mpiexec -p ALL -N nodes -n size ./b_eff result_3.3_SR8000_1GB_nodesnodes_sizePEs
MPI release:   P-1811-1113, HI-UX/MPP 
Execution time < 150 sec 

On a Hitachi SR 2201 with 32+8 processors and 256 MByte/processor

Used commands: mpicc -o b_eff -D MEMORY_PER_PROCESSOR=1024 b_eff.c -lm
               mpirun -n size ./b_eff result_3.2_SR2201_size
MPI release:   P-1811-1112, HI-UX/MPP 02-01 (based on MPICH Version 1.0.11) 

On a SwissTX T1-baby with 6*2 processors and 512 MByte/processor

Used commands: tnetcc -o b_eff -DMEMORY_PER_PROCESSOR=512 b_eff.c -lm
               bsub -Is -n size txrun b_eff result_3.3_SwissTX1baby_size
MPI release:   T-NET: 0.17 (see http://service.scs.ch/gb2/fci/revision/)
                           (sysconfig -q tnet Version) 
Execution time < 163 sec 

On IBM SP2

Used commands: mpcc -o b_eff -DMEMORY_PER_PROCESSOR=512 b_eff.c -lm
               poe b_eff result_SP2_512MB_sizePE -procs size
MPI release:   ???
Execution time < 980 sec (very long due to the extremly high Alltoallv latancy)

Shared Memory Systems

• In most cases, applications need not copy date between the processores. There is no way to measure these savings, although these saving are an important part of the communication capabilities of the system.
• MPI communication is not the appropriate programming model of shared memory systems. The latency and bandwidth of data copying with OpenMP is normally at least double as fast as MPI message passing. Therefore communication bandwidth should be measured inside of shared memory nodes by copying data with OpenMP.

On a NEC SX-4/32 with 32 processors and 256 MByte/processor

Used commands: mpicc -o b_eff -D MEMORY_PER_PROCESSOR=256 b_eff.c -lm
               mpirun -np size ./b_eff result_3.2_sx4_size
MPI release:   9.1

On a NEC SX-5/8B with 8 processors and 8 GByte/processor

Used commands: mpicc -o b_eff -D MEMORY_PER_PROCESSOR=256 b_eff_mthd1+2.c -lm
               mpirun -np size ./b_eff result_3.2_sx5_256MB_size
MPI release:   9.2

On a HP-V 9000/800/V2250 with 8 processors and 1024 MByte/processor

Used commands: mpicc -o b_eff -D MEMORY_PER_PROCESSOR=1024 b_eff.c -lm
               mpirun -np size ./b_eff result_3.2_hpv_size
MPI release:   HP MPI 1.4 implementation

On a SGI Cray SV1-B/16-8 with 16 processors and 512 MByte/processor

Used commands: cc -h taskprivate $LIBCM -o b_eff -D MEMORY_PER_PROCESSOR=512 b_eff.c
               qsub -eo -q nqebatch jobsize
               with jobsize:
                 #!/bin/sh
                 export MPI_GROUP_MAX=64
                 ja
                 mpirun -nt size ~/b_eff ~/result_3.3_SV1B16_size
                 ja -st 
MPI release:   mpt.1.3.0.2
Execution time < 360 sec (expected on a dedicated system)

References:

Links