GlusterFS 1.3.pre0-BENKI Aggregated I/O Benchmark

From GlusterDocumentation

GlusterFS 1.3.pre0-BENKI Aggregated I/O Benchmark on 16 brick server and 64 clients over Infiniband Verbs transport. Multiple 'dd' of varying blocks are read and written from multiple clients simultaneously.

Contents 1 GlusterFS Brick Config (16 bricks) 2 Client Config (64 clients) 3 Interconnect Switch 4 Aggregated Throughput of 16 bricks Vs Number of Clients (using ib-verbs)

GlusterFS Brick Config (16 bricks)

• Processor - Dual Intel(R) Xeon(R) CPU 5160 @ 3.00GHz
• RAM - 8GB FB-DIMM
• Linux Kernel - 2.6.18-5+em64t+ofed111 (Debian)
• Disk: SATA-II 500GB
• Mellanox MHGS18-XT/S Infiniband HCA

Client Config (64 clients)

• Processor - Single Intel(R) Pentium(R) D CPU 3.40GHz
• RAM - 4GB DDR2 (533 MHz)
• Linux Kernel - 2.6.18-5+em64t+ofed111 (Debian)
• Mellanox MHGS18-XT/S Infiniband HCA

Interconnect Switch

• Voltaire port Infiniband Switch (14U)

Aggregated Throughput of 16 bricks Vs Number of Clients (using ib-verbs)

Note that transfer rate is in BYTES per second (multiply by 8 for BITS per second).

Notes:

• Peak aggregated read throughput was 130Gbps.
• ib-verbs transport driver is about 30% faster than ib-sdp transport driver.
• After a particular threshold, write performance plateaus because of disk I/O bottleneck. System memory greater than the peak load will ensure best possible performance.