Talk:Guide to Optimizing GlusterFS

From GlusterDocumentation

Sorry if it is a wrong place to ask -- but, are there some standard configurations/examples for GlusterFS?

I've considered useing GlusterFS on parallel (two nodes, 10Gb Ethernet) centralized fileserver for our HPC clusters (several small ones, tens of CPUs). So I did a Bonnie++ benchmark using one of the servers (Dual Opteron, 4GB RAM, SATA disk) and a client (old P4 2.4GHz box, 512MB RAM, Gigabit Ethernet). I used 8Gb size for the Bonnie++ tests, and tried either forced flush IO or not (-b option).

Without performance translators, GlusterFS was as good as non-buffered IO (i.e., sequential reads and writes were about 16000-17000 K/s, either putc() or block i.o). At the same time, Rewrite and Seek tests were OK (about 8000 K/s and 170 seeks/s).

Then I applied read-ahead and write-behind translators. Blocked reads and writes reached over 40000 K/s, which was same or better with performance of NFS or Lustre 1.6.3; but the Rewrite test of Bonnie++ became much vorse (an order of magnitude, actually, below 800 K/s). So, is there a solution for this -- can one have it both ways with GlusterFS, good IO bandwidth and fast rewrite? Thank you!

Gas knc 09:56, 18 December 2007 (PST)

Dear GlusterFS developers,

I'm considering using GlusterFS on our ner parallel (two nodes, 10Gb Ethernet) centralized fileserver for our HPC clusters (several small ones, tens of CPUs). So I did performance tests for the latest GlusterFS as well as plain NFS and the recent Lustre-1.6.3.

The GlusterFS looks very attractive because I understand that unlike Lustre, one could use it for non-x86 Linux platforms as well, which we might have gotted in the near future.

Update: the actual numbers look like this:

===============================================
FileSystem:     Sequential Output , K/sec
                Per-char    Block     Rewrite
===============================================
NFS               14442      30419	 7710
Lustre            16012      35228     19018
GlusterFS         16582      15833      8358	
GlusterFS, wb     17988      43774      8409
GlusterFS, ra     18414      15863      1804
GlusterFS, ra, wb 22403      41821       355
===============================================
FileSystem:     Sequential Input, K/sec Random
                Per-char    Block      seeks, #/s 
===============================================
NFS               20229      49510      178.8
Lustre            17284      47753       53.0
GlusterFS         16791      16815      161.4
GlusterFS, wb     15304      17438      174.1
GlusterFS, ra     19420      54803      143.3
GlusterFS, ra, wb 19900      54427      144.4 
===============================================