<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p
{mso-style-priority:99;
margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:537620122;
mso-list-type:hybrid;
mso-list-template-ids:-1104488556 -1 134807555 134807557 134807553 134807555 134807557 134807553 134807555 134807557;}
@list l0:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-font-family:"Times New Roman";
mso-bidi-font-family:"Times New Roman";}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style></head><body lang=EN-AU link=blue vlink="#954F72"><div class=WordSection1><p>Morning, hope the folllowing ramble is ok, just examining the results of some extensive (and destructive <span style='font-family:"Segoe UI Emoji",sans-serif'>😊</span>) testing of gluster 3.6.4 on some disks I had spare. Cluster purpose is solely for hosting qemu vm’s via Proxmox 3.4</p><p><o:p> </o:p></p><p>Setup: 3 Nodes, well spec’d</p><p style='margin-left:36.0pt;text-indent:-18.0pt'>-<span style='font-size:7.0pt;font-family:"Times New Roman",serif'> </span>64 GB RAM</p><p style='margin-left:36.0pt;text-indent:-18.0pt'>- VNB & VNG</p><p style='margin-left:36.0pt;text-indent:-18.0pt'> <span style='font-size:7.0pt;font-family:"Times New Roman",serif'>* </span>CPU : E5-2620</p><p style='margin-left:36.0pt;text-indent:-18.0pt'>- VNA</p><p style='margin-left:36.0pt;text-indent:-18.0pt'> * CPU’s : Dual E5-2660</p><p style='margin-left:36.0pt;text-indent:-18.0pt'>-<span style='font-size:7.0pt;font-family:"Times New Roman",serif'> </span>Already in use as a Proxmox and Ceph Cluster running 30 Windows VM’s</p><p><o:p> </o:p></p><p>Gluster Bricks.</p><p style='margin-left:36.0pt;text-indent:-18.0pt'>-<span style='font-size:7.0pt;font-family:"Times New Roman",serif'> </span>All bricks on ZFS with 4 GB RAM ZIL, 1GB SSD SLOG and 10GB SSD Cache</p><p style='margin-left:36.0pt;text-indent:-18.0pt'>-<span style='font-size:7.0pt;font-family:"Times New Roman",serif'> </span>LZ4 Compression</p><p style='margin-left:36.0pt;text-indent:-18.0pt'>-<span style='font-size:7.0pt;font-family:"Times New Roman",serif'> </span>Sync disabled</p><p><o:p> </o:p></p><p>Brick 1:</p><p style='margin-left:36.0pt;text-indent:-18.0pt'>-<span style='font-size:7.0pt;font-family:"Times New Roman",serif'> </span>6 Velocitoraptors in a RAID10+ (3 Mirrors)</p><p style='margin-left:36.0pt;text-indent:-18.0pt'>-<span style='font-size:7.0pt;font-family:"Times New Roman",serif'> </span>High performance</p><p style='margin-left:36.0pt;text-indent:-18.0pt'>-<span style='font-size:7.0pt;font-family:"Times New Roman",serif'> </span>Already hosting 8 VM’s</p><p><o:p> </o:p></p><p>Bricks 2 & 3:</p><p style='margin-left:36.0pt;text-indent:-18.0pt'>-<span style='font-size:7.0pt;font-family:"Times New Roman",serif'> </span>Spare external USB 1TB Toshiba Drive attached via USB3</p><p style='margin-left:36.0pt;text-indent:-18.0pt'>-<span style='font-size:7.0pt;font-family:"Times New Roman",serif'> </span>Crap performance <span style='font-family:"Segoe UI Emoji",sans-serif'>😊</span> About 50/100 MB/s R/W</p><p><o:p> </o:p></p><p><o:p> </o:p></p><p>Overall impressions – pretty good. Installation is easy and now I’ve been pointed to up to date docs and got the hang of the commands, I’m happy with the administration – vastly simpler than Ceph. The ability to access the files on the native filesystem is good for peace of mind and enables some interesting benchmark comparisons. I simulated drive failure by killing all the gluster processes on a node and it seemed to cope ok. </p><p><o:p> </o:p></p><p>I would like to see better status information such as “Heal % progress”, “Rebalance % progress”<o:p></o:p></p><p><o:p> </o:p></p><p>NB: Pulling a USB external drive is a *<b>bad</b>* idea as it has no TLER support and this killed an entire node, had to hard reset it. In production I would use something like WD Red NAS drives.</p><p><o:p> </o:p></p><p><o:p> </o:p></p><p>Despite all the abuse I threw at it I had no problems with split brain etc and the integration with proxmox is excellent. When running write tests I was very pleased to see it max out my bonded 2x1GB connections, something ceph has never been able to do. I consistently got 110+ MB/s raw write results inside VM’s</p><p><o:p> </o:p></p><p>Currently running 4 VM’s off the Gluster datastore with no issues.</p><p><o:p> </o:p></p><p>Benchmark results – done using Crystal DiskMark inside a Windows 7 VM, with VIRTIO drivers and writeback enabled. I tested a Gluster replica 3 setup, replica 1 and direct off the disk (ZFS). Multpile tests were run to get a feel for average results.</p><p><o:p> </o:p></p><p>Node VNB</p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Replica 3</p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Local Brick: External USB Toshiba Drive</p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'>-----------------------------------------------------------------------<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'>CrystalDiskMark 3.0.3 x64 (C) 2007-2013 hiyohiyo<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'> Crystal Dew World : http://crystalmark.info/<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'>-----------------------------------------------------------------------<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'>* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'> Sequential Read : 738.642 MB/s<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'> Sequential Write : 114.461 MB/s<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'> Random Read 512KB : 720.623 MB/s<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'> Random Write 512KB : 115.084 MB/s<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'> Random Read 4KB (QD=1) : 9.684 MB/s [ 2364.3 IOPS]<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'> Random Write 4KB (QD=1) : 2.511 MB/s [ 613.0 IOPS]<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'> Random Read 4KB (QD=32) : 24.264 MB/s [ 5923.7 IOPS]<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'> Random Write 4KB (QD=32) : 5.685 MB/s [ 1387.8 IOPS]<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'> Test : 1000 MB [C: 70.1% (44.8/63.9 GB)] (x5)<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'> Date : 2015/10/09 9:30:37<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt'><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Courier New"'> OS : Windows 7 Professional N SP1 [6.1 Build 7601] (x64)<o:p></o:p></span></p><p><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p><p><o:p> </o:p></p><p>Node VNA</p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Replica 1 (So no writing over ethernet)</p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Local Brick: High performance Velocipraptors in RAID10</p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Sequential Read : 735.224 MB/s<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Sequential Write : 718.203 MB/s<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Random Read 512KB : 888.090 MB/s<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Random Write 512KB : 453.174 MB/s<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Random Read 4KB (QD=1) : 11.808 MB/s [ 2882.9 IOPS]<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Random Write 4KB (QD=1) : 4.249 MB/s [ 1037.4 IOPS]<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Random Read 4KB (QD=32) : 34.787 MB/s [ 8492.8 IOPS]<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Random Write 4KB (QD=32) : 5.487 MB/s [ 1339.5 IOPS]<o:p></o:p></span></p><p><span style='font-family:"Courier New"'><o:p> </o:p></span></p><p><o:p> </o:p></p><p>Node VNA</p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Direct on ZFS (No Gluster)</p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Sequential Read : 2841.216 MB/s<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Sequential Write : 1568.681 MB/s<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Random Read 512KB : 1753.746 MB/s<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Random Write 512KB : 1219.437 MB/s<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Random Read 4KB (QD=1) : 26.852 MB/s [ 6555.6 IOPS]<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Random Write 4KB (QD=1) : 20.930 MB/s [ 5109.8 IOPS]<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Random Read 4KB (QD=32) : 58.515 MB/s [ 14286.0 IOPS]<o:p></o:p></span></p><p style='margin-left:36.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>-<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]><span style='font-family:"Courier New"'> Random Write 4KB (QD=32) : 46.303 MB/s [ 11304.3 IOPS]<o:p></o:p></span></p><p><o:p> </o:p></p><p><o:p> </o:p></p><p><o:p> </o:p></p><p>Performance:<o:p></o:p></p><p>Raw read performance is excellent, averaging 700Mb/s – I’d say the ZFS & Cluster caches are working well.<o:p></o:p></p><p>As mentioned raw write maxed out at 110 MB/s, near the max ethernet speed.<o:p></o:p></p><p>Random I/O is pretty average, it could be the Toshba drives bring things down, though even when I took them out of the equation it wasn’t much improved.<o:p></o:p></p><p><o:p> </o:p></p><p>Direct off the disk was more than double the replica 1 brick in all areas, but I don’t find that surprising. I expected a fair amount of overhead with a cluster fs, and a 1-brick setup is not a real world usage. I was fairly impressed that adding two bricks to replica 3 made no real difference to the read results and the write results were obviously limuted by network speed. If only I could afford 10GB cards and a switch ...</p><p><o:p> </o:p></p><p>I would like to improve the IOPS – these are the current tunables I have set – any suggestions for improvements would be much appreciated:</p><p><o:p> </o:p></p><p><span style='font-family:"Courier New"'><o:p> </o:p></span></p><p><span style='font-family:"Courier New"'>Volume Name: datastore1<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>Type: Replicate<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>Volume ID: 3bda2eee-54de-4540-a556-2f5d045c033a<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>Status: Started<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>Number of Bricks: 1 x 3 = 3<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>Transport-type: tcp<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>Bricks:<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>Brick1: vna.proxmox.softlog:/zfs_vm/datastore1<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>Brick2: vnb.proxmox.softlog:/glusterdata/datastore1<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>Brick3: vng.proxmox.softlog:/glusterdata/datastore1<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>Options Reconfigured:<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>performance.io-thread-count: 32<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>performance.write-behind-window-size: 32MB<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>performance.cache-size: 1GB<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>performance.cache-refresh-timeout: 4<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>nfs.disable: on<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>nfs.addr-namelookup: off<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>nfs.enable-ino32: on<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>diagnostics.client-log-level: WARNING<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>diagnostics.brick-log-level: WARNING<o:p></o:p></span></p><p><span style='font-family:"Courier New"'>performance.write-behind: on<o:p></o:p></span></p><p><o:p> </o:p></p><p>Thanks,</p><p><o:p> </o:p></p><p>Lindsay</p><p><o:p> </o:p></p><p><o:p> </o:p></p><p><o:p> </o:p></p><p><o:p> </o:p></p><p>Sent from <a href="http://go.microsoft.com/fwlink/?LinkId=550986">Mail</a> for Windows 10</p></div></body></html>