<div dir="ltr">Hello,<div><br></div><div>you can try tuning your TCP settings to be more aggressives (both on 1 client and all gluster bricks) in /etc/systctl.conf:</div><div><br></div><div><div>net.core.rmem_max=67108864</div><div>net.core.wmem_max=67108864</div><div># increase Linux autotuning TCP buffer limit to 32MB</div><div>net.ipv4.tcp_rmem="4096 87380 33554432"</div><div>net.ipv4.tcp_wmem="4096 65536 33554432"</div><div># increase the length of the processor input queue</div><div>net.core.netdev_max_backlog=30000</div><div># recommended default congestion control is htcp</div><div>net.ipv4.tcp_congestion_control=htcp</div></div><div><br></div><div><br></div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature">Cordialement,<br>Mathieu CHATEAU<br><a href="http://www.lotp.fr" target="_blank">http://www.lotp.fr</a></div></div>
<br><div class="gmail_quote">2015-07-18 22:11 GMT+02:00 Roman <span dir="ltr"><<a href="mailto:romeo.r@gmail.com" target="_blank">romeo.r@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">But when I'm inside of VM, I'm getting 50-53 MB/s for replica and 120 MB for distributed volumes...<div>whats the logic? that slow write speed directly to mounted gfs volume slows down backup and restore process of VMs by VE.</div></div><div class="gmail_extra"><div><div class="h5"><br><div class="gmail_quote">2015-07-18 22:47 GMT+03:00 Roman <span dir="ltr"><<a href="mailto:romeo.r@gmail.com" target="_blank">romeo.r@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>Looked a lot from forums, but didn't find much..</div><div>So I've got glusterfs 3.6.4</div><div>1 Gbps network</div><div>replicate and distributed volumes</div><div><br></div><div>so when i read from any of them, I get maxed-out network performance (around 80-120 MB/sec)</div><div>If i write to distributed volume, I get 16 MB/sec</div><div>If i write to replicated volume, I get around half of it (which is logical) 8-10MB/sec</div><div><br></div><div>i use XFS (with all set recommended options) and EXT4 for gluster. no difference.</div><div>Disks are in HW RAID5 for Distributed volume and RAID0 for Replicated volume</div><div><br></div><div>volume options:</div><div><br></div><div><div>performance.write-behind: off</div><div>server.allow-insecure: on</div><div>cluster.server-quorum-type: server</div><div>cluster.quorum-type: auto</div><div>network.remote-dio: enable</div><div>cluster.eager-lock: enable</div><div>performance.stat-prefetch: off</div><div>performance.io-cache: off</div><div>performance.read-ahead: off</div><div>performance.quick-read: off</div></div><div><br></div><div>Am I missing something?</div><span><font color="#888888"><div><div><br></div>-- <br><div>Best regards,<br>Roman.</div>
</div></font></span></div>
</blockquote></div><br><br clear="all"><div><br></div></div></div><span class="HOEnZb"><font color="#888888">-- <br><div>Best regards,<br>Roman.</div>
</font></span></div>
<br>_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br></blockquote></div><br></div>