<div dir="ltr">Re,<div><br></div><div>putting back mailing list so they keep up.</div><div><br></div><div>With newer version, fuse perform much better, and provide transparent failover. As both brick are 2 VM in same host, latency will not be an issue.</div><div>Most important is to ensure you have drivers /tools inside VM to get best perf.</div><div><br></div><div>on client and servers, I use this in /etc/sysctl.conf</div><div>
<p class=""><span class=""><font face="monospace, monospace">vm.swappiness=0<br></font></span><span style="font-family:monospace,monospace">net.core.rmem_max=67108864<br></span><span style="font-family:monospace,monospace">net.core.wmem_max=67108864<br></span><span style="font-family:monospace,monospace"># increase Linux autotuning TCP buffer limit to 32MB<br></span><span style="font-family:monospace,monospace">net.ipv4.tcp_rmem="4096 87380 33554432"<br></span><span style="font-family:monospace,monospace">net.ipv4.tcp_wmem="4096 65536 33554432"<br></span><span style="font-family:monospace,monospace"># increase the length of the processor input queue<br></span><span style="font-family:monospace,monospace">net.core.netdev_max_backlog=30000<br></span><span style="font-family:monospace,monospace"># recommended default congestion control is htcp<br></span><span style="font-family:monospace,monospace">net.ipv4.tcp_congestion_control=htcp</span></p><p class=""><br></p><p class="">options I set on gluster volumes:</p><p class=""><span class=""><font face="monospace, monospace">server.allow-insecure: on<br></font></span><span style="font-family:monospace,monospace">performance.client-io-threads: on<br></span><span style="font-family:monospace,monospace">performance.read-ahead: on<br></span><span style="font-family:monospace,monospace">performance.readdir-ahead: enable<br></span><span style="font-family:monospace,monospace">performance.cache-size: 1GB<br></span><span style="font-family:monospace,monospace">performance.io-thread-count: 16</span></p><p class=""><span style="font-family:monospace,monospace">Options I set on brick in fstab for XFS mounted volumes used by gluster:</span></p><p class="">
</p><p class=""><span class=""><font face="monospace, monospace">defaults,noatime,nodiratime,logbufs=8,logbsize=256k,largeio,inode64,swalloc,allocsize=131072k,nobarrier</font></span></p></div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature">Cordialement,<br>Mathieu CHATEAU<br><a href="http://www.lotp.fr" target="_blank">http://www.lotp.fr</a></div></div>
<br><div class="gmail_quote">2015-08-24 16:11 GMT+02:00 Merlin Morgenstern <span dir="ltr"><<a href="mailto:merlin.morgenstern@gmail.com" target="_blank">merlin.morgenstern@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">re your questions:<div><br></div><div><span class=""><div style="font-size:12.8000001907349px">> did you do some basic tuning to help anyway ?</div><div style="font-size:12.8000001907349px"><br></div></span><div style="font-size:12.8000001907349px">no, this is a basica setup. Can you please direct me to the most important tuning parameters to look at?</div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px">> using latest version ?</div><div style="font-size:12.8000001907349px"><span style="font-size:12.8000001907349px">glusterfs 3.7.3 built on Jul 28 2015 15:14:43</span></div><span class=""><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px">> in replication or only distributed ?</div></span><div style="font-size:12.8000001907349px"><span style="font-size:12.8000001907349px">re 2</span><br></div><span class=""><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px">> Why using NFS and not native fuse client to mount volume?</div></span><div style="font-size:12.8000001907349px">I was reading that the NFS-Client is better with small files. (typical 2-20KB in my case)</div><span class=""><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px">> did you install VM tools (if using VMware fusion) ?</div></span></div><div style="font-size:12.8000001907349px">I am using virtualbox 5.0.3 on Mac OS X 10.10</div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px"><br></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">2015-08-24 16:01 GMT+02:00 Mathieu Chateau <span dir="ltr"><<a href="mailto:mathieu.chateau@lotp.fr" target="_blank">mathieu.chateau@lotp.fr</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hello,<div><br></div><div>did you do some basic tuning to help anyway ?</div><div>using latest version ?</div><div>in replication or only distributed ?</div><div>Why using NFS and not native fuse client to mount volume?</div><div>did you install VM tools (if using VMware fusion) ?</div></div><div class="gmail_extra"><br clear="all"><div><div>Cordialement,<br>Mathieu CHATEAU<br><a href="http://www.lotp.fr" target="_blank">http://www.lotp.fr</a></div></div>
<br><div class="gmail_quote"><div><div>2015-08-24 15:20 GMT+02:00 Merlin Morgenstern <span dir="ltr"><<a href="mailto:merlin.morgenstern@gmail.com" target="_blank">merlin.morgenstern@gmail.com</a>></span>:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr">I am running into trouble while syncing (rsync, cp ... ) my files to glusterfs. After about 50K files, one machine dies and has to be rebooted.<div><br></div><div>As there are about 300K files in one directory, I am thinking about to cluster that in a directory structure in order to overcome that problem.</div><div><br></div><div>e.g. /0001/filename /0002/filename</div><div><br></div><div>That would cut down the amount of files in one directory. However this is something I would like to avoid if possible due to SEO - changing the url of the file brings a lot of trouble.</div><div><br></div><div>The system underneith are 2 seperate VM instances, each running ubuntu 14.04. Cluster NFS client on same machine as Gluster server. Macbook Pro 13 retina with capable SSD and 1G internal network between the VMs.</div><div><br></div><div>Thank you for any help on this.</div></div>
<br></div></div>_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br></blockquote></div><br></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>