<div dir="ltr"><div><div><div><div><div><div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On 18 October 2015 at 00:17, Vijay Bellur <span dir="ltr"><<a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div id=":1pp" class="" style="overflow:hidden">Krutika
has been working on several performance improvements for sharding and
the results have been encouraging for virtual machine workloads.<br>
<br>
Testing feedback would be very welcome!<br>
</div></blockquote></div><br></div><div class="gmail_extra">I've managed to setup a replica 3 3.7.5 shard test volume, hosted using virtualised debian 8.2 servers, so performance is a bit crap :)<br><br></div><div class="gmail_extra">3 Nodes, gn1, hn2 & gn3<br></div><div class="gmail_extra">Each node has:<br></div><div class="gmail_extra">- 1GB RAM<br></div><div class="gmail_extra">- 1GB Ethernet<br></div><div class="gmail_extra">- 512 GB disk hosted on a ZFS External USB Drive :)<br></div><div class="gmail_extra"><br></div><div class="gmail_extra">- Datastore is shared out via NFS to the main cluster for running a VM<br></div><div class="gmail_extra">- I have the datastore mounted using glusterfs inside each test node so I can examine the data directly.<br></div><div class="gmail_extra"><br></div><div class="gmail_extra"><br><br></div><div class="gmail_extra">I've got two VM's running off it, one a 65GB (25GB sparse) Windows 7. I'be running bench marks and testing node failures by killing the cluster processes and killing actual nodes.<br><br></div><div class="gmail_extra">- Heal speed is immensely faster, a matter of minutes rather than hours.<br></div>- Read performance is quite good<br></div>- Write performance is atrocious, but given the limited resources not unexpected. <br>- I'll be upgrading my main cluster to jessie soon and will be able to test with real hardware and bonded connections, plus using gfapi direct. Then I'll be able to do real benchmarks.<br><br></div><div>One Bug:<br></div>After heals completed I shut down the VM's and run a MD5SUM on the VM image (via glusterfs) on each nodes. They all matched except for one time on gn3. Once I unmounted/remounted the datastore on gn3 the md5sum matched.<br><br></div>One Oddity:<br></div>gluster volume heals datastore info *always* shows a split brain on the directory, but it always heals without intervention. Dunno if this is normal on not.<br><br></div><div>Questions:<br></div>- I'd be interested to know how the shard are organsied and accessed - it looks like 1000's of 4mb files in the .shard directory, I'm concerned access times will go in the toilet once many large VM images are stored on the volume.<br><br></div>- Is it worth experimenting with different shard sizes?<br><br></div>- Anything you'd like me to test?<br><br></div>Thanks,<br clear="all"><div><div><div><div><div><div><div><div><div><br clear="all"><br>-- <br><div class="gmail_signature">Lindsay</div>
</div></div></div></div></div></div></div></div></div></div>