<div dir="ltr">That's because they're playing out of Gluster's own playbook: <a href="http://www.gluster.org/community/documentation/index.php/Virt-store-usecase">http://www.gluster.org/community/documentation/index.php/Virt-store-usecase</a><div><br></div><div>The point is that your data corruption issues are vastly more likely to have come from having write-behind enabled than having read-ahead enabled. Having write-behind enabled is like juggling your data with with a partner. Having write-behind disabled is like you and your partner handing data to each other, rather than tossing it. Having read-ahead disabled is like asking your partner for a page of data, and having him give you that page of data. Having read-ahead enabled is like asking your partner for a page of data, and having him give you a fifty page report, because he thinks you may need the extra information--except you already made allowances yourself in asking for that full page of data; the only data you *knew* you needed was a single table in that page.)<br><br>As another example of why you wouldn't normally need read-ahead enabled in gluster, <span style="font-size:13.1999998092651px;line-height:19.7999992370605px">I could easily write a small books' worth of theory into an email detailing the concept further, but I've already given sufficient information to illustrate the relevant concepts; anything further would be unnecessary detail I'm only guessing you might need. ;)</span><br><br><div style="font-size:13.1999998092651px;line-height:19.7999992370605px">The read-ahead setting is about performance, not about data integrity. Virtual machines will be running an operating system. That operating system will be running block-device drivers and filesystem drivers. Both of those types of drivers have their own tunable concepts of read-ahead, so any further read-ahead at the gluster layer is unnecessary.</div><div style="font-size:13.1999998092651px;line-height:19.7999992370605px"><br></div><div style="font-size:13.1999998092651px;line-height:19.7999992370605px">(I'm not suggesting you enable read-ahead in prod; it's pointless. I'm just trying to point out that read-ahead shouldn't *break* anything for you. At the same time, if you enable it, making no other changes, and it *does* break things, that's something people would want to know about.) </div></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Sat, Jul 18, 2015 at 2:42 PM Roman <<a href="mailto:romeo.r@gmail.com">romeo.r@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hi!</div><div>Thanks for reply.</div><div><br></div><a href="https://access.redhat.com/documentation/en-US/Red_Hat_Storage/2.0/html/Quick_Start_Guide/chap-Quick_Start_Guide-Virtual_Preparation.html" target="_blank">https://access.redhat.com/documentation/en-US/Red_Hat_Storage/2.0/html/Quick_Start_Guide/chap-Quick_Start_Guide-Virtual_Preparation.html</a><br><div>reading this, RH recommends to keep read-ahead off for gluster volumes used for VM-s.</div></div><div class="gmail_extra"></div><div class="gmail_extra"><br><div class="gmail_quote">2015-07-18 18:56 GMT+03:00 Michael Mol <span dir="ltr"><<a href="mailto:mikemol@gmail.com" target="_blank">mikemol@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">I think you'll find it's the write-behind that was killing you. Write-behind opens you up to a number of data consistency issues, and I strongly disrecommend it unless you have a rock-solid infrastructure from the writer all the way to the disk the data ultimately sits on.</p>
<p dir="ltr">I bet that if you re-enabled read-ahead, you won't see the problem. Just leave write-behind off.<br>
</p><div><div>
<p dir="ltr"></p>
<p dir="ltr">On Sat, Jul 18, 2015, 10:44 AM Roman <<a href="mailto:romeo.r@gmail.com" target="_blank">romeo.r@gmail.com</a>> wrote:</p>
<blockquote><p dir="ltr">solved after I've added (thanks to Niels de Vos) these options to the volumes:<br></p>
<p dir="ltr">performance.read-ahead: off</p>
<p dir="ltr">performance.write-behind: off<br>
</p>
</blockquote>
<blockquote><p dir="ltr"><br></p>
<p dir="ltr">2015-07-15 17:23 GMT+03:00 Roman <<a href="mailto:romeo.r@gmail.com" target="_blank">romeo.r@gmail.com</a>>:</p>
</blockquote>
<blockquote><blockquote><p dir="ltr">hey,<br></p>
<p dir="ltr">I've updated the bug, if some1 has some ideas - share plz.</p>
<p dir="ltr"><a href="https://bugzilla.redhat.com/show_bug.cgi?id=1242913" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1242913</a><br><br><br></p>
<p dir="ltr">2015-07-14 19:14 GMT+03:00 Kaushal M <<a href="mailto:kshlmster@gmail.com" target="_blank">kshlmster@gmail.com</a>>:</p>
</blockquote>
</blockquote>
<blockquote><blockquote><blockquote><p dir="ltr">Just a wild guess. What is the filesystem used for the debian 8<br>
installation? It could be the culprit.<br></p>
<p dir="ltr">On Tue, Jul 14, 2015 at 7:27 PM, Roman <<a href="mailto:romeo.r@gmail.com" target="_blank">romeo.r@gmail.com</a>> wrote:<br>
> I've done this way: installed debian8 on local disks using netinstall iso,<br>
> created a template of it and then cloned (full clone) it to glusterfs<br>
> storage backend. VM boots and runs fine... untill I start to install<br>
> something massive (DE ie). Last time it was mate failed to install due to<br>
> python-gtk2 package problems (complaing that it could not compile it)<br>
></p>
<p dir="ltr">> 2015-07-14 16:37 GMT+03:00 Scott Harvanek <<a href="mailto:scott.harvanek@login.com" target="_blank">scott.harvanek@login.com</a>>:<br>
>><br>
>> What happens if you install from a full CD and not a net-install?<br>
>><br>
>> Limit the variables. Currently you are relying on remote mirrors and<br>
>> Internet connectivity.<br>
>><br>
>> It's either a Proxmox or Debian issue, I really don't think it's Gluster.<br>
>> We have hundreds of Jessie installs running on GlusterFS backends.<br>
>><br>
>> --<br>
>> Scott H.<br>
>> Login, LLC.<br>
>><br>
>><br>
>><br>
>> Roman<br>
>> July 14, 2015 at 9:30 AM<br>
>> Hey,<br>
>><br>
>> thanks for reply.<br>
>> If it would be networking related, it would affect everything. But it is<br>
>> only debian 8 which won't install.<br>
>> And yes, i did iperf test between gluster and proxmox nodes. Its ok.<br>
>> Installation fails on every node, where i try to install d8. Sometimes it<br>
>> goes well (today 1 of 6 tries was fine). Other distros install fine.<br>
>> Sometimes installation process finishes, but VM won't start, just hangs<br>
>> with errors like in this attached.<br>
>><br>
>><br>
>><br>
>><br>
>> --<br>
>> Best regards,<br>
>> Roman.<br>
>> Scott Harvanek<br>
>> July 14, 2015 at 9:17 AM<br>
>> We don't have this issue, I'll take a stab tho-<br>
>><br>
>> Have you confirmed everything is good on the network side of things?<br>
>> MTU/Loss/Errors?<br>
>><br>
>> Is your inconsistency linked to one specific brick? Have you tried running<br>
>> a replica instead of distributed?<br>
>><br>
>><br>
>><br>
>> _______________________________________________<br>
>> Gluster-users mailing list</p>
<p dir="ltr">>> <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
>> <a href="http://www.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
>> Roman<br>
>> July 14, 2015 at 6:38 AM<br>
>> here is one of the errors example. its like files that debian installer<br>
>> copies to the virtual disk that is located on glusterfs storage are getting<br>
>> corrupted.<br>
>> in-target is /dev/vda1<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> --<br>
>> Best regards,<br>
>> Roman.<br>
>> _______________________________________________<br>
>> Gluster-users mailing list<br>
>> <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
>> <a href="http://www.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
>> Roman<br>
>> July 14, 2015 at 4:50 AM<br>
>> Ubuntu 14.04 LTS base install and then mate install were fine!<br>
>><br>
>><br>
>><br>
>><br>
>> --<br>
>> Best regards,<br>
>> Roman.<br>
>> _______________________________________________<br>
>> Gluster-users mailing list<br>
>> <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
>> <a href="http://www.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
>> Roman<br>
>> July 13, 2015 at 7:35 PM<br>
>> Bah... the randomness of this issue is killing me.<br>
>> Not only HA volumes are affected. Got an error during installation of d8<br>
>> with mate (on python-gtk2 pkg) on Distributed volume also.<br>
>> I've checked the MD5SUM of installation iso, its ok.<br>
>><br>
>> Shortly after that on the same VE node I installed D7 with Gnome without<br>
>> any problem on the HA glusterf volume.<br>
>><br>
>> And on the same VE node I've installed D8 with both Mate and Gnome using<br>
>> local storage disks without problems. There is a bug somewhere in gluster or<br>
>> qemu... Proxmox uses RH kernel btw:<br>
>><br>
>> Linux services 2.6.32-37-pve<br>
>> QEMU emulator version 2.2.1<br>
>> glusterfs 3.6.4<br>
>><br>
>> any ideas?<br>
>> I'm ready to help to investigate this bug.<br>
>> When sun will shine, I'll try to install latest Ubuntu also. But now I'm<br>
>> going to sleep.<br>
>><br>
>><br>
>><br>
>><br>
>> --<br>
>> Best regards,<br>
>> Roman.<br>
>> _______________________________________________<br>
>> Gluster-users mailing list<br>
>> <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
>> <a href="http://www.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
>><br>
>><br>
><br>
><br>
><br>
> --<br>
> Best regards,<br>
> Roman.<br>
><br>
> _______________________________________________<br>
> Gluster-users mailing list<br>
> <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
> <a href="http://www.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
</p>
</blockquote>
</blockquote>
</blockquote>
<blockquote><blockquote><p dir="ltr"><br><br></p>
<p dir="ltr">-- </p>
<p dir="ltr">Best regards,<br>
Roman.<br>
</p>
</blockquote>
</blockquote>
<blockquote><p dir="ltr"><br><br></p>
<p dir="ltr">-- </p>
<p dir="ltr">Best regards,<br>
Roman.</p>
<p dir="ltr">_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
</p>
</blockquote>
<p dir="ltr"><br>
</p>
</div></div></blockquote></div><br><br clear="all"><div><br></div></div><div class="gmail_extra">-- <br><div>Best regards,<br>Roman.</div>
</div></blockquote></div>