<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 19, 2015 at 5:40 PM, Raghavendra G <span dir="ltr"><<a href="mailto:raghavendra@gluster.com" target="_blank">raghavendra@gluster.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div>After discussion with Vijaykumar mallikarjuna and other inputs in this thread, we are proposing all quota tests to comply to following criteria:<br><br></div>* use dd always with oflag=append (to make sure there are no parallel writes) and conv=fdatasync (to make sure errors, if any are delivered to application. Turning off flush-behind is optional since fdatasync acts as a barrier)<br><br></div><div>OR<br><br></div>* turn off write-behind in nfs client and glusterfs server.<br></div></div></div></div></div></blockquote><div><br></div><div>s/glusterfs server/glusterfs nfs server.<br> <br></div><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div><div><br></div>What do you people think is a better test scenario?<br><br></div>Also, we don't have confirmation on the RCA that parallel writes are indeed the culprits. We are trying to reproduce the issue locally. @Shyam, it would be helpful if you can confirm the hypothesis :).<br><br></div>regards,<br></div>Raghavendra.<br></div><div class="gmail_extra"><div><div class="h5"><br><div class="gmail_quote">On Tue, May 19, 2015 at 5:27 PM, Raghavendra G <span dir="ltr"><<a href="mailto:raghavendra@gluster.com" target="_blank">raghavendra@gluster.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Tue, May 19, 2015 at 4:26 PM, Jeff Darcy <span dir="ltr"><<a href="mailto:jdarcy@redhat.com" target="_blank">jdarcy@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span>> No, my suggestion was aimed at not having parallel writes. In this case quota<br>
> won't even fail the writes with EDQUOT because of reasons explained above.<br>
> Yes, we need to disable flush-behind along with this so that errors are<br>
> delivered to application.<br>
<br>
</span>Would conv=sync help here? That should prevent any kind of write parallelism.<br></blockquote><div><br></div></span><div>An strace of dd shows that<br><br></div><div>* fdatasync is issued only once at the end of all writes when conv=fdatasync<br></div><div>* for some strange reason no fsync or fdatasync is issued at all when conv=sync<br><br></div><div>So, using conv=fdatasync in the test cannot prevent write-parallelism induced by write-behind. Parallelism would've been prevented only if dd had issued fdatasync after each write or opened the file with O_SYNC.<br></div><span><div><br></div><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
If it doesn't, I'd say that's a true test failure somewhere in our stack. A<br>
similar possibility would be to invoke dd multiple times with oflag=append.<br></blockquote><div><br></div></span><div>Yes, appending writes curb parallelism (at least in glusterfs, but not sure how nfs client behaves) and hence can be used as an alternative solution.<br><br></div><div>On a slightly unrelated note flush-behind is immaterial in this test since fdatasync is anyways acting as a barrier.<br><br></div><span><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div><div>_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-devel" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
</div></div></blockquote></span></div><span><font color="#888888"><br><br clear="all"><br>-- <br><div>Raghavendra G<br></div>
</font></span></div></div>
</blockquote></div><br><br clear="all"><br></div></div><span class="HOEnZb"><font color="#888888">-- <br><div>Raghavendra G<br></div>
</font></span></div>
</blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature">Raghavendra G<br></div>
</div></div>