<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Sep 28, 2016 at 7:37 PM, Shyam <span dir="ltr"><<a href="mailto:srangana@redhat.com" target="_blank">srangana@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On 09/27/2016 04:02 AM, Poornima Gurusiddaiah wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
W.r.t Samba consuming this, it requires a great deal of code change in Samba.<br>
Currently samba has no concept of getting buf from the underlying file system,<br>
the filesystem comes into picture only at the last layer(gluster plugin),<br>
where system calls are replaced by libgfapi calls. Hence, this is not readily<br>
consumable by Samba, and i think same will be the case with NFS_Ganesha, will<br>
let the Ganesha folksc comment on the same.<br>
</blockquote>
<br></span>
This is exactly my reservation about the nature of change [2] that is done in this patch. We expect all consumers to use *our* buffer management system, which may not be possible all the time.<br>
<br>
>From the majority of consumers that I know of, other than what Sachin stated as an advantage for CommVault, none of the others can use the gluster buffers at the moment (Ganesha, SAMBA, qemu. (I would like to understand how CommVault can use gluster buffers in this situation without copying out data to the same, just for clarity).<br></blockquote><div><br></div><div style="">+Jeff cody, for comments on QEMU</div><div style=""> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
This is the reason I posted the comments at [1], stating we should copy out the buffer, when Gluster needs it preserved, but use application provided buffers as long as we can.<br></blockquote><div><br></div><div style="">My concerns here are:</div><div style=""><br></div><div style="">* We are just moving the copy from gfapi layer to write-behind. Though I am not sure what percentage of writes that hit write-behind are "written-back", I would assume it to be a significant percentage (otherwise there is no benefit in having write-behind). However, we can try this approach and get some perf data before we make a decision.</div><div style=""><br></div><div style="">* Buffer management. All gluster code uses iobuf/iobrefs to manage the buffers of relatively large size. With the approach suggested above, I see two concerns:</div><div style=""> a. write-behind has to differentiate between iobufs that need copying (write calls through gfapi layer) and iobufs that can just be refed (writes from fuse etc) when "writing-back" the write. This adds more complexity.</div><div style=""> b. For the case where write-behind chooses to not "write-back" the write, we need a way of encapsulating the application buffer into iobuf/iobref. This might need changes in iobuf infra. </div><div style=""><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I do see the advantages of zero-copy, but not when gluster api is managing the buffers, it just makes it more tedious for applications to use this scheme, IMHO.<br>
<br>
Could we think and negate (if possible) thoughts around using the application passed buffers as is? One caveat here seems to be when using RDMA (we need the memory registered if I am not wrong), as that would involve a copy to RDMA buffers when using application passed buffers.</blockquote><div><br></div><div style="">Actually RDMA is not a problem in the current implementation (ruling out suggestions by others to use a pre-registered iobufs for managing io-cache etc). This is because, in current implementation the responsibility of registering the memory region lies in transport/rdma. In other words transport/rdma doesn't expect pre-registered buffers.</div><div style=""><br></div><div style=""><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> What are the other pitfalls?<br>
<br>
[1] <a href="http://www.gluster.org/pipermail/gluster-devel/2016-August/050622.html" rel="noreferrer" target="_blank">http://www.gluster.org/piperma<wbr>il/gluster-devel/2016-August/<wbr>050622.html</a><br>
<br>
[2] <a href="http://review.gluster.org/#/c/14784/" rel="noreferrer" target="_blank">http://review.gluster.org/#/c/<wbr>14784/</a><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
Regards,<br>
Poornima<br>
</blockquote><div class="HOEnZb"><div class="h5">
______________________________<wbr>_________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">http://www.gluster.org/mailman<wbr>/listinfo/gluster-devel</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">Raghavendra G<br></div>
</div></div>