<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Sep 15, 2016 at 12:02 PM, Nithya Balachandran <span dir="ltr"><<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On 8 September 2016 at 12:02, Mohit Agrawal <span dir="ltr"><<a href="mailto:moagrawa@redhat.com" target="_blank">moagrawa@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi All,<br><br> I have one another solution to heal user xattr but before implement it i would like to discuss with you.<br> <br> Can i call function (dht_dir_xattr_heal internally it is calling syncop_setxattr) to heal xattr in dht_getxattr_cbk in last <br> after make sure we have a valid xattr.<br> In function(dht_dir_xattr_heal) it will copy blindly all user xattr on all subvolume or i can compare subvol xattr with valid xattr if there is any mismatch then i will call syncop_setxattr otherwise no need to call. syncop_setxattr.<br></div></blockquote><div><br></div><div><br></div></span><div>This can be problematic if a particular xattr is being removed - it might still exist on some subvols. IIUC, the heal would go and reset it again? </div><div><br></div><div>One option is to use the hash subvol for the dir as the source - so perform xattr op on hashed subvol first and on the others only if it succeeds on the hashed. This does have the problem of being unable to set xattrs if the hashed subvol is unavailable. This might not be such a big deal in case of distributed replicate or distribute disperse volumes but will affect pure distribute. However, this way we can at least be reasonably certain of the correctness (leaving rebalance out of the picture).</div></div></div></div></blockquote><div><br></div><div>Yes, this seems fine.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="h5"><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br> Let me know if this approach is suitable.<br><br> <br><br>Regards<span><font color="#888888"><br>Mohit Agrawal<br></font></span></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Sep 7, 2016 at 10:27 PM, Pranith Kumar Karampuri <span dir="ltr"><<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Wed, Sep 7, 2016 at 9:46 PM, Mohit Agrawal <span dir="ltr"><<a href="mailto:moagrawa@redhat.com" target="_blank">moagrawa@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hi Pranith,</div><div><br></div><div><br></div><div>In current approach i am getting list of xattr from first up volume and update the user attributes from that xattr to </div><div>all other volumes.</div><div><br></div><div>I have assumed first up subvol is source and rest of them are sink as we are doing same in dht_dir_attr_heal.</div></div></blockquote><div><br></div></span><div>I think first up subvol is different for different mounts as per my understanding, I could be wrong.<br></div><div><div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><br></div><div>Regards</div><span><font color="#888888"><div>Mohit Agrawal</div></font></span></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Sep 7, 2016 at 9:34 PM, Pranith Kumar Karampuri <span dir="ltr"><<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>hi Mohit,<br></div> How does dht find which subvolume has the correct list of xattrs? i.e. how does it determine which subvolume is source and which is sink?<br></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div>On Wed, Sep 7, 2016 at 2:35 PM, Mohit Agrawal <span dir="ltr"><<a href="mailto:moagrawa@redhat.com" target="_blank">moagrawa@redhat.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr">Hi,<br><br> I am trying to find out solution of one problem in dht specific to user xattr healing.<br> I tried to correct it in a same way as we are doing for healing dir attribute but i feel it is not best solution.<br> <br> To find a right way to heal xattr i want to discuss with you if anyone does have better solution to correct it.<br><br> Problem:<br> In a distributed volume environment custom extended attribute value for a directory does not display correct value after stop/start the brick. If any extended attribute value is set for a directory after stop the brick the attribute value is not updated on brick after start the brick.<br> <br> Current approach:<br> 1) function set_user_xattr to store user extended attribute in dictionary <br> 2) function dht_dir_xattr_heal call syncop_setxattr to update the attribute on all volume <br> 3) Call the function (dht_dir_xattr_heal) for every directory lookup in dht_lookup_revalidate_cbk<br><br> Psuedocode for function dht_dir_xatt_heal is like below<br><br> 1) First it will fetch atttributes from first up volume and store into xattr.<br> 2) Run loop on all subvolume and fetch existing attributes from every volume <br> 3) Replace user attributes from current attributes with xattr user attributes <br> 4) Set latest extended attributes(current + old user attributes) inot subvol.<br><br><br> In this current approach problem is <br> <br> 1) it will call heal function(dht_dir_xattr_heal) for every directory lookup without comparing xattr. <br> 2) The function internally call syncop xattr for every subvolume that would be a expensive operation. <br> <br> I have one another way like below to correct it but again in this one it does have dependency on time (not sure time is synch on all bricks or not)<br><br> 1) At the time of set extended attribute(setxattr) change time in metadata at server side<br> 2) Compare change time before call healing function in dht_revalidate_cbk<br><br> Please share your input on this.<br> Appreciate your input.<br><br>Regards<span><font color="#888888"><br>Mohit Agrawal<br></font></span></div>
<br></div></div><span>______________________________<wbr>_________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">http://www.gluster.org/mailman<wbr>/listinfo/gluster-devel</a><br></span></blockquote></div><span><font color="#888888"><br><br clear="all"><br>-- <br><div data-smartmail="gmail_signature"><div dir="ltr">Pranith<br></div></div>
</font></span></div>
</blockquote></div><br></div>
</div></div></blockquote></div></div></div><span><font color="#888888"><br><br clear="all"><br>-- <br><div data-smartmail="gmail_signature"><div dir="ltr">Pranith<br></div></div>
</font></span></div></div>
</blockquote></div><br></div>
</div></div><br>______________________________<wbr>_________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">http://www.gluster.org/mailman<wbr>/listinfo/gluster-devel</a><br></blockquote></div></div></div><br></div></div>
</blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Pranith<br></div></div>
</div></div>