<div dir="ltr"><div><br></div><div>I can understand the concern. But I think instead of generally converting all the ESTALE errors ENOENT, probably we should try to analyze the errors that are generated by lower layers (like posix).</div><div><br></div><div>Even fuse kernel module some times returns ESTALE. (Well, I can see it returning ESTALE errors in some cases in the code. Someone please correct me if thats not the case). And aso I am not sure if converting all the ESTALE errors to ENOENT is ok or not.</div><div><br></div><div>For fd based operations, I am not sure if ENOENT can be sent or not (as though the file is unlinked, it can be accessed if there were open fds on it before unlinking the file).</div><div><br></div><div>I feel, we have to look into some parts to check if they generating ESTALE is a proper error or not. Also, if there is any bug in below layers fixing which can avoid ESTALE errors, then I feel that would be the better option.</div><div><br></div><div>My 2 cents.</div><div><br></div><div>Regards,</div><div>Raghavendra</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 28, 2016 at 1:39 AM, Prashanth Pai <span dir="ltr"><<a href="mailto:ppai@redhat.com" target="_blank">ppai@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">TL;DR: +1 to report ESTALE as ENOENT<br>
<br>
While ESTALE is an acceptable errno for NFS clients, it's not so much for<br>
FUSE clients. Many applications that talk to a FUSE mount do not handle<br>
ESTALE and expect the behavior to be analogous to that of local<br>
filesystems such as XFS. While it's okay for brick to send ESTALE to<br>
glusterfs client stack, one has to be very careful about errno returned by<br>
FUSE back to applications.<br>
<br>
For example, syscalls such as fgetxattr are not expected (at least from<br>
manpage) to throw ESTALE but with glusterfs, it does[1]. Further, POSIX<br>
guarantees that once an application has a valid fd, operations like<br>
fgetxattr() on the fd should succeed even after another application(client)<br>
issues an unlink()<br>
<br>
[1]:<a href="http://paste.openstack.org/show/335506/" rel="noreferrer" target="_blank">http://paste.openstack.org/show/335506/</a><br>
<br>
Regards,<br>
-Prashanth Pai<br>
<div><div class="h5"><br>
----- Original Message -----<br>
> From: "FNU Raghavendra Manjunath" <<a href="mailto:rabhat@redhat.com">rabhat@redhat.com</a>><br>
> To: "Soumya Koduri" <<a href="mailto:skoduri@redhat.com">skoduri@redhat.com</a>><br>
> Cc: "Ira Cooper" <<a href="mailto:icooper@redhat.com">icooper@redhat.com</a>>, "Gluster Devel" <<a href="mailto:gluster-devel@gluster.org">gluster-devel@gluster.org</a>><br>
> Sent: Thursday, March 24, 2016 8:11:19 PM<br>
> Subject: Re: [Gluster-devel] Report ESTALE as ENOENT<br>
><br>
><br>
> I would still prefer not converting all the ESTALE to ENOENT. I think we need<br>
> to understand this specific case of parallel rm -rfs getting ESTALE errors<br>
> and handle it accordingly.<br>
><br>
> Regarding, gfapi not honoring the ESTALE errors, I think it would be better<br>
> to do revalidates upon getting ESTALE.<br>
><br>
> Just my 2 cents.<br>
><br>
> Regards,<br>
> Raghavendra<br>
><br>
><br>
> On Thu, Mar 24, 2016 at 10:31 AM, Soumya Koduri < <a href="mailto:skoduri@redhat.com">skoduri@redhat.com</a> > wrote:<br>
><br>
><br>
> Thanks for the information.<br>
><br>
> On 03/24/2016 07:34 PM, FNU Raghavendra Manjunath wrote:<br>
><br>
><br>
><br>
> Yes. I think the caching example mentioned by Shyam is a good example of<br>
> ESTALE error. Also User Serviceable Snapshots (USS) relies heavily on<br>
> ESTALE errors. Because the files/directories from the snapshots are<br>
> assigned a virtual gfid on the fly when being looked up. If those inodes<br>
> are purged out of the inode table due to lru list becoming full, then a<br>
> access to that gfid from the client, will make snapview-server send<br>
> ESTALE and either fuse (I think our fuse xlator does a revalidate upon<br>
> getting ESTALE) or NFS client can revalidate via path based resolution.<br>
><br>
> So wouldn't it be wrong not to send ESTALE to NFS-clients and map it to<br>
> ENOENT, as was intended in the original mail.<br>
><br>
> NFSv3 rfc [1] mentions that NFS3ERR_STALE is a valid error for REMOVE fop.<br>
><br>
> Also (at least in gfapi) the resolve code path doesn't seem to be honoring<br>
> ESTALE errors - glfs_resolve_component(..), glfs_refresh_inode_safe(..)<br>
> etc.. Do we need to fix them?<br>
><br>
><br>
> Thanks,<br>
> Soumya<br>
><br>
> [1] <a href="https://www.ietf.org/rfc/rfc1813.txt" rel="noreferrer" target="_blank">https://www.ietf.org/rfc/rfc1813.txt</a> (section# 3.3.12)<br>
><br>
><br>
><br>
><br>
> Regards,<br>
> Raghavendra<br>
><br>
><br>
> On Thu, Mar 24, 2016 at 9:51 AM, Shyam < <a href="mailto:srangana@redhat.com">srangana@redhat.com</a><br>
> <mailto: <a href="mailto:srangana@redhat.com">srangana@redhat.com</a> >> wrote:<br>
><br>
> On 03/23/2016 12:07 PM, Ravishankar N wrote:<br>
><br>
> On 03/23/2016 09:16 PM, Soumya Koduri wrote:<br>
><br>
> If it occurs only when the file/dir is not actually present<br>
> at the<br>
> back-end, shouldn't we fix the server to send ENOENT then?<br>
><br>
> I never fully understood it here is the answer:<br>
> <a href="http://review.gluster.org/#/c/6318/" rel="noreferrer" target="_blank">http://review.gluster.org/#/c/6318/</a><br>
><br>
><br>
> The intention of ESTALE is to state that the inode#/GFID is stale,<br>
> when using that for any operations. IOW, we did not find the GFID in<br>
> the backend, that does not mean the name is not present. This hence<br>
> means, if you have a pGFID+bname, try resolving with that.<br>
><br>
> For example, a client side cache can hang onto a GFID for a bname,<br>
> but another client could have, in the interim, unlinked the bname<br>
> and create a new file there.<br>
><br>
> A presence test using GFID by the client that cached the result the<br>
> first time, is an ESTALE. But a resolution based on pGFID+bname<br>
> again by the same client would be a success.<br>
><br>
> By extension, a GFID based resolution, when not really present in<br>
> the backend will return ESTALE, it could very well mean ENOENT, but<br>
> that has to be determined by the client again, if possible.<br>
><br>
> See "A10. What does it mean when my application fails because of an<br>
> ESTALE error?" for NFS here [1] and [2] (there maybe better sources<br>
> for this information)<br>
><br>
> [1] <a href="http://nfs.sourceforge.net/" rel="noreferrer" target="_blank">http://nfs.sourceforge.net/</a><br>
> [2] <a href="https://lwn.net/Articles/272684/" rel="noreferrer" target="_blank">https://lwn.net/Articles/272684/</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Gluster-devel mailing list<br>
> <a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a> <mailto: <a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a> ><br>
> <a href="http://www.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
><br>
> _______________________________________________<br>
> Gluster-devel mailing list<br>
> <a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a> <mailto: <a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a> ><br>
> <a href="http://www.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Gluster-devel mailing list<br>
> <a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
</div></div>> <a href="http://www.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
</blockquote></div><br></div>