<div dir="ltr">Sorry for the delayed reply. Had missed out this mail. Please find my comments inlined.<div class="gmail_extra"><br><div class="gmail_quote">On Thu, Dec 31, 2015 at 4:56 AM, Rick Macklem <span dir="ltr"><<a href="mailto:rmacklem@uoguelph.ca" target="_blank">rmacklem@uoguelph.ca</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">Jordan Hubbard wrote:<br>
><br>
> > On Dec 30, 2015, at 2:31 AM, Niels de Vos <<a href="mailto:ndevos@redhat.com">ndevos@redhat.com</a>> wrote:<br>
> ><br>
> >> I'm guessing that Linux uses the event-epoll stuff instead of event-poll,<br>
> >> so it wouldn't exhibit this. Is that correct?<br>
> ><br>
> > Well, both. most (if not all) Linux builds will use event-poll. But,<br>
> > that calls epoll_wait() with a timeout of 1 millisecond as well.<br>
> ><br>
> >> Thanks for any information on this, rick<br>
> >> ps: I am tempted to just crank the timeout of 1msec up to 10 or 20msec.<br>
> ><br>
> > Yes, that is probably what I would do too. And have both poll functions<br>
> > use the same timeout, have it defined in libglusterfs/src/event.h. We<br>
> > could make it a configurable option too, but I do not think it is very<br>
> > useful to have.<br>
><br>
> I guess this begs the question - what’s the actual purpose of polling for an<br>
> event with a 1 millisecond timeout? If it was some sort of heartbeat check,<br>
> one might imagine that would be better served by a timer with nothing close<br>
> to 1 millisecond as an interval (that would be one seriously aggressive<br>
> heartbeat) and if filesystem events are incoming that glusterfs needs to<br>
> respond to, why timeout at all?<br>
><br>
</span>If I understand the code (I probably don't) the timeout allows the loop<br>
to call a function that may add new fd's to be polled. (If I'm right,<br>
the new ones might not get serviced.)<br></blockquote><div><br></div><div style="">Yes, that's correct. Since in poll we pass the fds to be polled in an array as an argument, the only place where we can add/remove fds to be polled is at the time we call poll sycall. To make adding/removing fds from polling to be more responsive, poll timeouts "frequently enough". The trade-off we are considering here is between:</div><div style=""><br></div><div style="">1. Number of calls to poll</div><div style=""> vs</div><div style="">2. Responsiveness of adding/removing a new fd from polling.</div><div style=""><br></div><div style="">For clients, there is not much change of the list of fds that are polled. However, for bricks/server this list can vary frequently as new clients are connected/disconnected.</div><div style=""><br></div><div style="">Since epoll provides a way to add new fds for polling while an epoll_wait is in progress (unlike poll), the timeout of epoll_wait is infinite. Also note that on systems where both epoll and poll are available, epoll is preferred over poll.</div><div style=""><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I'll post once I've tried a longer timeout and if it seems ok, I will<br>
put it in the Redhat bugs database (as mentioned in the last post).<br>
In its current form, it's fine for testing.<br>
<span class=""><br>
> I also have a broader question to go with the specific one: We (at<br>
> iXsystems) were attempting to engage with some of the Red Hat folks back<br>
> when the FreeBSD port was first done, in the hope of getting it more<br>
> “officially supported” for FreeBSD and perhaps even donating some more<br>
> serious stress-testing and integration work for it, but when those Red Hat<br>
> folks moved on we lost continuity and the effort stalled. Who at Red Hat<br>
> would / could we work with in getting this back on track? We’d like to<br>
> integrate glusterfs with FreeNAS 10, and in fact have already done so but<br>
> it’s still early days and we’re not even really sure what we have yet.<br>
><br>
</span>Just fyi..sofar, working with FreeBSD11/head and the port of 3.7.6 (the port tarball<br>
is in FreeBSD PR#194409), the only GlusterFS problem I've encountered is<br>
the above one. I'm not sure why this isn't in /usr/ports, but that would be<br>
nice as it might get more people trying it. (I'm a src comitter, but not a<br>
ports one.)<br>
<br>
However, I have several patches for the FreeBSD fuse interface and for<br>
a mount_glusterfs mount to work ok you need a couple of them.<br>
1 - When an open decides to do DIRECT_IO after the file has done buffer<br>
cache I/O the buffer cache needs to be invalidated so you don't get<br>
stale cached data.<br>
2 - For a WRONLY write, you need to force DIRECT_IO (or do a read/write open).<br>
If you don't do this, the buffer cache code will get stuck when trying<br>
to read a block in before writing a partial block. (I think this is<br>
what FreeBSD PR#194293 is caused by.)<br>
<br>
Because I won't be able to do svn until April, these patches won't make it<br>
into head for a while, but they will both be in PR#194293 within hours.<br>
<br>
The others add features like extended attributes, advisory byte range locking<br>
and the changes needed to export the fuse/glusterfs mount via the FreeBSD<br>
kernel nfsd. If anyone wants/needs these patches, email and I can send you<br>
them.<br>
<br>
A bit off your topic, but until you have the fixes for FreeBSD fuse, you<br>
probably can't do a lot of serious testing.<br>
(I don't know, but I'd guess that FreeNAS has about the same fuse module<br>
code as FreeBSD's head, since it hasn't been changed much in head recently.)<br>
<br>
Thanks everyone for your help with this, rick<br>
<div class="HOEnZb"><div class="h5"><br>
> Thanks,<br>
><br>
> - Jordan<br>
><br>
><br>
_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-devel</a></div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">Raghavendra G<br></div>
</div></div>