<div dir="ltr">Great summary Jeff - thanks!<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 4, 2016 at 3:50 AM, Jeff Darcy <span dir="ltr"><<a href="mailto:jdarcy@redhat.com" target="_blank">jdarcy@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">One of my recurring action items at community meetings is to report to<br>
the list on how 4.0 is going. So, here we go.<br>
<br>
The executive summary is that 4.0 is on life support. Many features<br>
were proposed - some quite ambitious. Many of those *never* had anyone<br>
available to work on them. Of those that did, many have either been<br>
pulled forward into 3.8 (which is great) or lost what resources they had<br>
(which is bad). Downstream priorities have been the biggest cause of<br>
those resource losses, though other factors such as attrition have also<br>
played a part. Net result is that, with the singular exception of<br>
GlusterD 2.0, progress on 4.0 has all but stopped. I'll provide more<br>
details below. Meanwhile, I'd like to issue a bit of a call to action<br>
here, in two parts.<br>
<br>
* Many of the 4.0 sub-projects are still unstaffed. Some of them are<br>
in areas of code where our combined expertise is thin. For example,<br>
"glusterfsd" is where we need to make many brick- and<br>
daemon-management changes for 4.0, but it has no specific maintainer<br>
other than the project architects so nobody touches it. Over the<br>
past year it has been touched by fewer than two patches per month,<br>
mostly side effects of patches which were primarily focused elsewhere<br>
(less than 400 lines changed). It can be challenging to dive into<br>
such a "fallow" area, but it can also be an opportunity to make a big<br>
difference, show off one's skill, and not have to worry much about<br>
conflicts with other developers' changes. Taking on projects like<br>
these is how people get from contributing to leading (FWIW it's how I<br>
did), so I encourage people to make the leap.<br>
<br>
* I've been told that some people have asked how 4.0 is going to affect<br>
existing components for which they are responsible. Please note that<br>
only two components are being replaced - GlusterD and DHT. The DHT2<br>
changes are going to affect storage/posix a lot, so that *might* be<br>
considered a third replacement. JBR (formerly NSR) is *not* going to<br>
replace AFR or EC any time soon. In fact, I'm making significant<br>
efforts to create common infrastructure that will also support<br>
running AFR/EC on the server side, with many potential benefits to<br>
them and their developers. However, just about every other component<br>
is going to be affected to some degree, if only to use the 4.0<br>
CLI/volgen plugin interfaces instead of being hard-coded into their<br>
current equivalents. 4.0 tests are also expected to be based on<br>
Distaf rather than TAP (the .t infrastructure) so there's a lot of<br>
catch-up to be done there. In other cases there are deeper issues to<br>
be resolved, and many of those discussions - e.g. regarding quota or<br>
georep - have already been ongoing. There will eventually be a<br>
Gluster 4.0, even if it happens after I'm retired and looks nothing<br>
like what I describe below. If you're responsible for any part of<br>
GlusterFS, you're also responsible for understanding how 4.0 will<br>
affect that part.<br>
<br>
With all that said, I'm going to give item-by-item details of where we<br>
stand. I'll use<br>
<br>
<a href="http://www.gluster.org/community/documentation/index.php/Planning40" rel="noreferrer" target="_blank">http://www.gluster.org/community/documentation/index.php/Planning40</a><br>
<br>
as a starting point, even though (as you'll see) in some ways it's out<br>
of date.<br>
<br>
* GlusterD 2 is still making good progress, under Atin's and Kaushal's<br>
leadership. There are designs for most of the important pieces, and<br>
a significant amount of code which we should be able to demo soon.<br>
<br>
* DHT2 had been making good progress for a while, but has been stalled<br>
recently as its lead developer (Shyam) has been unavailable.<br>
Hopefully we'll get him back soon, and progress will accelerate<br>
again.<br>
<br>
* Sharding got pulled forward because of its importance for other<br>
efforts, so it's no longer a 4.0 feature.<br>
<br>
* Client-side caching has been dropped for now, though it could still<br>
return with a new design based on the lease infrastructure.<br>
<br>
* Data classification (beyond just tiering) has been dropped.<br>
<br>
* Multiple-network support and network QoS are very much still part of<br>
the 4.0 plan as far as I'm concerned, but there's still nobody<br>
available to work on them.<br>
<br>
* "Better brick management" is also still an un-resourced part of the<br>
4.0 plan. A lot of the higher-level logic will go into Heketi, but<br>
exporting multiple bricks through a single daemon (and port) can't<br>
be.<br>
<br>
* Compression/dedup have been dropped.<br>
<br>
* Composite operations are already being implemented, either as part of<br>
3.8 or as part of the Samba/Ganesha efforts depending on how you look<br>
at it, so that's not a 4.0 feature any more.<br>
<br>
* Stat/xattr caching (on the server) is a bit of a question mark. On<br>
the one hand, it should be pretty simple to implement. On the other<br>
hand, nobody has made even a minimal effort to do so. Recent events<br>
have also raised the issue of needing to do this for correctness<br>
(especially around maintaining ctime across replicas) as well as<br>
performance. This would be a *great* opportunity for a currently<br>
junior/novice Gluster contributor to make their mark.<br>
<br>
* Code generation already exists, and is actively being used to<br>
implement other 4.0 features. My only other comment here is that<br>
people should start using it instead of continuing to use macros in<br>
many cases. Every macro we add is another little nugget of technical<br>
debt, causing all sorts of headaches for anyone who has to edit or<br>
debug the code later. Please do your part to stamp out macro abuse.<br>
<br>
* Management plugins are part of the GlusterD 2 plan.<br>
<br>
* Performance monitoring etc. (last item on list) has been dropped, for<br>
lack of a well defined scope or requirements.<br>
_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
</blockquote></div><br></div>