What a busy month this past month for Gluster!
We’ve got updates from SCaLE, FOSDEM, our Developer Gatherings in Brno, DevConf, noteworthy threads from the mailing lists, and upcoming events.
Richard Wareing gave a talk at the Southern California Linux Expo about Scaling Gluster at Facebook.
More at Scaling GlusterFS at Facebook
Humble and Kaushal have posted thoughts on FOSDEM
Me @ FOSDEM 2016
Gluster (and me) @ FOSDEM’16
We had a group of developers gather in Brno ahead of DevConf to discuss a number of different Gluster related things.
Highlights were:
GD2 with Kaushal – https://public.pad.fsfe.org/p/gluster-gd2-kaushal
Heketi & Eventing with Luis – https://public.pad.fsfe.org/p/gluster-heketi
DHT2 with Venky- https://public.pad.fsfe.org/p/gluster-4.0-dht2
Ceph vs Gluster vs Swift: Similarities and Differences – Prashanth Pai, Thiago da Silva
Automated GlusterFS Volume Management with Heketi – Luis Pabon
NFS-Ganesha and Distributed Storage Systems – Kaleb S. Keithley
Build your own Scale-Out Storage with Gluster – Niels de Vos
Freak show (#2): CTDB — Scaling The Aliens Back To Outer Space – Günther Deschner, Michael Adam
oVirt and Gluster Hyperconvergence – Ramesh Nachimuthu
Improvements in gluster for virtualization usecase – Prasanna Kumar Kalever
Test Automation and CI using DiSTAF – Vishwanath Bhat
Noteworthy threads:
Gluster-users
Soumya Koduri investigates the issue of memory leaks in GlusterFS FUSE client and suggests a re-run after application of a few specific patches.
Oleksandr reported that it did not make an impact; Xavier confirmed a similar issue with 3.7.6 release.
The thread is a good read around the topic of understanding how to work through diagnosis and fixes of memory leaks.
Sachidananda provided an update about the gdeploy v2.0 including design changes to enable modularity and separation of core functionality into self-contained unit.
Kyle Harris reported an issue around high I/O and processor utilization.
Ravishankar, Krutika and Pranith worked with the reporter to identify specific ways to address the topic.
A query was raised about the 3.6.8 release notes and a suggestion to include them at download.gluster.org; Niels responded stating that the notes should be part of the repository and added the release manager to provide additional detail.
Vijay provided an update around the changes being discussed for 3.8.
The maintainers feel it is worthwhile to include some of the key features for 4.0 eg. NSR, dht2, glusterd2.0 as experimental in the release; ensure a better component coverage for tests in distaf; add a forward compatibility section to all the feature pages proposed for 3.8 in order to facilitate review for the Gluster.next features. In the same mail Vijay proposed that Niels de Vos would be the maintainer for the 3.8 release. And lastly, the projected GA date for 3.8 is now set to end-May or, early-June 2016.
Ramesh Nachimuthu linked to a blog around designing HyperConverged Infrastructure using oVirt 3.6 and Gluster 3.7.6.
Lindsay Mathieson brought up the topic of ‘File Corruption when adding bricks to live replica volumes.
Further along the discussion Krutika volunteered to write in detail about the specifics of the heal activity and client+server side heal design.
A discussion around heal processes led to a description of a large deployments that consists of over 1000 clients. Again, this is a thread where debugging and diagnosis of GlusterFS issues in large deployments are highlighted along with the typical workload.
DHT2:
A new presentation from Shyam was added around developments in DHT2.
gluster-devel
Pranith and Joseph discuss the issue where using ctime/mtime causes confusion for application software especially for backup which identifies the differences in the values and attempts to suggest remedies.
Ravishankar introduced a proposal around a throttling translator on the server side to regulate FOPS. He believes that it will address frequently posted issues around AFR, Self-Heal consuming high CPU and causing resource starvation. The proposal plans to use the Token Bucket Filter algorithm (also used by bitrot) to regulate the check-sum calculation.
Richard Wareing pointed pointed to a bug report: “Mis-behaving brick clients (gNFSd, FUSE, gfAPI) can cause cluster instability and eventual complete unavailability due to failures in releasing entry/inode locks in a timely manner”. The locks revocation feature was part of his talk at SCaLE14x
Raghavendra Talur has initiated a thread on tips and tricks which can be collated and become an useful resource for new developers contributing to GlusterFS. Vijay added specifics from his work flow and others provided insights.
Niels de Vos picks up the conversation around closing bug reports filed against a mainline version. He proposed that the Bug Report Life Cycle policy could be defined and updated. In addition to that, the script/bugzilla query used to retrieve the bugs could be stored in the release-tools repository, to be run after each release.
As a lead to his talk at FOSDEM on “Gluster Roadmap, Recent Improvements and Upcoming Features”, Niels de Vos sought sought short description from the feature owners/developers to be able to include in his slide deck.
Avra asked for inputs on a possible name for ‘New Style Replication’ (or, NSR) to make it less generic and more representative of the fundamental design idea.
Kaleb provided an update on Python 3.5 being approved for Fedora 24 and suggested that the requirement for Python 2 be looked into and discussed. Prashant suggested usage of “six” module to help maintain code that will run on both Python 2 and Python 3.
Humble responded that the search feature of documentation and readthedocs.org was known to be broken due to rearrangement of the directory structures of ‘features’ and ‘feature planning’ and was expected to be addressed soon.
gluster-infra
Raghavendra Talur informed about enabling comment based triggers on the Jenkins instance.
Michael Scherer informed about a Gerrit outage in order to fix issues around index of the Lucene datastore used by Gerrit. Later on he was happy to note that the time taken was lesser than estimated and things worked according to plan.
In a thread around the possibility of enabling individual accounts (on Jenkins) for maintainers, Michael brought up a topic of governance around who can be considered a ‘contributor’ and is thus entitled to such privileges. He highlights the need for setting clear qualifying criteria which can then use the FreeIPA infrastructure available to access and privilege controls.
2020 has not been a year we would have been able to predict. With a worldwide pandemic and lives thrown out of gear, as we head into 2021, we are thankful that our community and project continued to receive new developers, users and make small gains. For that and a...
It has been a while since we provided an update to the Gluster community. Across the world various nations, states and localities have put together sets of guidelines around shelter-in-place and quarantine. We request our community members to stay safe, to care for their loved ones, to continue to be...
The initial rounds of conversation around the planning of content for release 8 has helped the project identify one key thing – the need to stagger out features and enhancements over multiple releases. Thus, while release 8 is unlikely to be feature heavy as previous releases, it will be the...