The Gluster Blog

Gluster blog stories provide high-level spotlights on our users all over the world

Hacking Filesystems Is Easy


If you want to hack on distributed filesystems, there is no easier way to get started than by writing a GlusterFS translator. To prove this point, I’ve recently implemented two new translators which are very simple but provide significant benefits in certain situations. These have nothing to do with HekaFS, really, except that HekaFS takes advantage of this same simplicity to do what it does. The first translator does negative lookup caching.

This is a very simple translator to cache “negative lookups” for workloads in which the same file is looked up many times in places where it doesn’t exist. In particular, web script files with many includes/requires and long paths can generate hundreds of such lookups per front-end request. If we don’t cache the negative results, this can mean hundreds of back-end network round trips per front-end request. So we cache. Very simple tests for this kind of workload on two machines connected via GigE show an approximately 3x performance improvement.

The second translator bypasses replication.

This is a proof-of-concept translator for an idea that was proposed at FUDcon 2012 in Blacksburg, VA. The idea is simply that we can forward writes only to local storage, bypassing AFR but setting the xattrs ourselves to indicate that self-heal is needed. This gives us near-local write speeds, and we can mount later without the bypass to force self-heal when it’s convenient. We can do almost the same thing for reads as well.

They weigh in at 224 and 229 lines respectively, with some of that taken up by licenses and white space. Each took less than a day to write. Please bear in mind, though, that these are only prototypes. They exist to teach and to make a point, not – in their current form – to be used in production. Making them suitable for real-world use would at least double their size and triple the time needed for testing. That’s still orders of magnitude better than what you’d have to do to implement similar functionality in other projects that claim to be competitive with GlusterFS, and the result is still much more functional than one of those stripped-down jokes that just have “FS” in the name to mislead users. If you’re a developer and you think you can do distribution or replication or caching or anything else better than GlusterFS, show us. Translators let you implement your ideas quickly, and then do a true “apples to apples” comparison vs. what came before. That could revolutionize distributed storage, but only if people take advantage of the opportunity.


  • 06 Dec 2020
    Looking back at 2020 – with g...

    2020 has not been a year we would have been able to predict. With a worldwide pandemic and lives thrown out of gear, as we head into 2021, we are thankful that our community and project continued to receive new developers, users and make small gains. For that and a...

    Read more
  • 27 Apr 2020
    Update from the team

    It has been a while since we provided an update to the Gluster community. Across the world various nations, states and localities have put together sets of guidelines around shelter-in-place and quarantine. We request our community members to stay safe, to care for their loved ones, to continue to be...

    Read more
  • 03 Feb 2020
    Building a longer term focus for Gl...

    The initial rounds of conversation around the planning of content for release 8 has helped the project identify one key thing – the need to stagger out features and enhancements over multiple releases. Thus, while release 8 is unlikely to be feature heavy as previous releases, it will be the...

    Read more