The Gluster Blog

Gluster blog stories provide high-level spotlights on our users all over the world

libgfapi and the Linux target driver (iSCSI)


This post describes modifications to the Linux Target driver to work with Gluster’s “gfapi” . It is a follow up to an earlier post on Gluster’s block IO performance over iSCSI. Those tests used FUSE, which incurred data copies and context switches. 

That “FUSE penalty” can be avoided using libgfapi. The libgfapi library can be inserted into the Linux target driver rather easily, as it has a nice extensible framework. In this manner IO commands may be forwarded directly to the storage server.

The diagram below depicts gluster’s datapath when FUSE is present and absent. Using gfapi, the red boxes are removed (a kernel trip and extra hop).



To make that happen, I’ve modified the Linux target driver to have a “gluster” module. The implementation was mostly a one–to-one replacement of POSIX APIs with “gluster” APIs. For example read and writes were translated into gf_read and gf_writes.  Some performance results are at the end of this post. 

Feel free to download it off the forge. I expect it will be open sourced after some cleanup, additional testing, etc.

Interestingly, Ceph has already done this. Their plug-in to the Linux target driver uses librados to access the Ceph OSD.  It is not enabled by default – you need to recompile the target driver to use it.



  • The Linux target driver has been replaced by LIO in RHEL7. LIO is a kernel entity similar to SCST. It got into version 2.6.38 (March 2011). The performance implications of using it are unclear. I’ll take a look at that in January.
  • Performance seemed to get worse as the number of streams goes up. This may be due to the synchronous gfapi calls used. A future post will look into some of the options to increase parallelism, such as:
    • The target API has an option to set the number of worker threads.
    • gfapi has asynchronous APIs.



  1. Same 2 node configuration as in this earlier post.
  2. The performance improvement of gfapi in lieu of FUSE varied. But generally, libgfapi bought a 10-20% speedup.
  3. I’ll update this post with random read performance.

libgfapi performance



  • 06 Dec 2020
    Looking back at 2020 – with g...

    2020 has not been a year we would have been able to predict. With a worldwide pandemic and lives thrown out of gear, as we head into 2021, we are thankful that our community and project continued to receive new developers, users and make small gains. For that and a...

    Read more
  • 27 Apr 2020
    Update from the team

    It has been a while since we provided an update to the Gluster community. Across the world various nations, states and localities have put together sets of guidelines around shelter-in-place and quarantine. We request our community members to stay safe, to care for their loved ones, to continue to be...

    Read more
  • 03 Feb 2020
    Building a longer term focus for Gl...

    The initial rounds of conversation around the planning of content for release 8 has helped the project identify one key thing – the need to stagger out features and enhancements over multiple releases. Thus, while release 8 is unlikely to be feature heavy as previous releases, it will be the...

    Read more