Several talks related to the Gluster Community have been proposed for the OpenStack Summit in Hong Kong in November. You have to vote for your favorites so that we can be sure to get on the program. Remember to vote early and often! Proposed Talks: Shared Storage: Data Availability Across Clouds and Traditional Datacenters Scaling …Read more

For development of Glance I have recently been using the publicly available Fedora 19 VM which RDO made available here.  From time to time I have found that I need to boot the VM clean (eg: verifying that my environment is not influencing recent changes etc).  In this brief post I will describe how to […]

Recently, I co-authored a patch for Nova that allows for GlusterFS volumes to be accessed directly from Qemu using libgfapi. Previous to this patch, it was only possible to access glusterfs by mounting them using GlusterFS FUSE client.

About QEMU and libgfapi

Recently, in version 1.3, QEMU introduced a GlusterFS block driver based on libgfapi.

Libgfapi is a POSIX-like C library shipped along with GlusterFS, which allows to access Gluster’s volumes without passing through its FUSE client. This integration brings in some benefits but, the most relevant ones are:

• Performance improvements by removing FUSE’s overhead.
• Reduce the number of steps required to get to GlusterFS

There’s no special configuration needed to use this, as long as you have QEMU >=1.3 and GlusterFS>=3.4 you should be fine. This is an example of what you can do:

qemu-img create gluster://$GLUSTER_HOST/$GLUSTER_VOLUME/images 5G

If you’d like to know more about this specific implementation, I suggest you to read this well-explained blog post where all the details about GlusterFS’s driver are explained.

What changed in Nova?

Small changes were required in order to make this work.

Configuration changes

A new configuration parameter – qemu_allowed_storage_drivers – was added to enable or disable this feature. Direct access is disabled by default – this means LibvirtGlusterfsVolumeDriver will mount gluster’s volume – and can be enabled by adding ‘gluster’ to the new configuration parameter.

Code Changes

The patch is quite small – pasted right bellow – and just required to modify libvirt’s configuration object to let QEMU know it should use use a network device and access it using gluster’s protocol.

-        options = connection_info['data'].get('options')
-        path = self._ensure_mounted(connection_info['data']['export'], options)
-        path = os.path.join(path, connection_info['data']['name'])
-        conf.source_type = 'file'
-        conf.source_path = path
+
+        data = connection_info['data']
+
+        if 'gluster' in CONF.qemu_allowed_storage_drivers:
+            vol_name = data['export'].split('/')[1]
+            source_host = data['export'].split('/')[0][:-1]
+
+            conf.source_ports = [None]
+            conf.source_type = 'network'
+            conf.source_protocol = 'gluster'
+            conf.source_hosts = [source_host]
+            conf.source_name = '%s/%s' % (vol_name, data['name'])
+        else:
+            path = self._ensure_mounted(data['export'], data.get('options'))
+            path = os.path.join(path, data['name'])
+            conf.source_type = 'file'
+            conf.source_path = path

Using it

NOTE: From now on, I’ll assume you’ve installed nova, cinder, GlusterFS, libvirt and all the required pieces (I suggest you to use devstack if you’re just testing this feature).

$ # Edit nova.conf and add gluster to the qemu_allowed_storage_drivers list.
$ dd if=/dev/zero of=gluster-volume bs=1M count=4096
$ mkfs.xfs -f gluster-volume

So far we created an XFS loop device. We now have to mount it and add it to glusterfs. Execute as root:

$ mkdir /srv/brick2
$ mount -o loop -t xfs gluster-volume /srv/brick2
$ gluster volume create <ip>:/srv/brick2 testvol1
$ gluster volume start testvol1
$ # Edit /etc/cinder/glusterfs_shares and add "$GLUSTER_HOST:/testvol1"
$ # Restart cinder-volume

We just created a directory for our brick and mounted our loop device there. Then we created a volume in glusterfs, started it and then we added it to the glusterfs_shares file, which is were all glusterfs “shares” are specified. It is important to restart Cinder’s volume service so it can reload the glusterfs_shares file.

We can now create our volume and attach it to our running instance.

$ cinder create 1 # Creates a 1GB volume
$ nova volume-attach <vm> <vol> auto

If nothing bad happened – you know, Murphy – you should see a new device attached to your instance. You can verify this by either ssh’ing into the running instance and listing all available devices – $ ls /dev/vd* – or dumping libvirt instance’s XML.

Enjoy!

(NOTE: if you’re interested in contributing or presenting at the community day, add your ideas in the comments) The Summer of Gluster continues! We’re happy to announce a Community Day for San Francisco on Tuesday, August 27 at Rackspace’s SOMA office in San Francisco. RSVP here. Thanks to Rackspace for agreeing to host – this …Read more

As you may have already heard, the GlusterFS 3.5 release planning cycle is well underway. The deadline is this Thursday, August 14. If you have a feature or patchset you’ve been working on and want to include it in the GlusterFS 3.5 release, please update the page and add your feature proposal to the list. …Read more

In my last blog post on QEMU-GlusterFS, I described the integration of QEMU with GlusterFS using libgfapi. In this post, I give an overview of the recently added discard support to QEMU’s GlusterFS back-end and how it can be used. Newer SCSI devices support UNMAP command that is used to return the unused/freed blocks back […]

I retweeted the other day:

99 little bugs in the code99 little bugs in the codeTake one down, patch it around117 little bugs in the code

Looks like it happened again. In an effort to protect you from having self-heal fill up your root partition shoul…

A few months back I installed RDO on a Fedora 18 and it just worked!  Unfortunately I ran into a few blockers when trying to install it on Fedora 19.  In this post I will describe my experiences of setting up RDO on Fedora 19 and how I made it work. Cheat-Sheet Here is the set […]

We are in the process of formalizing the governance model of the GlusterFS project. Historically, the governance of the project has been loosely structured. This is an invitation to all of you to participate in this discussion and provide your feedback and suggestions on how we should evolve a formal model. Feedback from this thread will […]

When I first started working on OpenStack I spent too much time trying to find a good development environment. I even started a series of blog posts (that I never finished) talking about how I rolled my own development environments. I had messed around with devstack a bit, but I had a hard time configuring […]

A couple times a year someone comes in to #gluster and thinks it would be awesome to ave a puppet module that you could define servers and the pupet module would automatically manage adding those servers as peers and dynamically changing volumes to uti…

Anyone who knows me knows, that I’ve been a VMware user for a long time. I’ve spent a large chunk of my career building virtualization solutions for different companies based on VMware tech. I’ve been active in the VMware community, and I’ve got to say it’s one of the healthiest I’ve seen in a long… Read more »

John Mark Walker, Gluster Community Leader at Red Hat, discusses the current state of open source dominance, what can be

Thanks to a little hack session with bradley childs over at Red Hat this week, I learned a new trick: Remote debugging of JVM (Hadoop + MR2) apps in eclipse. This post summarizes that trick… which has has been derived from other work online…

In this post I will be exploring the current state of quality of service (QoS) in OpenStack.  I will be looking at both what is possible now and what is on the horizon and targeted for the Havana release.  Note that I am truly only intimately familiar with Glance and thus part of the intention […]

Now that GlusterFS 3.4.0 is out, here are some mechanisms to upgrade from earlier installed versions of GlusterFS. Upgrade from GlusterFS 3.3.x: GlusterFS 3.4.0 is compatible with 3.3.x (yes, you read it right!). You can upgrade your deployment by following one of the two procedures below. a) Scheduling a downtime (Recommended) For this approach, schedule […]

We are very pleased to inform you that GlusterFS 3.4 has now hit GA! This marks an incredible milestone for the Gluster community, and pushes GlusterFS into exciting new directions, including virtual block storage, OpenStack integration and a lot more. -> Download here. Changes Of Note Improvements for Virtual Machine Image Storage A number of …Read more

I wanted to take a moment and share all the things that are going on in the Gluster Community. It really has been an amazing year, and we’re only halfway through. Here’s a recap for those of you watching from … Continue reading →

I wanted to take a moment and share all the things that are going on in the Gluster Community. It really has been an amazing year, and we’re only halfway through. Here’s a recap for those of you watching from home:

• Launched the Gluster Community Forge in early May – http://forge.gluster.org/

  • as of now, there are over 20 incubating projects on the Forge, 100 developers and over 1,000 commits to git repositories on the site.
  • future plans include upgrading to gitorious 3, adding integrated bug tracking capability and merging with the global look-and-feel under development for gluster.org

• Announced the charter members of the Gluster Community: Intel, Red Hat, DataLab.es, OSUOSL, Linux Foundation, Hortonworks, NTTPC

  • The Gluster Community is the home of Open Software-defined Storage development
  • Held first board meeting in June

As we looked at the growth of the Gluster Community over the last year, it became clear that the community has evolved to be more than Red Hat, and that we needed a governance model that recognized this growth. For example, there are countless projects scattered across the internet that utilize GlusterFS, but there was no “one-stop shop” to find them. We also knew there are many organizations that contribute to the success of the Gluster Community, but there was no way to formalize their involvement. And finally, we understood that this movement of which we are a part, the movement away from traditional, proprietary storage vendors, needed a name: Open Software-defined Storage.

In response, we have plotted out a series of steps to make the Gluster Community vision grander, more ubiquitous, and more integral to open source cloud and big data communities than ever before. Here are just some of the things that you can expect to see:

• Graduation of incubating projects. Leading candidates thus far include gluster-swift, pmux and gflocator. The former cements our standing in the OpenStack object storage camp, and the latter two form a very interesting project that allows users to conduct file-based Map/Reduce jobs on distributed Gluster volumes.

• GlusterFS 3.4 – we are very very close to GA. Hang tight This is the release that includes QEMU integration and libgfapi, a new client library for developers

• Much better performance for the vast majority of workloads. This will become more apparent when you try the imminent releases of 3.3.2 or 3.4.0.

• Gluster Community Software Distribution. As the Gluster Community forms a software ecosystem around GlusterFS, we will formalize a timely release schedule that allows multiple projects to participate.

• Higher frequency of point releases. This has been a big deal the past year. We have worked hard to fix this, and you’ll notice it very shortly.

• More and better integration with multiple projects that make up OpenStack, CloudStack and Hadoop distributions

• New Gluster.org site with complete redesign from the ground up and new branding

• More Gluster Community Workshops, including at OSCON, LinuxCon North America & Europe, Stockholm, London and more. If you would like to run a Gluster Community Workshop in your area, contact us – cfp@gluster.org

• More presence at OpenStack Summit , Hadoop Summit, Apache CloudStack Collaboration Conference and other related events. Gluster engineers will be more visible than ever at open source cloud and big data events

These steps are essential for building on our momentum and making a successful community that will, in turn, make all participants and collaborators more successful.

Want to be part of a winning team? Get involved – host a meetup, present at a workshop or conference, help out new users on gluster-users and #gluster.

We’re deeply committed to making the Gluster community a wide tent for innovation in cloud storage, and we want to know how we can serve you in this mission. Let us know what you’d like to see from us and how we can best meet your needs:

• Twitter

• Facebook

• Mailing lists

• IRC on Freenode network

We have an amazing community day scheduled in Portland, OR, on July 23. If you’re in town for OSCON, swing by – we’ll be at the Mission Theater, which is close to the MAX for easy access. Here are just a few of the highlights: Theron Conrey will talk about the integration points between OpenStack …Read more