GlusterFS communicates over TCP. This allows for stateful handling of file descriptors and locks. If, however, a server fails completely, kernel panic, power loss, some idiot with a reset button… the client will wait for ping-timeout (42 by the defau…

As aweseome as HDFS is, the beauty of a FUSE mounted FileSystem is the fact that you can monitor everything using standard *inx utils (well – as long as they are happening locally at least, thanks martin for pointing that out – see comments below)…&n…

Oh, hi there. In case you’re interested, I’ve just made a first release of my puppet-shorewall module. This isn’t meant as an exhaustive shorewall module, but it does provide most of the usual functionality that most users need. In particular, … Continue reading →

LinuxCon and CloudOpen Europe are just a few days away, and the line-up for talks looks really good. If you’re putting together your schedule, we have a couple of suggestions for talks that you’d probably find interesting. The full schedule … Continue reading →

Doing much work in the cloud? If so, I’d encourage you to take a few minutes to spin up the latest beta test candidate cloud image for Fedora 20. (This is not the final beta release, this is a candidate for the beta release that’s coming shortly.) You can grab the images for x86_64 or […]

If you’re attending LinuxCon Europe, you’ll want to get signed up for the Gluster Workshop on Thursday, October 24th. The program starts at a very reasonable 10 a.m. This full-day, free workshop includes talks on using Gluster with OpenStack, KVM/QEMU, … Continue reading →

This is why everyone uses SLF4J nowadays :)Here is how you track down and update your SLF4J Logging: First, scan the classpath for slf4j bindings.Then, make sure there is only one such binding.Finally, when you find what binding is being used:  Co…

Short post, long command… I’ve decided to start showing the current git branch in my PS1. However, since I don’t want to know when I’m on master, I had to write a new PS1 that I haven’t yet seen anywhere. … Continue reading →

InputFormats in hadoop are commonly used to abstract the process of reading input records from mappers.  Here;s how they work:1) The InputFormat itself is defined at Runtime.2) The InputFormat class provides a iterator-like API:  – nextKeyVal…

Please excuse the garbled proverb, I don’t mean to say that people in Rome shouldn’t use GitHub. Instead, I’m talking about using the right tool for the job and the community. Sometimes the “right” tool is completely wrong, because the audience is wrong for the tool. Collaboration works well when you use the right tools. For developers, […]

more virtual big data love w/ gluster, vagrant, and mattf’s little fake disk hack :)For those of you who need to spin up virtual gluster clusters for development and testing: Just finished creating a vagrantized, fully automated, totally rebuildable a…

In my attempt to push puppet to its limits, (for no particular reason), to develop more powerful puppet modules, to build in a distributed lock manager, and to be more dynamic, I’m now attempting to build a Finite State Machine … Continue reading →

0) Vagrant shell provisioner runs as root .  Seems obvious but, if you dont know this you can get burnt.  For example, when it comes to ssh’ing into other machines in your private vagrant cluster.  See (5) for details on that.  In a…

Shortly before VMware’s VSAN was released, I had designed my new lab using GlusterFS across 2 to 4 nodes on my Dell C6100. Since this server did not have a proper RAID card and had 4 nodes total, I needed to design something semi-redundant incase a host were to fail. Scaling: You have a few […]

Slides used by me for the talk on “Distributed Storage in OpenStack” during the recent OpenStack India Day 2013 event can be found here.  The presentation uses GlusterFS as an example for distributed storage and discusses integration between GlusterFS and various OpenStack components.

Theron Conrey writes about using: BitTorrent Sync as Geo-Replication for Storage We got a chance to talk about this idea at Linuxcon. I’m not entirely convinced there aren’t some problem edge cases with this solution, but I think it will … Continue reading →

I’m here in New Orleans hacking up a storm and getting to meet fellow gluster users IRL. John Mark Walker started off with a great “State of the GlusterFS union” style talk. Today Louis (semiosis) gave a great talk about … Continue reading →

After hacking away on Monday and Tuesday and meeting fellow nerds IRL, I’ve landed even more changes to puppet-gluster. My git master branch now sits at 47 commits. $ git clone https://github.com/purpleidea/puppet-gluster.git Cloning into ‘puppet-gluster’… remote: Counting objects: 317, done. … Continue reading →

Continuing on from yesterday, I’ve met even more interesting people. I chatted with Dianne Mueller about some interesting ideas for gluster+openshift. More to come on that front soon. Hung out with Jono Bacon and talked a bit about puppet-gluster on … Continue reading →

I’m here in New Orleans at Linux Con, hacking on puppet-gluster and talking to lots of interesting folks. I’ve met gluster hacker Theron Conrey, and my host John Mark Walker, Fedora and Raspberry Pi experts Spot and Ruth Suehle, and many … Continue reading →