InputFormats in hadoop are commonly used to abstract the process of reading input records from mappers.  Here;s how they work:1) The InputFormat itself is defined at Runtime.2) The InputFormat class provides a iterator-like API:  – nextKeyVal…

Please excuse the garbled proverb, I don’t mean to say that people in Rome shouldn’t use GitHub. Instead, I’m talking about using the right tool for the job and the community. Sometimes the “right” tool is completely wrong, because the audience is wrong for the tool. Collaboration works well when you use the right tools. For developers, […]

more virtual big data love w/ gluster, vagrant, and mattf’s little fake disk hack :)For those of you who need to spin up virtual gluster clusters for development and testing: Just finished creating a vagrantized, fully automated, totally rebuildable a…

In my attempt to push puppet to its limits, (for no particular reason), to develop more powerful puppet modules, to build in a distributed lock manager, and to be more dynamic, I’m now attempting to build a Finite State Machine … Continue reading →

0) Vagrant shell provisioner runs as root .  Seems obvious but, if you dont know this you can get burnt.  For example, when it comes to ssh’ing into other machines in your private vagrant cluster.  See (5) for details on that.  In a…

Shortly before VMware’s VSAN was released, I had designed my new lab using GlusterFS across 2 to 4 nodes on my Dell C6100. Since this server did not have a proper RAID card and had 4 nodes total, I needed to design something semi-redundant incase a host were to fail. Scaling: You have a few […]

Slides used by me for the talk on “Distributed Storage in OpenStack” during the recent OpenStack India Day 2013 event can be found here.  The presentation uses GlusterFS as an example for distributed storage and discusses integration between GlusterFS and various OpenStack components.

Theron Conrey writes about using: BitTorrent Sync as Geo-Replication for Storage We got a chance to talk about this idea at Linuxcon. I’m not entirely convinced there aren’t some problem edge cases with this solution, but I think it will … Continue reading →

I’m here in New Orleans hacking up a storm and getting to meet fellow gluster users IRL. John Mark Walker started off with a great “State of the GlusterFS union” style talk. Today Louis (semiosis) gave a great talk about … Continue reading →

After hacking away on Monday and Tuesday and meeting fellow nerds IRL, I’ve landed even more changes to puppet-gluster. My git master branch now sits at 47 commits. $ git clone https://github.com/purpleidea/puppet-gluster.git Cloning into ‘puppet-gluster’… remote: Counting objects: 317, done. … Continue reading →

Continuing on from yesterday, I’ve met even more interesting people. I chatted with Dianne Mueller about some interesting ideas for gluster+openshift. More to come on that front soon. Hung out with Jono Bacon and talked a bit about puppet-gluster on … Continue reading →

I’m here in New Orleans at Linux Con, hacking on puppet-gluster and talking to lots of interesting folks. I’ve met gluster hacker Theron Conrey, and my host John Mark Walker, Fedora and Raspberry Pi experts Spot and Ruth Suehle, and many … Continue reading →

The All-in-One install I detailed in Up and Running with oVirt 3.3 includes everything you need to run virtual machines and get a feel for what oVirt can do, but the downside of the local storage domain type is that it limits … Continue reading →

The oVirt 3.3 release may not quite let you manage all the things in the data center, but it’s getting awfully close. Just shy of six months after the oVirt 3.2 release, the team has delivered an update with groundbreaking … Continue reading →

Last  Saturday on 14th September’13 I gave on GlusterFS presentation at LSPE-IN. The title for the presentation was Performance Characterization in Large distributed file system with GlusterFS . Few days before the talk I looked at the attendee list to get  the … Continue reading →

Next week is a big one for Linux and Open Cloud practitioners and enthusiasts: It’s the annual North American LinuxCon (and CloudOpen), put on by the Linux Foundation – and, following LinuxCon, the Gluster community is putting on a Gluster Community Day on September 19th. Naturally, LinuxCon is on my “don’t miss” list of events, […]

That is a pretty drastic title, especially given that I spend a significant part of my day job working with EMC storage arrays. The other day I replied to a tweet by Scott Lowe :

@scott_lowe with things like Gluster and Ceph what does shared storage a…

The oVirt Project is now putting the finishing touches on version 3.3 of its KVM-based virtualization management platform. The release will be feature-packed, including expanded support for Gluster storage, new integration points for OpenStack’s Neutron networking and Glance image services, … Continue reading →

Hey there, I’ve done a bit of puppet-gluster hacking lately to try to squeeze some extra features and testing in before Linuxcon. Here’s a short list: SELinux fixes to keep Dan Walsh happy 🙂 Ping and status checks before volume … Continue reading →

A git workflow that always give you clean pull requests :)This post is for all you naughty folks out there who forked a github repo, hacked on master, and now are repenting because you have no way of clearly and efficiently sharing your hacks with the …