The thoughtful bodepd has been kind enough to help me get my puppet-gluster module off the ground and publicized a bit too. My first few commits have been all clean up to get my initial hacking up to snuff with … Continue reading →

Apparently, someone in Hadoop-land is getting worried about alternatives to HDFS, and has decided to address that fear via social media instead of code. Two days ago we had Daniel Abadi casting aspersions on Hadoop adapters. Today we have Charles Zedlewski explaining why Cloudera uses HDFS. He mentions a recent GigaOm article listing eight alternatives, […]

It seems that the maintainer of the wireshark package in Fedora has updated to version 1.8.1 in the current Fedora Rawhide, which will become Fedora 18. The schedule tells us that Fedora 18 is planned to be released on 2012-11-06 (the latest schedule m…

It seems that the maintainer of the wireshark package in Fedora has updated to version 1.8.1 in the current Fedora Rawhide, which will become Fedora 18. The schedule tells us that Fedora 18 is planned to be released on 2012-11-06 (the latest schedule m…

I am an avid cobbler+puppet user. This allows me to rely on my cobbler server and puppet manifests to describe how servers/workstations are setup. I only backup my configs and data, and I regenerate failed machines PRN. I’ll be publishing … Continue reading →

(This was originally posted on our Q&A site at community.gluster.org) Problem: VERY slow performance when using ‘bedtools’ and other apps that write zillions of small output chunks. If this was a self-writ app or an infrequently used one, I wouldn’t bother writing this up, but ‘bedtools’ is a fairly popular genomics app and since many …Read more

Many thanks to johnmark in #gluster for syndicating my “gluster” tagged blog posts on http://www.gluster.org/blog/ I aim to keep these posts technical and informative, aimed mostly at other sysadmins and gluster users. Please don’t  be shy to comment on my … Continue reading →

(This is a guest post from Red Hat engineering manager, Vidya Sakar, originally at The Fifth Elephant blog) In this digital universe where data is growing at a fast pace, the infrastructure to store, manage and retrieve data is of paramount importance. Just about everyone in this universe is generating data at a pace never …Read more

Daniel Abadi described his blog entry about Hadoop connectors as a “Stonebraker-style rant” and then delivered on the threat. Like everything Stonebraker has written in the last five years, it’s based on a fundamentally flawed premise, which is that HDFS stores unstructured data. This assumption is not clearly stated, but it’s pretty clear from context, […]

When I first read about reduce side joins in hadoop, I spent some time walking through a bunch of examples from this whitepaper by Jairam Chandar on Hadoop join-algorithms.In the beggining, everything seemed simple enough – because I was focusing on jo…

Using RAW Devices In VirtualBox VMs

Usually, VirtualBox creates its virtual machines in disk images
(.vdi, .vmdk, etc.). This tutorial explains how you can use RAW devices
from the host (partitions, LVM volumes, etc.) and create a VirtualBox VM
in …

I’ve been having some strange networking issues with gluster. “Eco__” from #gluster suggested I try an up to date Intel nic driver. Here are the steps I followed to make that happen. No news yet on if that solved the … Continue reading →

For the last ~two or so years I’ve played and tested gluster on and off and hanging out in the awesome #gluster channel on Freenode. In case you haven’t heard, gluster was acquired by RedHat back in October 2011. This post … Continue reading →

With the addition of automated self-heal in GlusterFS 3.3, a new hidden directory structure was added to each brick: “.glusterfs”. This complicates split-brain resolution as you now not only have to remove the “bad” file from the brick, but it’s counte…

The new hi1.4xlarge instances in EC2 are pretty exciting, not only because they’re equipped with SSDs but because they’re also equipped with 10GbE and placement groups allow you to create server clusters that are closely colocated with full bandwidth among them. I was about ready to do another round of GlusterFS testing to see the […]

OpenVZ: Mounting Host Devices/Partitions/Directories In A Container With Bind Mounts (Debian/Ubuntu)

Sometimes you are in a situation where you need to mount a hard
drive, partiiton or directory from the OpenVZ host inside an OpenVZ
container – fo…

Ring-based consistent hashing is one of my favorite algorithms. It’s an elegant solution to a common set of problems, and many times I’ve seen people’s eyes light up when they realize that finding data among a set of servers doesn’t have to involve central directories or expensive lookups. Systems based on this capability are still […]

Striping Across Four Storage Nodes With GlusterFS 3.2.x On Ubuntu 12.04

This tutorial shows how to do data striping (segmentation of
logically sequential data, such as a single file, so that segments can
be assigned to multiple physical devices in a…

A while ago, I wrote about my High Speed Replication translator, comparing it to the existing method of replication. Bottom line: my version was about 50% faster for small synchronous random writes at the time. Well, times have changed. Here’s part of the commit message for my latest update to HSR. This version works with […]

Node Install
So lets begin by installing the a node. The node install is pretty standard but there are a few very important steps that have to be done or it wont work correctly. All the issues I have found have been put into bugzilla. Before you begi…