If you’re a reader of my code or of this blog, it’s no secret that I hack on a lot of puppet and vagrant. Recently I’ve fooled around with a bit of docker, too. I realized that the vagrant, environments … Continue reading →
Over the past few years, there was an enormous increase in the number of user-space filesystems being developed and deployed. But one of the common challenges which all those filesystems’ users had to face was that there was a huge performance hit when their filesystems were exported via kernel-NFS (well-known and widely used network protocol).To […]
Seagate has just publicly announced 8TB HDD’s in a 3.5″ form factor. I decided to do some rough calculations to understand the density a bit better… Note: I have decided to ignore the distinction between Terabytes (TB) and Tebibytes (TiB), since … Continue reading →
Chitika Inc., an online advertising network based in Westborough, MA, sought to provide its data scientists with faster and simpler access to its massive store of ad impression data. The company managed to boost availability and broaden access to its data by swapping out HDFS for GlusterFS as the filesystem backend for its Hadoop deployment. …Read more
I decided to try the upgrade process from EL 6 to 7 on the servers I used in my previous blog post “Windows (CIFS) fileshares using GlusterFS and CTDB for Highly available data” Following the instructions here I found the process fairly painless. However there were 1 or two little niggles which caused various issues which …read more
The post Upgrade CentOS 6 to 7 with Upgrade Tools appeared first on Jon Archer.
Tachyon, an in-memory distributed filesystem, is among the most dynamic projects in big data analytics stack. It provides java io like API, support Apache Spark, and vastly improves Spark’s performance under large data set. As illustrated in this paradigm, Tachyon retrieves data from underlying filesystems (HDFS, S3, Glusterfs, and Posix compliant filesystems), caches data in …Read more
Scenario: You are operating a busy GlusterFS cluster and for whatever reason the volume data gets corrupted. Luckily, you have been backing up the underlying bricks so you are able to restore the bricks to a usable state, but now…
Ovirt is an open source tool used to create/manage gluster nodes through an easy to use web interface. This document is to cover how gluster can be used with ovirt. Want to manage gluster nodes with ease using ovirt ? Create your own ovirt by following these simple steps. Machine Requirements : Fedora19 with 4GB […]
A small blog on how to put Ovirt inside a docker. Install docker on your system. Get an account in docker. pull a base image from docker which ovirt supports. For example : Fedora and centos. Let us install ovirt on centos, by pulling centos base image from docker. Instructions to follow: docker run -i -t centos […]
You want to learn scala. And you want to learn spark. And you’ve heard of SBT. Where do you start?There are alot of different idioms for developing spark apps. One possibility is to use Ipython and Pyspark, which I’ve writ…
GlusterFS 3.5.2beta1 has just been released. This is the first beta to allow users to verify the fixes for the bugs that were reported. See the bug reports below for more details on how to test and confirm the fix (or not). This is a bugfix only releas…
When I’m enjoying the sun/wind/rain on the balcony, I tend to use my XO-1.75 for duties where most people would use a tablet. Reading/writing emails, browsing the internet, bug triaging or writing small fixes, release notes and all can be done fine on …
How to install GlusterFS with a replicated volume over 2 nodes on Ubuntu 14.04
In this tutorial I will explain GlusterFS configuration in Ubuntu 14.04. GlusterFS is an open source distributed file system which provides easy replication over multip…
RPMS are now available for GlusterFS 3.4.5 beta2. We have them for EL5, EL6, EL7, F19, F20, F21, F22 at download.gluster.org [1] with yum repos. [1] http://download.gluster.org/pub/gluster/glusterfs/qa-releases/3.4.5beta2/ This is a bugfix release, mainly for resolving these two important bugs: https://bugzilla.redhat.com/show_bug.cgi?id=1116514 https://bugzilla.redhat.com/show_bug.cgi?id=1116503 All users of GlusterFS 3.4.x are strongly encouraged to test this version, then report …Read more
A few quick notes on adding new packages to RHEL 7.Nice stuff about RHEL 7Docker is now a first class citizen. You can read about how RHEL is moving to support Docker here.Exciting stuff in the EPELsThere’s docker, gluster, and loads of other goo…
One time someone asked me why I liked build tools so much. Here is why.The answer is : because nobody else does. Its like the same reason why my wife is passionate about what kind of crib the kids get. Its because I’m not. You see th…
Simple recipe for spinning up VMs on Libvirt from zerosu -c “yum install @virtualization”This sets up all the virtualization entries. I never remember to use it, but more about why you should use it here.Create disk for a VM and attach itqemu-img…
James (who happens to be a coworker of mine now) recently posted some vagrant on libvirt tutorials. Ironically, I was (I think) one of the original dudes who prodded him to post about vagrant (specifically to demonstrate his puppet-gluster …
GlusterFS 3.4.5beta1 RPMs for el5-7 (RHEL, CentOS, etc.) and Fedora (19, 20, 21/rawhide), are now available in YUM repos at http://download.gluster.org/pub/gluster/glusterfs/qa-releases/3.4.5beta1/ These packages include the fix for bz# https://bugzilla.redhat.com/show_bug.cgi?id=1112844 All users of GlusterFS 3.4.x are strongly encouraged to test this version. We welcome your suggestions/comments/feedback about this release through the GlusterFS Developers mailing list. Mailing …Read more
This tutorial will walk through the setup and configuration of GlusterFS and CTDB to provide highly available file storage via CIFS. GlusterFS is used to replicate data between multiple servers. CTDB provides highly available CIFS/Samba functionality. Prerequisites: 2 servers (virtual or physical) with RHEL 6 or derivative (CentOS, Scientific Linux). When installing create a partition …read more
The post Windows (CIFS) fileshares using GlusterFS and CTDB for Highly available data appeared first on Jon Archer.