As I mentioned yesterday, the GlusterFest is nigh. This time, we’ll break out testing into two types: Performance testing Feature testing To learn about the GlusterFest and what it is, visit the GlusterFest home at gluster.org/gfest Remember that if you file a bug that is verified by the Gluster QE team, you’ll win a t-shirt …Read more

Like a lot of folks, I caught the “4K is for Programmers” post off Hacker News a few days ago (it’s here, but the link seems to be borked at the moment) and got to thinking about more desktop space. Much more. My current setup, when not traveling, involves a 27″ 2560×1440 display – usually […]

If you’re a fan of scale-out storage, datacenter virtualization, or (like me) a mixture of the two, you’ll want to mark your calendar for this pair of upcoming test days for the Gluster and oVirt projects.

This weekend, from Friday the 17th at midnight UTC to Monday the 20th at midnight UTC, the Gluster project is having a test weekend (aka Glusterfest) for its 3.5 release. There’s a breakdown of all the features slated for 3.5 on the release’s planning page.

For this Glusterfest iteration, the project hopes to smooth the process of multi-node testing with the availability of James Shubin’s Puppet-Gluster module. James recently wrote the how-to post, Automatically deploying GlusterFS with Puppet-Gluster + Vagrant, which ought to come in handy this weekend.

After giving James’ post a run-through myself, I’m interested in trying it out with this Vagrant oVirt/RHEV Provider I found on github.

Speaking of oVirt, the project has its own test day coming up on January 23rd, this one for the project’s 3.4 release.

The oVirt 3.4 feature I’m most excited about is the Self Hosted Engine option, in which one or more oVirt compute nodes can host the oVirt management engine. This feature promises to eliminate the engine as a single point of failure, as well as free up the separate machine that’s typically used to host the engine.

If you’re raring to jump on the 3.4 code right away, alpha packages are now available. Check out these release notes for more information about how to get rolling.

For more information on this weekend’s Glusterfest, check out the Gluster project wiki. To find out more about next week’s oVirt 3.4 Test Day, check out the oVirt project wiki.

The first GlusterFS 3.5 Beta is here! See what features made it in over at the 3.5 planning page. Here are some of the marquee features: File Snapshot Quota Scalability Brick Failure Detection On-wire Compression and Decompression Disk Encryption With this first beta, we’ll have the next weekend GlusterFest! We’ll kick it off on Friday, …Read more

Dan Mons came across GlusterFS at his job with Cutting Edge, a VFX company. He needed lots of storage space that was available to many different users – and he needed it to be able to expand as he needed. That it was free and ran on commodity systems was a big plus. Come join …Read more

I always finding my self clicking UI’s and modifying command line incantations to create VMs with new virtual block devices.  This is probably pretty common to anyone who is using VMs to mimic a real world environment where there is a dedicated de…

In about 90 minutes, Louis Zuckerman and I will be “hanging out” and talking about how he came to deploy GlusterFS on AWS, and why he’d developing a Java Filesystem integration with GlusterFS. I’ll post the embedded YouTube link here when we’re about to go live. Hangout starts at 11am EST, 8am PST, 16:00GMT – …Read more

Configuring Glace, Cinder and Nova for OpenStack Havana to run on GlusterFS is actually quite simple; assuming that you’ve already got GlusterFS up and running.

So lets first look at my Gluster configuration. As you can see below, I have a Gluster volume defined for Cinder, Glance and Nova.… Read the rest

The post Configuring OpenStack Havana Cinder, Nova and Glance to run on GlusterFS appeared first on vmware admins.

Some of you have been asking what the Red Hat + CentOS deal means for the Gluster Community, so we’re writing this in response to those inquiries. Some of us work for Red Hat, but the Gluster Community made the strategic decision to explicitly support all distributions, not just those in the Red Hat/RPM universe. …Read more

Puppet-Gluster was always about automating the deployment of GlusterFS. Getting your own Puppet server and the associated infrastructure running was never included “out of the box“. Today, it is! (This is big news!) I’ve used Vagrant to automatically build these … Continue reading →

As we ring in the new year, we also ring in a new release – GlusterFS 3.4.2, available at your local download server! This is a maintenance release, fixing a few bugs, which you can read in the release notes. In addition to bug fixes, you’ll notice that we’re welcoming a new distribution to the …Read more

Heavyweight interfaces sometimes occlude the simple, platform neutral, static directory structure that made maven famous to begin with.  If your server goes down, your jars are unavailable.  S3 solves this problem by providing serverless, htt…

Some fun updates for vagrant hackers… I wanted to use the venerable clustered SSH (cssh) and screen with vagrant. I decided to expand on my vsftp script. First read: Vagrant on Fedora with libvirt and Vagrant vsftp and other tricks … Continue reading →

As I previously wrote, I’ve been busy with Vagrant on Fedora with libvirt, and have even been submitting, patches and issues! (This “closed” issue needs solving!) Here are some of the tricks that I’ve used while hacking away. Default provider: … Continue reading →

The OpenCompute systems are the the ideal hardware platform for distributed filesystems. Period. Why? Cheap servers with 10GB NIC’s and a boatload of locally attached cheap storage!

In preparation for deploying RedHat RDO on RHEL, the distributed filesystem I chose was GlusterFS.… Read the rest

The post Installing GlusterFS on RHEL 6.4 for OpenStack Havana (RDO) appeared first on vmware admins.

Submodules and subtrees are too complicated… Sometimes you can just use git fetch to share code between repos.

Disclaimer: This post is EXTREMELY simple.  It is for begginers in git that want to create composite projects but don’t really want to dive into the labarynthine realm of submodules / subtrees. 

This post outlines an easy and simple way to share code from “project A” into “project B”, using “git fetch”, and leveraging the fact that git supports multiple remotes.  In the end, you’r “project B” will have two remotes: its own, and that of “project A”.  It will replicate the history of project A also (which is not necessarily ideal, but, thats the approach that we will go with – if you want REAL dependency management : use a real dependency manager / build tool like maven/rpm/etc).

This workflow for maintaining a composite git project:

1) Doesnt require custom subtree stuff
2) Won’t trip you up with new semantics in your git workflow
3) Is totally transparent : All history will be in your commit log and no where else.

Why bother posting a manual methodology for managing git based dependent projects?  Because I’m not sure there is a good git based tool for this yet.

A breif aside on the alternatives:

Submodules

“if you forget to update your submodule to the new version, it’s then quite easy to commit the old submodule version in your next parent repository commit – thus effectively reverting the submodule bump by the other developer.” – from http://codingkilledthecat.wordpress.com/2012/04/28/why-your-company-shouldnt-use-git-submodules/.

        Thus : submodules are really complicated.  At first glance, thats not a big deal, but over time, you can lose alot of code and lose commits when submodules arent managed properly, because of the fact that they descend into a detached state.

Subtrees

Subtrees aren’t part of the standard “git” that everyone has.   Its kind of strange to force people to have to set up a “special” git component just to use subtrees.  After all, subtrees aren’t really perfect either !

Since they duplicate the commits and code into a your super repository, they are actually quite similar to a fetch (correct me if im wrong here – im not subtree expert, but they really seem like little more than an source embedding tool with a little extra syntactic sugar). 

A simpler way to share code between projects: git fetch!

I know this is dead simple.  So simple maybe it doesnt even deserve a blog post. But… I bet somebody will find it useful.  After all, there are alot of folks using git out there and not all of them want to spend all day figuring out peripheral git utilities just to pull a few files into their superproject.

So here’s a dead simple example of how to feed code from one project into another using git fetch:

Lets say I have two repos: A community one and a private one.

I can easily pull from my community  into my private by the following workflow:

1. Create two repos (one called “community”,  the other called “private”).
2. Go into your community repo: git@github.com:my_name/my_community_repo.git
1. touch community_file
2. git add community_file ; 
3. git commit -m “first communal commit”
4. git push origin master # <– payload = 1 commit.
3. Go into your private repo:
1. git remote add community git@github.com:my_name/my_community_repo.git
2. git fetch community
3. git merge remotes/community/master 
1. NOTE: If you DONT want to put the commits in with the merge, you can run

   "git merge remotes/community/master --no-commit --no-ff"

4. git push origin master # <– payload = 2 commits.

You can see how it pans out here:

https://github.com/jayunit100/mock_community <– has commits from community repo

And

https://github.com/jayunit100/mock_private <– has commits from community repo + its own commit.

^^ If that looks like what you wanted, then you can (maybe) forget about using the git sub* tools and just read the man page for “git fetch”.

The downside

Okay so , what happens when your code diverges ?  You have to manually pull in the latest from your upstream repo and commit it.  Well… thats not so bad.  At least you know what you have to do and have the tools to do it in your head and in any old git distro.

Moral of the story 

Well, i guess if you want REAL dependency management : you can use a dependency manager like maven, gradle, pip, yum, or whatever.   From what I can tell, git wasn’t built to manage dependencies.  Until then – just fetch from different repositories and push those commits into your own repo 🙂
 
Remember that git really does a great job supporting multiple remotes!  So use them to your advantage 🙂

Louis Zuckerman, CTO of Picture Marketing, is working on not one, but two interesting projects for Gluster. Zuckerman is working on a Java filesystem backed by GlusterFS and Java Native Interface (JNI) bindings for GlusterFS’s native library (libgfapi). Zuckerman says he’s using GlusterFS with storing media for Picture Marketing. “Brand ambassadors use our mobile apps …Read more

This post describes modifications to the Linux Target driver to work with Gluster’s “gfapi” . It is a follow up to an earlier post on Gluster’s block IO performance over iSCSI. Those tests used FUSE, which incurred data copies and context switches.  That “FUSE penalty” can be avoided using libgfapi. The libgfapi library can be inserted into the …Read more

Or “How I learned to start worrying and never trust the cloud.” The Clouderati have been derping for some time now about how we’re all going towards the public cloud and “private cloud” will soon become a distant, painful memory, much like electric generators filled the gap before power grids became the norm. They seem …Read more

Jenkins seems to do EVERYTHING for you , sometimes.  Well.. here’s some stuff it doesnt do:Update !I’ve posted a video on how we glued together our CI according to the diagram above for glusterfs-hadoop.  In particular, it goes over:Setting u…