I always finding my self clicking UI’s and modifying command line incantations to create VMs with new virtual block devices. This is probably pretty common to anyone who is using VMs to mimic a real world environment where there is a dedicated de…
In about 90 minutes, Louis Zuckerman and I will be “hanging out” and talking about how he came to deploy GlusterFS on AWS, and why he’d developing a Java Filesystem integration with GlusterFS. I’ll post the embedded YouTube link here when we’re about to go live. Hangout starts at 11am EST, 8am PST, 16:00GMT – …Read more
Configuring Glace, Cinder and Nova for OpenStack Havana to run on GlusterFS is actually quite simple; assuming that you’ve already got GlusterFS up and running.
So lets first look at my Gluster configuration. As you can see below, I have a Gluster volume defined for Cinder, Glance and Nova.… Read the rest
The post Configuring OpenStack Havana Cinder, Nova and Glance to run on GlusterFS appeared first on vmware admins.
Some of you have been asking what the Red Hat + CentOS deal means for the Gluster Community, so we’re writing this in response to those inquiries. Some of us work for Red Hat, but the Gluster Community made the strategic decision to explicitly support all distributions, not just those in the Red Hat/RPM universe. …Read more
Puppet-Gluster was always about automating the deployment of GlusterFS. Getting your own Puppet server and the associated infrastructure running was never included “out of the box“. Today, it is! (This is big news!) I’ve used Vagrant to automatically build these … Continue reading →
As we ring in the new year, we also ring in a new release – GlusterFS 3.4.2, available at your local download server! This is a maintenance release, fixing a few bugs, which you can read in the release notes. In addition to bug fixes, you’ll notice that we’re welcoming a new distribution to the …Read more
Heavyweight interfaces sometimes occlude the simple, platform neutral, static directory structure that made maven famous to begin with. If your server goes down, your jars are unavailable. S3 solves this problem by providing serverless, htt…
Some fun updates for vagrant hackers… I wanted to use the venerable clustered SSH (cssh) and screen with vagrant. I decided to expand on my vsftp script. First read: Vagrant on Fedora with libvirt and Vagrant vsftp and other tricks … Continue reading →
As I previously wrote, I’ve been busy with Vagrant on Fedora with libvirt, and have even been submitting, patches and issues! (This “closed” issue needs solving!) Here are some of the tricks that I’ve used while hacking away. Default provider: … Continue reading →
The OpenCompute systems are the the ideal hardware platform for distributed filesystems. Period. Why? Cheap servers with 10GB NIC’s and a boatload of locally attached cheap storage!
In preparation for deploying RedHat RDO on RHEL, the distributed filesystem I chose was GlusterFS.… Read the rest
The post Installing GlusterFS on RHEL 6.4 for OpenStack Havana (RDO) appeared first on vmware admins.
Submodules and subtrees are too complicated… Sometimes you can just use git fetch to share code between repos. |
Disclaimer: This post is EXTREMELY simple. It is for begginers in git that want to create composite projects but don’t really want to dive into the labarynthine realm of submodules / subtrees.
This post outlines an easy and simple way to share code from “project A” into “project B”, using “git fetch”, and leveraging the fact that git supports multiple remotes. In the end, you’r “project B” will have two remotes: its own, and that of “project A”. It will replicate the history of project A also (which is not necessarily ideal, but, thats the approach that we will go with – if you want REAL dependency management : use a real dependency manager / build tool like maven/rpm/etc).
This workflow for maintaining a composite git project:
1) Doesnt require custom subtree stuff
2) Won’t trip you up with new semantics in your git workflow
3) Is totally transparent : All history will be in your commit log and no where else.
Why bother posting a manual methodology for managing git based dependent projects? Because I’m not sure there is a good git based tool for this yet.
A breif aside on the alternatives:
Submodules
“if you forget to update your submodule to the new version, it’s then quite easy to commit the old submodule version in your next parent repository commit – thus effectively reverting the submodule bump by the other developer.” – from http://codingkilledthecat.wordpress.com/2012/04/28/why-your-company-shouldnt-use-git-submodules/.
Thus : submodules are really complicated. At first glance, thats not a big deal, but over time, you can lose alot of code and lose commits when submodules arent managed properly, because of the fact that they descend into a detached state.
Subtrees
Subtrees aren’t part of the standard “git” that everyone has. Its kind of strange to force people to have to set up a “special” git component just to use subtrees. After all, subtrees aren’t really perfect either !
Since they duplicate the commits and code into a your super repository, they are actually quite similar to a fetch (correct me if im wrong here – im not subtree expert, but they really seem like little more than an source embedding tool with a little extra syntactic sugar).
A simpler way to share code between projects: git fetch!
I know this is dead simple. So simple maybe it doesnt even deserve a blog post. But… I bet somebody will find it useful. After all, there are alot of folks using git out there and not all of them want to spend all day figuring out peripheral git utilities just to pull a few files into their superproject.
So here’s a dead simple example of how to feed code from one project into another using git fetch:
Lets say I have two repos: A community one and a private one.
I can easily pull from my community into my private by the following workflow:
"git merge remotes/community/master --no-commit --no-ff"
You can see how it pans out here:
https://github.com/jayunit100/mock_community <– has commits from community repo
And
https://github.com/jayunit100/mock_private <– has commits from community repo + its own commit.
^^ If that looks like what you wanted, then you can (maybe) forget about using the git sub* tools and just read the man page for “git fetch”.
The downside
Okay so , what happens when your code diverges ? You have to manually pull in the latest from your upstream repo and commit it. Well… thats not so bad. At least you know what you have to do and have the tools to do it in your head and in any old git distro.
Moral of the story
Well, i guess if you want REAL dependency management : you can use a dependency manager like maven, gradle, pip, yum, or whatever. From what I can tell, git wasn’t built to manage dependencies. Until then – just fetch from different repositories and push those commits into your own repo 🙂
Remember that git really does a great job supporting multiple remotes! So use them to your advantage 🙂
Louis Zuckerman, CTO of Picture Marketing, is working on not one, but two interesting projects for Gluster. Zuckerman is working on a Java filesystem backed by GlusterFS and Java Native Interface (JNI) bindings for GlusterFS’s native library (libgfapi). Zuckerman says he’s using GlusterFS with storing media for Picture Marketing. “Brand ambassadors use our mobile apps …Read more
This post describes modifications to the Linux Target driver to work with Gluster’s “gfapi” . It is a follow up to an earlier post on Gluster’s block IO performance over iSCSI. Those tests used FUSE, which incurred data copies and context switches. That “FUSE penalty” can be avoided using libgfapi. The libgfapi library can be inserted into the …Read more
Or “How I learned to start worrying and never trust the cloud.” The Clouderati have been derping for some time now about how we’re all going towards the public cloud and “private cloud” will soon become a distant, painful memory, much like electric generators filled the gap before power grids became the norm. They seem …Read more
Jenkins seems to do EVERYTHING for you , sometimes. Well.. here’s some stuff it doesnt do:Update !I’ve posted a video on how we glued together our CI according to the diagram above for glusterfs-hadoop. In particular, it goes over:Setting u…
Gluster users have different opinions on when the Gluster daemons should be restarted. This seems to be a very common discussion for a lot daemons, and pops up on the Fedora Developers mailinglist regularly.An explanation on how and when Gluster starts…
Gluster users have different opinions on when the Gluster daemons should be restarted. This seems to be a very common discussion for a lot daemons, and pops up on the Fedora Developers mailinglist regularly.An explanation on how and when Gluster starts…
Cutting Edge, a visual effects company that’s worked on films such as The Great Gatsby and I, Frankenstein, had outgrown its NAS storage system and was in search of a way to boost its storage capacity and performance in the face of several large upcoming projects. The Australia-based firm turned to GlusterFS as an alternative …Read more
Apparently lots of people are using Vagrant these days, so I figured I’d try it out. I wanted to get it working on Fedora, and without Virtualbox. This is an intro article on Vagrant, and what I’ve done. I did … Continue reading →
If you’ve been keeping up with our weekly meetings and the 3.5 planning page, then you know that tomorrow, December 6, is the first testing “day” for 3.5. But since this is a Friday, we’re going to make the party last all weekend, through mid-day Monday. Here’s what you need to do: Take a look …Read more