The Gluster Blog

Gluster blog stories provide high-level spotlights on our users all over the world

How I do vm management using Puppet, KVM, and GlusterFS on RHEL/CentOS


A GlusterFS user from IRC asked me about my puppet management of KVM in RHEL/CentOS and how it works. I started to write this post two weeks ago and had to stop because although it works great, I figured that wasn’t the answer he was looking for. I looked at the puppet module that I knew I’d hacked on from Carla Souza and after 2 hours of trying to figure out what I had done, gave up and went to bed. I finally was able to look at it again and it wasn’t quite as hacked up as I thought I remembered. I’ve since forked hers and added my changes.

Carla made a really nice module. It’s able to create the vm in several ways and ensure it’s up and running. There were only a few difficiencies with regard to EL that I had to address, and some missing features from newer versions of libvirt. I also added a define to wrap around the class instance so I could more easily set all my defaults.

My node only has this little snippet to define a VM.

  virt::kvm { "blog":
    desc      => "",
    eth1_addr => "",
    ensure    => running,

That’s all wrapped in a define that sets all my standard settings,

define virt::kvm($desc,$eth1_addr,$memory = 1024, $targetmem = 524288, $disk_size = 6, $ensure = running) {
  $hostname = $name

  virt { "$hostname":
    desc        =>  "$desc",
    memory      =>  $memory,
    cpus        =>  4,
    graphics    =>  "disable",
    clocksync   =>  "UTC",
    kickstart   =>  "http://puppet/centos.ks.php?hostname=$hostname&eth1_addr=$eth1_addr ksdevice=eth0 console=ttyS0,115200",
    boot_location => "/mnt/gluster/centos/6/os/x86_64",
    virt_path   =>  "/mnt/gluster/vmimages/$hostname.img",
    disk_size   =>  $disk_size,
    disk_cache  =>  "writethrough",
    os_variant  =>  "virtio26",
    virt_type   =>  "kvm",
    interfaces  =>  ["br0", "br1"],
    autoboot    =>  true,
    ensure      =>  $ensure,
    on_poweroff =>  "destroy",
    on_reboot   =>  "restart",
    on_crash    =>  "restart",

  exec { "/usr/bin/virsh setmem $hostname $targetmem": 
  unless  => "/usr/bin/virsh dommemstat $hostname | /bin/grep $targetmem",
  returns => [0,1] 

As you can see there, I mount GlusterFS volumes in /mnt/gluster. My images are in a volume called vmimages, and I have a CentOS mirror on another volume. I have two bridged network connections, br0 and br1 which map to eth0 and eth1 in the VM. eth1 is defined statically, eth0 has a dhcp server assign it’s address.

Once the image is created and the instance boots for the first time, it reads it’s kickstart file from the web server on the puppet master.


$hostname = $_GET['hostname'];
$eth1_addr = $_GET['eth1_addr'];

echo "
firewall --enabled --ssh
url --url=http://puppet/centos/6/os/x86_64
rootpw --iscrypted \$1\$xxxxx
user --name=myuser --iscrypted --password='\$1\$yyyyy''
network --device eth0 --bootproto dhcp --hostname $hostname --noipv6 --onboot=on
network --device eth1 --bootproto static --ip $eth1_addr --netmask --onboot=on
auth  --useshadow  --enablemd5
keyboard us
lang en_US
selinux --disabled
logging --level=info
timezone --utc America/Los_Angeles
bootloader --location=mbr --append='console=ttyS0'
clearpart --all --initlabel
part /boot --asprimary --fstype='ext4' --size=256
part /var/log --fstype='ext4' --size=512
part / --fstype='ext4' --grow --size=1
part swap --recommended
firstboot --disable
repo --name='Extra Packages for Enterprise Linux 6 - x86_64' --baseurl=''
repo --name='Puppet for EL 6 - x86_64' --baseurl=''
services --enabled puppet,network
services --disabled cups,smartd,autofs,bluetooth,ip6tables,smartd,yum-updatesd,avahi-daemon,NetworkManager
echo 'S0:12345:respawn:/sbin/agetty ttyS0 115200' >> /etc/inittab
echo 'ttyS0' >> /etc/securetty
echo 'IPV6INIT=no' >> /etc/sysconfig/network
echo 'DHCP_HOSTNAME=$hostname' >> /etc/sysconfig/network
sed -i '/^NM_CONTROLLED/d' /etc/sysconfig/network-scripts/ifcfg-eth*
sed -i '/^DHCP_HOSTNAME/d' /etc/sysconfig/network-scripts/ifcfg-eth*
echo 'install ipv6 /bin/true' >> /etc/modprobe.d/noipv6.conf
) 1>/root/post_install.log 2>&1

This creates an install that runs puppet as soon as it reboots, finalizing all it’s configuration.

There are several things that could be improved, like it would be nice to create an exported resource with the VMs on any one node and if that VM is moved to another node’s manifest, it could migrate itself. This is also, obviously, not the best method for everyone, but hopefully this is helpful.


  • 06 Dec 2020
    Looking back at 2020 – with g...

    2020 has not been a year we would have been able to predict. With a worldwide pandemic and lives thrown out of gear, as we head into 2021, we are thankful that our community and project continued to receive new developers, users and make small gains. For that and a...

    Read more
  • 27 Apr 2020
    Update from the team

    It has been a while since we provided an update to the Gluster community. Across the world various nations, states and localities have put together sets of guidelines around shelter-in-place and quarantine. We request our community members to stay safe, to care for their loved ones, to continue to be...

    Read more
  • 03 Feb 2020
    Building a longer term focus for Gl...

    The initial rounds of conversation around the planning of content for release 8 has helped the project identify one key thing – the need to stagger out features and enhancements over multiple releases. Thus, while release 8 is unlikely to be feature heavy as previous releases, it will be the...

    Read more