<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<STYLE type=text/css> P, UL, OL, DL, DIR, MENU, PRE { margin: 0 auto;}</STYLE>
<META name=GENERATOR content="MSHTML 11.00.9600.17924"></HEAD>
<BODY leftMargin=1 rightMargin=1 topMargin=1><FONT size=2 face=Tahoma>
<DIV>Hello,</DIV>
<DIV> </DIV>
<DIV>Running Gluster 3.7.2, distributed(3) / replicated(2) / 20Gb teamed network. Pretty much default options.</DIV>
<DIV><BR>I am having an issue with self heal in Gluster. The second gluster node disconnected overnight, and now large files such as VM images are out of sync.</DIV>
<DIV><BR>The self-heal does not use the full disc IO, nor the full network bandwidth, but it does appear to slow down the volume during a heal, and lock up the files it is healing.</DIV>
<DIV><BR>A simple # df -h takes 1 - 2 minutes, where the volumes are FUSE mounted.</DIV>
<DIV><BR>The VMs using the healing image files freeze completely, also freezing Virt-Manager, and then all of the other VMs either freeze or become slow.</DIV>
<DIV><BR>In the interest of getting our office network operational this morning, I have removed the second node from the network - this has stopped the self heal and allowed us to work with just 1 storage node - but it is very slow and repeatedly freezes for a few seconds every minute or so.</DIV>
<DIV><BR>As I understand, gluster uses granular locking during a heal, so that a VM image may not entirely lock up during the heal / diff checking. However, it appears the entire VM hangs waiting for the heal to finish, with errors such as 'Device or Resource Busy'.</DIV>
<DIV><BR>The effect we are observing with the second node disconnected (few second freezing every few minutes) is was I would expect if it was healing with granular locking, where it would freeze each time it attempted to heal a block that was being accessed.</DIV>
<DIV> </DIV>
<DIV>I have been looking at the qemu / gluster libgfapi, but have been unable to make this work. From what I understand, this will only improve performance. Will this also allow a VM to be accessible during a self heal?</DIV>
<DIV>Other than not using the libgfapi for the VM images, is there anything else I am doing wrong?</DIV>
<DIV>Is there a way to use a VM image whilst it is healing?</DIV>
<DIV> </DIV>
<DIV>Many thanks<BR><BR><BR></DIV></FONT></BODY></HTML>