<div dir="ltr"><p class="MsoNormal">Hello,</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">I have a rather odd situation I’m hoping someone can help me
out with. I have a 2 node gluster
replica with an arbiter running on the 3.7 branch. I don’t appear to have a split-brain yet I am
unable able to heal the cluster after a power failure. Perhaps someone can tell me how to fix
this? I don't see much in the logs that may help. ’Here is some command output that
might be helpful:</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">gluster volume info gv0:</p>
<p class="MsoNormal">Volume Name: gv0</p>
<p class="MsoNormal">Type: Replicate</p>
<p class="MsoNormal">Volume ID: 14e7bb9c-aa5e-4386-8dd2-83a88d93dc54</p>
<p class="MsoNormal">Status: Started</p>
<p class="MsoNormal">Number of Bricks: 1 x 3 = 3</p>
<p class="MsoNormal">Transport-type: tcp</p>
<p class="MsoNormal">Bricks:</p>
<p class="MsoNormal">Brick1: server1:/export/brick1</p>
<p class="MsoNormal">Brick2: server2:/export/brick1</p>
<p class="MsoNormal">Brick3: kvm:/export/brick1</p>
<p class="MsoNormal">Options Reconfigured:</p>
<p class="MsoNormal">nfs.acl: off</p>
<p class="MsoNormal">performance.readdir-ahead: on</p>
<p class="MsoNormal">performance.quick-read: off</p>
<p class="MsoNormal">performance.read-ahead: off</p>
<p class="MsoNormal">performance.io-cache: off</p>
<p class="MsoNormal">performance.stat-prefetch: off</p>
<p class="MsoNormal">cluster.eager-lock: enable</p>
<p class="MsoNormal">network.remote-dio: enable</p>
<p class="MsoNormal"><br></p><p class="MsoNormal">--- </p>
<p class="MsoNormal">gluster volume status gv0 detail:</p>
<p class="MsoNormal">Status of volume: gv0</p>
<p class="MsoNormal">------------------------------------------------------------------------------</p>
<p class="MsoNormal">Brick :
Brick server1:/export/brick1</p>
<p class="MsoNormal">TCP Port :
49152</p>
<p class="MsoNormal">RDMA Port :
0</p>
<p class="MsoNormal">Online :
Y</p>
<p class="MsoNormal">Pid : 4409</p>
<p class="MsoNormal">File System :
ext3</p>
<p class="MsoNormal">Device :
/dev/sdb1</p>
<p class="MsoNormal">Mount Options :
rw</p>
<p class="MsoNormal">Inode Size :
128</p>
<p class="MsoNormal">Disk Space Free :
1.7TB</p>
<p class="MsoNormal">Total Disk Space :
1.8TB</p>
<p class="MsoNormal">Inode Count :
244203520</p>
<p class="MsoNormal">Free Inodes :
244203413</p>
<p class="MsoNormal">------------------------------------------------------------------------------</p>
<p class="MsoNormal">Brick :
Brick server2:/export/brick1</p>
<p class="MsoNormal">TCP Port :
49152</p>
<p class="MsoNormal">RDMA Port :
0</p>
<p class="MsoNormal">Online :
Y</p>
<p class="MsoNormal">Pid :
4535</p>
<p class="MsoNormal">File System : ext3</p>
<p class="MsoNormal">Device :
/dev/sdb1</p>
<p class="MsoNormal">Mount Options :
rw</p>
<p class="MsoNormal">Inode Size :
128</p>
<p class="MsoNormal">Disk Space Free :
1.7TB</p>
<p class="MsoNormal">Total Disk Space :
1.8TB</p>
<p class="MsoNormal">Inode Count :
244203520</p>
<p class="MsoNormal">Free Inodes :
244203405</p>
<p class="MsoNormal"><br></p><p class="MsoNormal">--- </p>
<p class="MsoNormal">Why doesn’t this accomplish anything?</p>
<p class="MsoNormal">gluster volume heal gv0:</p>
<p class="MsoNormal">Launching heal operation to perform index self heal on
volume gv0 has been successful</p>
<p class="MsoNormal">Use heal info commands to check status</p>
<p class="MsoNormal"><br></p><p class="MsoNormal">--- </p>
<p class="MsoNormal">Or this?</p>
<p class="MsoNormal">gluster volume heal gv0 full:</p>
<p class="MsoNormal">Launching heal operation to perform full self heal on volume
gv0 has been successful</p>
<p class="MsoNormal">Use heal info commands to check status</p>
<p class="MsoNormal"><br></p><p class="MsoNormal">--- </p>
<p class="MsoNormal">gluster volume heal gv0 info split-brain:</p>
<p class="MsoNormal">Brick server1:/export/brick1</p>
<p class="MsoNormal">Number of entries in split-brain: 0</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Brick server2:/export/brick1</p>
<p class="MsoNormal">Number of entries in split-brain: 0</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Brick kvm:/export/brick1</p>
<p class="MsoNormal">Status: Transport endpoint is not connected</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">--- </p><p class="MsoNormal">I can't seem to get these to heal?</p>
<p class="MsoNormal">gluster volume heal gv0 info:</p>
<p class="MsoNormal">Brick server1:/export/brick1</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/8c524ed9-e382-40cd-9361-60c23a2c1ae2.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/0b16f938-e859-41e3-bb33-fefba749a578.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/715ddb6c-67af-4047-9fa0-728019b49d63.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/b0cdf43c-7e6b-44bf-ab2d-efb14e9d2156.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/d2873b74-f6be-43a9-bdf1-276761e3e228.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/asdf</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/940ee016-8288-4369-9fb8-9c64cb3af256.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/72a33878-59f7-4f6e-b3e1-e137aeb19ced.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/03070877-9cf4-4d55-a66c-fbd3538eedb9.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/c2645723-efd9-474b-8cce-fe07ac9fbba9.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/930196aa-0b85-4482-97ab-3d05e9928884.vhd</p>
<p class="MsoNormal">Number of entries: 12</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Brick server2:/export/brick1</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/03070877-9cf4-4d55-a66c-fbd3538eedb9.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/d2873b74-f6be-43a9-bdf1-276761e3e228.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/715ddb6c-67af-4047-9fa0-728019b49d63.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/930196aa-0b85-4482-97ab-3d05e9928884.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/c2645723-efd9-474b-8cce-fe07ac9fbba9.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/asdf</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/940ee016-8288-4369-9fb8-9c64cb3af256.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/0b16f938-e859-41e3-bb33-fefba749a578.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/8c524ed9-e382-40cd-9361-60c23a2c1ae2.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/72a33878-59f7-4f6e-b3e1-e137aeb19ced.vhd</p>
<p class="MsoNormal">/4b37411d-97cd-0d4c-f898-a3b93cfe1b34/b0cdf43c-7e6b-44bf-ab2d-efb14e9d2156.vhd</p>
<p class="MsoNormal">Number of entries: 12</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Brick kvm:/export/brick1</p>
<p class="MsoNormal">Status: Transport endpoint is not connected</p><div><br></div><div>---</div><div><br></div><div>Thank you.</div>
</div>