<div dir="ltr">When i have used the packages from "centos-gluster37" to setup my Gluster with ZFS backend, ganesha-nfsd will throw me a ABRT signal when i tried to copy, or simply rsync a directory to the share exported from nfs-ganesha.<div><br></div><div>Environment:</div><div>CentOS 7 - kernel 3.10.0-327.22.2.el7.x86_64</div><div>ZFS Version: 0.6.5.7 Release : 1.el7.centos</div>
<div>Gluster 3.7.13.1.el7 from centos-gluster37</div>
<div>nfs-ganesha 2.3.0.1.el7 from centos-gluster37</div>
<div><br></div><div>This is the only line i got from strace, off from the PID of gaensha-nfsd</div><div>
<p class="gmail-p1"><span class="gmail-s1"><font face="monospace, monospace">futex(0x7f623ffff9d0, FUTEX_WAIT, 38303, NULL <detached ...></font></span></p><p class="gmail-p1"><span class="gmail-s1">In </span>ganesha-gfapi.log, when i try to copy files - the log will pop up the following entries, which keep complaining split-brain, and stale-file-handle, issues in the Gluster volume.<br></p><p class="gmail-p1"><span style="color:rgb(0,0,0);white-space:pre-wrap;line-height:1.38;background-color:transparent"><font size="1" face="monospace, monospace">[2016-08-01 23:06:09.423901] W [MSGID: 108008] [afr-read-txn.c:244:afr_read_txn] 0-nfsvol1-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid 3f713211-7573-45b1-aed8-503c8e17714b. (Possible split-brain)</font></span></p><p class="gmail-p1"><span id="gmail-docs-internal-guid-cca3b4b4-4c70-e8c1-7ef2-0aa06b6e0ea2"><span style="color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"><font size="1" face="monospace, monospace">[2016-08-01 23:06:09.425664] E [MSGID: 109040] [dht-helper.c:1190:dht_migration_complete_check_task] 0-nfsvol1-dht: <gfid:3f713211-7573-45b1-aed8-503c8e17714b>: failed to lookup the file on nfsvol1-dht [Stale file handle]</font></span></span><br></p><p class="gmail-p1"><span><span style="color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"><font size="1" face="monospace, monospace"><br></font></span></span></p><p class="gmail-p1">I tried with both Gluster 3.7.13, and 3.7.12, these versions both give me the same problem. Until i downgrade Gluster to 3.7.11, nfs-ganesha then plays nicely with Gluster. I once wondered if that's related to my ZFS backend setup, then i set something quick in my laptop using XFS as the backend with 3 nodes running Gluster 3.7.13, and nfs-ganesha 2.3.0, and i got the same result. Rsync/Copy files to the NFS shares exported from Gluster+Ganesha aborted after a few files got copied. For your reference, pacemaker+corosync are both still running in the background even when this happens.</p><p class="gmail-p1">I am wondering if there are something introduced since 3.7.12, which somehow breaks the interface between nfs-ganesha, and Gluster. Any pointers will be appreciated. <br></p>
</div></div>