[Gluster-users] Errors from PHP stat() on files and directories in a glusterfs mount

Thu Dec 5 19:05:20 UTC 2013

Hi,

I'm trying to use glusterfs to mirror the ownCloud "data" area between 2
servers.

They are using debian jessie due to some dependancies that I have for
other components.

This is where my issue rears it's ugly head. This is failing because I
can't stat the files and directories on my glusterfs mount.

/var/www/owncloud/data is where I am mounting the volume and I can
reproduce the error using a simple php test application, so I don't
think that it is apache or owncloud related.

I'd be grateful for any pointers on how to resolve this problem.

Thanks,

    Bill

Attached is "simple.php" test and the results of executing "strace php5
simple.php" twice, once with the glusterfs mounted
(simple.php.strace-glusterfs) and once against the file system when
unmounted (simple.php.strace-unmounted).

------------------------------------------------------------------------

Here is what I get in the gluster log when I run the test (as root):

/var/log/glusterfs/var-www-owncloud-data.log

[2013-12-05 18:33:50.802250] D
[client-handshake.c:185:client_start_ping] 0-gv-ocdata-client-0:
returning as transport is already disconnected OR there are no frames (0
|| 0)
[2013-12-05 18:33:50.825132] D
[afr-self-heal-common.c:138:afr_sh_print_pending_matrix]
0-gv-ocdata-replicate-0: pending_matrix: [ 0 0 ]
[2013-12-05 18:33:50.825322] D
[afr-self-heal-common.c:138:afr_sh_print_pending_matrix]
0-gv-ocdata-replicate-0: pending_matrix: [ 0 0 ]
[2013-12-05 18:33:50.825393] D
[afr-self-heal-common.c:887:afr_mark_sources] 0-gv-ocdata-replicate-0:
Number of sources: 0
[2013-12-05 18:33:50.825456] D
[afr-self-heal-data.c:825:afr_lookup_select_read_child_by_txn_type]
0-gv-ocdata-replicate-0: returning read_child: 0
[2013-12-05 18:33:50.825511] D
[afr-common.c:1380:afr_lookup_select_read_child]
0-gv-ocdata-replicate-0: Source selected as 0 for /
[2013-12-05 18:33:50.825579] D
[afr-common.c:1117:afr_lookup_build_response_params]
0-gv-ocdata-replicate-0: Building lookup response from 0
[2013-12-05 18:33:50.827069] D
[afr-common.c:131:afr_lookup_xattr_req_prepare] 0-gv-ocdata-replicate-0:
/check.txt: failed to get the gfid from dict
[2013-12-05 18:33:50.829409] D
[client-handshake.c:185:client_start_ping] 0-gv-ocdata-client-0:
returning as transport is already disconnected OR there are no frames (0
|| 0)
[2013-12-05 18:33:50.836719] D
[afr-self-heal-common.c:138:afr_sh_print_pending_matrix]
0-gv-ocdata-replicate-0: pending_matrix: [ 0 0 ]
[2013-12-05 18:33:50.836870] D
[afr-self-heal-common.c:138:afr_sh_print_pending_matrix]
0-gv-ocdata-replicate-0: pending_matrix: [ 0 0 ]
[2013-12-05 18:33:50.836941] D
[afr-self-heal-common.c:887:afr_mark_sources] 0-gv-ocdata-replicate-0:
Number of sources: 0
[2013-12-05 18:33:50.837002] D
[afr-self-heal-data.c:825:afr_lookup_select_read_child_by_txn_type]
0-gv-ocdata-replicate-0: returning read_child: 0
[2013-12-05 18:33:50.837058] D
[afr-common.c:1380:afr_lookup_select_read_child]
0-gv-ocdata-replicate-0: Source selected as 0 for /check.txt
[2013-12-05 18:33:50.837129] D
[afr-common.c:1117:afr_lookup_build_response_params]
0-gv-ocdata-replicate-0: Building lookup response from 0

Other bits of information

root at bbb-1:/var/www/owncloud# uname -a
Linux bbb-1 3.8.13-bone30 #1 SMP Thu Nov 14 02:59:07 UTC 2013 armv7l
GNU/Linux

root at bbb-1:/var/www/owncloud# dpkg -l glusterfs-*
Desired=Unknown/Install/Remove/Purge/Hold
|
Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                        
Version                     Architecture                Description
+++-============================================-===========================-===========================-==============================================================================================
ii  glusterfs-client                            
3.4.1-1                     armhf                       clustered
file-system (client package)
ii  glusterfs-common                            
3.4.1-1                     armhf                       GlusterFS common
libraries and translator modules
ii  glusterfs-server                            
3.4.1-1                     armhf                       clustered
file-system (server package)

mount

bbb-1:gv-ocdata on /var/www/owncloud/data type fuse.glusterfs
(rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)

/etc/fstab

UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /sdhc ext4 defaults 0 0
bbb-1:gv-ocdata /var/www/owncloud/data glusterfs
defaults,_netdev,log-level=DEBUG 0 0

ls -al on the various paths

root at bbb-1:/var/log/glusterfs# ll -d /sdhc/
drwxrwxr-x 7 root root 4096 Nov 28 19:15 /sdhc/

root at bbb-1:/var/log/glusterfs# ll -d /sdhc/gv-ocdata/
drwxrwx--- 5 www-data www-data 4096 Dec  5 00:50 /sdhc/gv-ocdata/

root at bbb-1:/var/log/glusterfs# ll -d /sdhc/gv-ocdata/check.txt
-rw-r--r-- 2 root root 10 Dec  5 00:50 /sdhc/gv-ocdata/check.txt

root at bbb-1:/var/www/owncloud# ll -d /var/www/owncloud/data/
drwxrwx--- 5 www-data www-data 4096 Dec  5 00:50 /var/www/owncloud/data/

root at bbb-1:/var/www/owncloud# ll -d /var/www/owncloud/data/check.txt
-rw-r--r-- 1 root root 10 Dec  5 00:50 /var/www/owncloud/data/check.txt

file & dir attr information:

root at bbb-1:/var/www/owncloud# attr -l /var/www/owncloud/data
Attribute "glusterfs.volume-id" has a 16 byte value for
/var/www/owncloud/data

root at bbb-1:/var/www/owncloud# attr -l /var/www/owncloud/data/check.txt
root at bbb-1:/var/www/owncloud#

root at bbb-1:/var/www/owncloud# attr -l /sdhc/gv-ocdata/
Attribute "glusterfs.volume-id" has a 16 byte value for /sdhc/gv-ocdata/
Attribute "gfid" has a 16 byte value for /sdhc/gv-ocdata/
Attribute "glusterfs.dht" has a 16 byte value for /sdhc/gv-ocdata/
Attribute "afr.gv-ocdata-client-0" has a 12 byte value for /sdhc/gv-ocdata/
Attribute "afr.gv-ocdata-client-1" has a 12 byte value for /sdhc/gv-ocdata/

root at bbb-1:/var/www/owncloud# attr -l /sdhc/gv-ocdata/check.txt
Attribute "gfid" has a 16 byte value for /sdhc/gv-ocdata/check.txt
Attribute "afr.gv-ocdata-client-0" has a 12 byte value for
/sdhc/gv-ocdata/check.txt
Attribute "afr.gv-ocdata-client-1" has a 12 byte value for
/sdhc/gv-ocdata/check.txt
root at bbb-1:/var/www/owncloud#

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131205/a11c93d9/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: simple-php-example.zip
Type: application/zip
Size: 21249 bytes
Desc: not available
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131205/a11c93d9/attachment.zip>