<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p><br>
</p>
<p>Hi Pranith,</p>
<p>Here is the output when I'm trying a touch command that fails
with "Permission denied"<br>
</p>
<p><tt>[root@compute-11-10 ~]# tcpdump -nnSs 0 host 10.1.1.4</tt><tt><br>
</tt><tt>tcpdump: verbose output suppressed, use -v or -vv for
full protocol decode</tt><tt><br>
</tt><tt>listening on eth1, link-type EN10MB (Ethernet), capture
size 65535 bytes</tt><tt><br>
</tt><tt>12:30:46.248293 IP 10.255.255.124.4215828946 >
10.1.1.4.2049: 208 getattr fh 0,0/22</tt><tt><br>
</tt><tt>12:30:46.252509 IP 10.1.1.4.2049 >
10.255.255.124.4215828946: reply ok 240 getattr NON 3 ids 0/3 sz
0</tt><tt><br>
</tt><tt>12:30:46.252596 IP 10.255.255.124.4232606162 >
10.1.1.4.2049: 300 getattr fh 0,0/22</tt><tt><br>
</tt><tt>12:30:46.253308 IP 10.1.1.4.2049 >
10.255.255.124.4232606162: reply ok 52 getattr ERROR: Permission
denied</tt><tt><br>
</tt><tt>12:30:46.253358 IP 10.255.255.124.4249383378 >
10.1.1.4.2049: 216 getattr fh 0,0/22</tt><tt><br>
</tt><tt>12:30:46.260347 IP 10.1.1.4.2049 >
10.255.255.124.4249383378: reply ok 52 getattr ERROR: No such
file or directory</tt><tt><br>
</tt><tt>12:30:46.300306 IP 10.255.255.124.931 > 10.1.1.4.2049:
Flags [.], ack 1979284005, win 501, options [nop,nop,TS val
490628016 ecr 75449144], length 0</tt><tt><br>
</tt><tt>^C</tt><tt><br>
</tt><tt>7 packets captured</tt><tt><br>
</tt><tt>7 packets received by filter</tt><tt><br>
</tt><tt>0 packets dropped by kernel</tt><tt><br>
</tt><br>
</p>
<br>
<div class="moz-cite-prefix">On 09/01/2016 03:31 AM, Pranith Kumar
Karampuri wrote:<br>
</div>
<blockquote
cite="mid:CAOgeEnbv1+Am4tMfnUqXC5Nj-iZf0D-e9KkGS9WzcByqQe9x0A@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div dir="ltr">hi Pat,<br>
I think the other thing we should probably look for would
be to see the tcp dump of what uid/gid parameters are sent over
network when this command is executed.<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Sep 1, 2016 at 7:14 AM, Pat
Haley <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:phaley@mit.edu" target="_blank">phaley@mit.edu</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>--<span
class=""><br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>hi Pat,<br>
</div>
Are you seeing this issue only after
migration or even before? May be we should look at
the gid numbers on the disk and the ones that are
coming from client for the given user to see if
they match or not?<br>
</div>
</div>
</blockquote>
</span>
------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>-------<br>
This issue was not being seen before the migration. We
have copied the /etc/passwd and /etc/group files from the
front-end machine (the client) to the data server, so they
all match<br>
------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>-------<span
class=""><br>
<blockquote type="cite">
<div dir="ltr">Could you give stat output of the
directory in question from both the brick and the
nfs client<br>
<div><br>
</div>
</div>
</blockquote>
</span>
------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>--------<br>
From the server for gluster:<br>
<tt>[root@mseas-data2 ~]# stat /gdata/projects/nsf_alpha</tt><tt><br>
</tt><tt> File: `/gdata/projects/nsf_alpha'</tt><tt><br>
</tt><tt> Size: 4096 Blocks: 8 IO
Block: 131072 directory</tt><tt><br>
</tt><tt>Device: 13h/19d Inode: 13094773206281819436
Links: 13</tt><tt><br>
</tt><tt>Access: (2775/drwxrwsr-x) Uid: ( 0/
root) Gid: ( 598/nsf_alpha)</tt><tt><br>
</tt><tt>Access: 2016-08-31 19:08:59.735990904 -0400</tt><tt><br>
</tt><tt>Modify: 2016-08-31 16:37:09.048997167 -0400</tt><tt><br>
</tt><tt>Change: 2016-08-31 16:37:41.315997148 -0400</tt><tt><br>
</tt><tt><br>
From the server for first underlying brick<br>
[root@mseas-data2 ~]# stat /mnt/brick1/projects/nsf_<wbr>alpha/<br>
File: `/mnt/brick1/projects/nsf_<wbr>alpha/'<br>
Size: 4096 Blocks: 8 IO Block:
4096 directory<br>
Device: 800h/2048d Inode: 185630 Links: 13<br>
Access: (2775/drwxrwsr-x) Uid: ( 0/ root) Gid:
( 598/nsf_alpha)<br>
Access: 2016-08-31 19:08:59.669990907 -0400<br>
Modify: 2016-08-31 16:37:09.048997167 -0400<br>
Change: 2016-08-31 16:37:41.315997148 -0400<br>
</tt><br>
<tt>From the server for second underlying brick<br>
[root@mseas-data2 ~]# stat /mnt/brick2/projects/nsf_<wbr>alpha/<br>
File: `/mnt/brick2/projects/nsf_<wbr>alpha/'<br>
Size: 4096 Blocks: 8 IO Block:
4096 directory<br>
Device: 810h/2064d Inode: 24085468 Links: 13<br>
Access: (2775/drwxrwsr-x) Uid: ( 0/ root) Gid:
( 598/nsf_alpha)<br>
Access: 2016-08-31 19:08:59.735990904 -0400<br>
Modify: 2016-08-03 14:01:52.000000000 -0400<br>
Change: 2016-08-31 16:37:41.315997148 -0400<br>
<br>
From the client<br>
[root@mseas FixOwn]# stat /gdata/projects/nsf_alpha<br>
File: `/gdata/projects/nsf_alpha'<br>
Size: 4096 Blocks: 8 IO Block:
1048576 directory<br>
Device: 23h/35d Inode: 13094773206281819436 Links:
13<br>
Access: (2775/drwxrwsr-x) Uid: ( 0/ root) Gid:
( 598/nsf_alpha)<br>
Access: 2016-08-31 19:08:59.735990904 -0400<br>
Modify: 2016-08-31 16:37:09.048997167 -0400<br>
Change: 2016-08-31 16:37:41.315997148 -0400<br>
<br>
</tt><tt> </tt>
<blockquote type="cite"> </blockquote>
------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>------<span
class=""><br>
<blockquote type="cite"> </blockquote>
<blockquote type="cite">
<div dir="ltr">
<div>Could you also let us know version of gluster
you are using</div>
</div>
</blockquote>
</span>
<blockquote type="cite">
<div dir="ltr">
<div>------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>-------<br>
</div>
</div>
</blockquote>
<br>
<blockquote type="cite">
<div dir="ltr">
<div> <tt>[root@mseas-data2 ~]# gluster --version</tt><tt><br>
</tt><tt>glusterfs 3.7.11 built on Apr 27 2016
14:09:22</tt><tt><br>
</tt><br>
</div>
</div>
</blockquote>
<tt>[root@mseas-data2 ~]# gluster volume info</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Volume Name: data-volume</tt><tt><br>
</tt><tt>Type: Distribute</tt><tt><br>
</tt><tt>Volume ID: c162161e-2a2d-4dac-b015-<wbr>f31fd89ceb18</tt><tt><br>
</tt><tt>Status: Started</tt><tt><br>
</tt><tt>Number of Bricks: 2</tt><tt><br>
</tt><tt>Transport-type: tcp</tt><tt><br>
</tt><tt>Bricks:</tt><tt><br>
</tt><tt>Brick1: mseas-data2:/mnt/brick1</tt><tt><br>
</tt><tt>Brick2: mseas-data2:/mnt/brick2</tt><tt><br>
</tt><tt>Options Reconfigured:</tt><tt><br>
</tt><tt>performance.readdir-ahead: on</tt><tt><br>
</tt><tt>nfs.disable: on</tt><tt><br>
</tt><tt>nfs.export-volumes: off</tt><tt><br>
</tt><br>
<tt>[root@mseas-data2 ~]# gluster volume status </tt><tt><br>
</tt><tt>Status of volume: data-volume</tt><tt><br>
</tt><tt>Gluster process <wbr>
TCP Port RDMA Port Online Pid</tt><tt><br>
</tt><tt>------------------------------<wbr>------------------------------<wbr>------------------</tt><tt><br>
</tt><tt>Brick mseas-data2:/mnt/brick1 <wbr>
49154 0 Y 5005 </tt><tt><br>
</tt><tt>Brick mseas-data2:/mnt/brick2 <wbr>
49155 0 Y 5010 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Task Status of Volume data-volume</tt><tt><br>
</tt><tt>------------------------------<wbr>------------------------------<wbr>------------------</tt><tt><br>
</tt><tt>Task : Rebalance </tt><tt><br>
</tt><tt>ID : 892d9e3a-b38c-4971-b96a-<wbr>8e4a496685ba</tt><tt><br>
</tt><tt>Status : completed <br>
<br>
</tt><tt><br>
</tt><tt>[root@mseas-data2 ~]# gluster peer status</tt><tt><br>
</tt><tt>Number of Peers: 0</tt><tt><br>
</tt><br>
<blockquote type="cite">
<div class="gmail_extra"><br>
<div class="gmail_quote">------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>-------
<div>
<div class="h5"><br>
On Thu, Sep 1, 2016 at 2:46 AM, Pat Haley <span
dir="ltr"><<a moz-do-not-send="true"
href="mailto:phaley@mit.edu" target="_blank">phaley@mit.edu</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p><br>
</p>
<p>Hi,</p>
<p>Another piece of data. There are 2
distinct volumes on the file server</p>
<ol>
<li>a straight nfs partition</li>
<li>a gluster volume (served over nfs)<br>
</li>
</ol>
The straight nfs partition does respect the
group write permissions, while the gluster
volume does not. Any suggestions on how to
debug this or what additional information
would be helpful would be greatly
appreciated<br>
<br>
Thanks<span><br>
<br>
<div>On 08/30/2016 06:01 PM, Pat Haley
wrote:<br>
</div>
<blockquote type="cite"> <br>
Hi <br>
<br>
We have just migrated our data to a new
file server (more space, old server was
showing its age). We have a volume for
collaborative use, based on group
membership. In our new server, the
group write permissions are not being
respected (e.g. the owner of a
directory can still write to that
directory but any other member of the
associated group cannot, even though the
directory clearly has group write
permissions set). This is occurring
regardless of how many groups the user
is a member of (i.e. users that are
members of fewer then 16 groups are
still affected). <br>
<br>
the relevant fstab line from the server
looks like <br>
localhost:/data-volume /gdata
glusterfs defaults 0 0 <br>
<br>
and for a client: <br>
mseas-data2:/gdata /gdata
nfs defaults 0 0 <br>
<br>
Any help would be greatly appreciated. <br>
<br>
Thanks <br>
<br>
</blockquote>
<br>
<pre cols="72">--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-
Pat Haley Email: <a moz-do-not-send="true" href="mailto:phaley@mit.edu" target="_blank">phaley@mit.edu</a>
Center for Ocean Engineering Phone: <a moz-do-not-send="true" href="tel:%28617%29%20253-6824" value="+16172536824" target="_blank">(617) 253-6824</a>
Dept. of Mechanical Engineering Fax: <a moz-do-not-send="true" href="tel:%28617%29%20253-8125" value="+16172538125" target="_blank">(617) 253-8125</a>
MIT, Room 5-213 <a moz-do-not-send="true" href="http://web.mit.edu/phaley/www/" target="_blank">http://web.mit.edu/phaley/www/</a>
77 Massachusetts Avenue
Cambridge, MA 02139-4301
</pre>
</span></div>
______________________________<wbr>_________________
Gluster-users mailing list
<a moz-do-not-send="true" href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a moz-do-not-send="true" href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/mailman<wbr>/listinfo/gluster-users</a>
</blockquote></div></div></div><span class="HOEnZb"><font color="#888888">
--
<div data-smartmail="gmail_signature"><div dir="ltr">Pranith
</div></div>
</font></span></div>
</blockquote><div><div class="h5">
<pre cols="72">--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-
Pat Haley Email: <a moz-do-not-send="true" href="mailto:phaley@mit.edu" target="_blank">phaley@mit.edu</a>
Center for Ocean Engineering Phone: <a moz-do-not-send="true" href="tel:%28617%29%20253-6824" value="+16172536824" target="_blank">(617) 253-6824</a>
Dept. of Mechanical Engineering Fax: <a moz-do-not-send="true" href="tel:%28617%29%20253-8125" value="+16172538125" target="_blank">(617) 253-8125</a>
MIT, Room 5-213 <a moz-do-not-send="true" href="http://web.mit.edu/phaley/www/" target="_blank">http://web.mit.edu/phaley/www/</a>
77 Massachusetts Avenue
Cambridge, MA 02139-4301
</pre></div></div></div>
______________________________<wbr>_________________
Gluster-users mailing list
<a moz-do-not-send="true" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a moz-do-not-send="true" href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/<wbr>mailman/listinfo/gluster-users</a>
</blockquote></div>
--
<div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Pranith
</div></div>
</div>
</blockquote>
<pre class="moz-signature" cols="72">--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Pat Haley Email: <a class="moz-txt-link-abbreviated" href="mailto:phaley@mit.edu">phaley@mit.edu</a>
Center for Ocean Engineering Phone: (617) 253-6824
Dept. of Mechanical Engineering Fax: (617) 253-8125
MIT, Room 5-213 <a class="moz-txt-link-freetext" href="http://web.mit.edu/phaley/www/">http://web.mit.edu/phaley/www/</a>
77 Massachusetts Avenue
Cambridge, MA 02139-4301
</pre></body></html>