<div dir="ltr">Hi gluster folks,<div><br></div><div>I&#39;m looking for some configuration or debugging advice for a distributed-replicated volume that uses SSL and at rest encryption.</div><div><br></div><div>SSL certs are self-signed and generated on all servers. Combined into a <a href="http://glusterfs.ca">glusterfs.ca</a> in /etc/ssl. By itself the SSL is working well.</div><div><br></div><div>I&#39;ve also turned on the disk encryption feature. Master key was generated with &#39;openssl rand -hex 32&#39; as per the docs and copied to all gluster servers.</div><div><br></div><div><div>Status of volume: data</div><div>Gluster process<span class="" style="white-space:pre">                                                </span>Port<span class="" style="white-space:pre">        </span>Online<span class="" style="white-space:pre">        </span>Pid</div><div>------------------------------------------------------------------------------</div><div>Brick ip-10-9-0-62.ec2.internal:/export/brick<span class="" style="white-space:pre">                </span>49152<span class="" style="white-space:pre">        </span>Y<span class="" style="white-space:pre">        </span>13393</div><div>Brick ip-10-9-0-101.ec2.internal:/export/brick<span class="" style="white-space:pre">                </span>49152<span class="" style="white-space:pre">        </span>Y<span class="" style="white-space:pre">        </span>8412</div><div>Brick ip-10-9-0-103.ec2.internal:/export/brick<span class="" style="white-space:pre">                </span>49152<span class="" style="white-space:pre">        </span>Y<span class="" style="white-space:pre">        </span>10125</div><div>Brick ip-10-9-0-102.ec2.internal:/export/brick<span class="" style="white-space:pre">                </span>49152<span class="" style="white-space:pre">        </span>Y<span class="" style="white-space:pre">        </span>8266</div><div>Brick ip-10-9-0-100.ec2.internal:/export/brick<span class="" style="white-space:pre">                </span>49152<span class="" style="white-space:pre">        </span>Y<span class="" style="white-space:pre">        </span>8263</div><div>Brick ip-10-9-0-105.ec2.internal:/export/brick<span class="" style="white-space:pre">                </span>49152<span class="" style="white-space:pre">        </span>Y<span class="" style="white-space:pre">        </span>8277</div><div>Brick ip-10-9-0-104.ec2.internal:/export/brick<span class="" style="white-space:pre">                </span>49152<span class="" style="white-space:pre">        </span>Y<span class="" style="white-space:pre">        </span>8261</div><div>Brick ip-10-9-0-106.ec2.internal:/export/brick<span class="" style="white-space:pre">                </span>49152<span class="" style="white-space:pre">        </span>Y<span class="" style="white-space:pre">        </span>8272</div><div><br></div><div>Task Status of Volume data</div><div>------------------------------------------------------------------------------</div><div>There are no active volume tasks</div></div><div><br></div><div><div>Volume Name: data</div><div>Type: Distributed-Stripe</div><div>Volume ID: afad6283-5bee-42c1-b9e5-c3ed64e04aae</div><div>Status: Started</div><div>Number of Bricks: 4 x 2 = 8</div><div>Transport-type: tcp</div><div>Bricks:</div><div>Brick1: ip-10-9-0-62.ec2.internal:/export/brick</div><div>Brick2: ip-10-9-0-101.ec2.internal:/export/brick</div><div>Brick3: ip-10-9-0-103.ec2.internal:/export/brick</div><div>Brick4: ip-10-9-0-102.ec2.internal:/export/brick</div><div>Brick5: ip-10-9-0-100.ec2.internal:/export/brick</div><div>Brick6: ip-10-9-0-105.ec2.internal:/export/brick</div><div>Brick7: ip-10-9-0-104.ec2.internal:/export/brick</div><div>Brick8: ip-10-9-0-106.ec2.internal:/export/brick</div><div>Options Reconfigured:</div><div>server.allow-insecure: on</div><div>nfs.ports-insecure: on</div><div>auth.allow: *</div><div>client.ssl: on</div><div>server.ssl: on</div><div>auth.ssl-allow: *</div><div>features.encryption: on</div><div>encryption.master-key: /root/keystore/master.key</div><div>performance.quick-read: off</div><div>performance.write-behind: off</div><div>performance.open-behind: off</div><div>nfs.disable: on</div></div><div><br></div><div>If I run dd or any i/o operations I see a flurry of these messages in the logs.</div><div><br></div><div>[2015-02-24 16:58:51.144099] W [stripe.c:5288:stripe_internal_getxattr_cbk] (--&gt; /usr/lib64/libglusterfs.so.0(_gf_log_callingfn+0x1e0)[0x3fd0620550] (--&gt; /usr/lib64/glusterfs/3.6.2/xlator/cluster/stripe.so(stripe_internal_getxattr_cbk+0x36a)[0x7f6a152a12ba] (--&gt; /usr/lib64/glusterfs/3.6.2/xlator/protocol/client.so(client3_3_fgetxattr_cbk+0x174)[0x7f6a154db284] (--&gt; /usr/lib64/libgfrpc.so.0(rpc_clnt_handle_reply+0xa5)[0x3fd0e0ea75] (--&gt; /usr/lib64/libgfrpc.so.0(rpc_clnt_notify+0x142)[0x3fd0e0ff02] ))))) 0-data-stripe-3: invalid argument: frame-&gt;local<br></div><div><br></div><div>Thanks in advance for any tips/suggestions!</div><div><br></div><div>-Adam</div></div>