<div dir="ltr"><div>As I understand this error appear because BWA opening hg19.fa.pac twice. One handler for writing out and one for reading.</div><div>And second thread is trying to read some data which is not preset on disk yet.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 9, 2015 at 11:49 AM, Adam <span dir="ltr"><<a href="mailto:adamnkraut@gmail.com" target="_blank">adamnkraut@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Jeff/all,<div><br></div><div>I took the recommendation of disabled the stripes. Now I just have encryption (at rest) and SSL enabled. The test I am running is a bwa indexing. Basic dd read/writes work fine and I don't see any errors in the gluster logs. Then when I try the bwa index I see the following:</div><div><br></div><div><div>/shared/perftest/bwa/bwa index -a bwtsw hg19.fa</div><div>[bwa_index] Pack FASTA... 26.29 sec</div><div>[bwa_index] Construct BWT for the packed sequence...</div><div>BWTIncConstructFromPacked() : Can't read from hg19.fa.pac : Unexpected end of file</div></div><div><br></div><div>These are my current volume settings:</div><div>glusterfs 3.6.2 built on Jan 22 2015 12:58:11<br></div><div><div>Volume Name: data</div><div>Type: Distribute</div><div>Volume ID: 55d1c37b-bfba-47d8-8467-0b28b0e04aa2</div><div>Status: Started</div><div>Number of Bricks: 3</div><div>Transport-type: tcp</div><div>Bricks:</div><div>Brick1: ip-10-9-0-32.ec2.internal:/export/brick</div><div>Brick2: ip-10-9-0-141.ec2.internal:/export/brick</div><div>Brick3: ip-10-9-0-142.ec2.internal:/export/brick</div><div>Options Reconfigured:</div><div>performance.open-behind: off</div><div>performance.write-behind: off</div><div>performance.quick-read: off</div><div>encryption.master-key: /root/keystore/master.key</div><div>features.encryption: on</div><div>auth.ssl-allow: *</div><div>server.ssl: on</div><div>client.ssl: on</div><div>auth.allow: *</div></div><div><br></div><div>There are no messages in the logs during the job. However there are some errors from previous lines:</div><div><div>[2015-03-09 15:21:47.868160] E [socket.c:2481:socket_poller] 0-data-client-0: poll error on socket</div><div>[2015-03-09 15:21:47.868184] E [socket.c:2481:socket_poller] 0-data-client-1: poll error on socket</div><div>[2015-03-09 15:21:47.868288] E [socket.c:2481:socket_poller] 0-data-client-2: poll error on socket</div></div><div><br></div><div>If I take out the encryption and leave just SSL mode on the bwa index is successful. SSL may be good enough for our needs but I would like to know if we have the option of at rest encryption. Any ideas? Many thanks in advance!</div><div><br></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Feb 24, 2015 at 12:33 PM, Jeff Darcy <span dir="ltr"><<a href="mailto:jdarcy@redhat.com" target="_blank">jdarcy@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span>> SSL certs are self-signed and generated on all servers. Combined into a<br>
> <a href="http://glusterfs.ca" target="_blank">glusterfs.ca</a> in /etc/ssl. By itself the SSL is working well.<br>
<br>
</span>Glad to hear it. ;)<br>
<span><br>
> If I run dd or any i/o operations I see a flurry of these messages in the<br>
> logs.<br>
><br>
> [2015-02-24 16:58:51.144099] W [stripe.c:5288:stripe_internal_getxattr_cbk]<br>
> (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn+0x1e0)[0x3fd0620550]<br>
> (--><br>
> /usr/lib64/glusterfs/3.6.2/xlator/cluster/stripe.so(stripe_internal_getxattr_cbk+0x36a)[0x7f6a152a12ba]<br>
> (--><br>
> /usr/lib64/glusterfs/3.6.2/xlator/protocol/client.so(client3_3_fgetxattr_cbk+0x174)[0x7f6a154db284]<br>
> (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_handle_reply+0xa5)[0x3fd0e0ea75] (--><br>
> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify+0x142)[0x3fd0e0ff02] )))))<br>
> 0-data-stripe-3: invalid argument: frame->local<br>
<br>
<br>
</span>Have you tried encryption (at rest) without striping, or vice versa? I<br>
suspect some kind of bad interaction between the two, but before we go<br>
down that path it would be nice to make sure they're working separately.<br>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>