<div dir="ltr"><div class="gmail_extra"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><br></div></div></div></div><div class="gmail_quote">On Tue, Jul 12, 2016 at 4:06 AM, Frank Rothenstein <span dir="ltr"><<a href="mailto:f.rothenstein@bodden-kliniken.de" target="_blank">f.rothenstein@bodden-kliniken.de</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Hi all,<br>
<br>
I am stuck with a strange problem using ovirt/gluster on ZFSonLinux.<br>
<br>
I detached a data-volume form my Cluster an reimported it to change<br>
from POSIX-type/glusterfs to GlusterFS-Storage. When I try to attach it<br>
to the Cluster I always get a Sanlock error. I checked the logs and what I found were some "operation not permitted"<br>
errors in the brick logs:<br>
<br>
[2016-07-08 09:19:24.542046] E [MSGID: 113040] [posix-<br>
helpers.c:1684:__posix_fd_ctx_get] 0-datadom-posix: Failed to get<br>
anonymous real_path:<br>
/tank2/datadom/brick1/br/.glusterfs/c1/43/c1433bab-cd72-4395-ac62-<br>
b278670c333a _fd = -1 [No such file or directory]<br>
[2016-07-08 09:19:24.542103] W [MSGID: 113006]<br>
[posix.c:3010:posix_readv] 0-datadom-posix: pfd is NULL from<br>
fd=0x7f15f0003b48 [Operation not permitted]<br>
[2016-07-08 09:19:24.542142] E [MSGID: 115068] [server-rpc-<br>
fops.c:1486:server_readv_cbk] 0-datadom-server: 4836: READV -2<br>
(c1433bab-cd72-4395-ac62-b278670c333a) ==> (Operation not permitted)<br>
[Operation not permitted]<br>
<br>
Anyone ever expierienced such problem?<br>
<br>
In fact I am able to create/change/delete files in the brick path<br>
itself and the mount points (NFS/Fuse).<br>
Ovirt itself seems to test the volumes with /usr/bin/dd, but uses<br>
iflag=direct. Using this parameter I'm also unable to create any file -<br>
is there any gluster-option that changed recently? Or is it ZFS, that<br>
introduced something that blocks now?<br></blockquote><div><br></div><div>What version of Gluster are you on?</div><div><br></div><div>My errors aren't quite the same as yours, but I have a similar setup</div><div><br></div><div>Ovirt 3.6.6/Gluster3.7.11 on ZFSonLinux</div><div><br></div><div>Updating from 3.7.11 -> 12 or 13 this last weekend I had these issues pop up for me which while not exactly the same did result in <span style="font-size:12.8px">[Operation not permitted]</span></div><div><br></div><div style="font-size:12.8px">[2016-07-09 15:27:46.935694] I [fuse-bridge.c:4083:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 kernel 7.22</div><div style="font-size:12.8px">[2016-07-09 15:27:49.555466] W [MSGID: 114031] [client-rpc-fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-1: remote operation failed [Operation not permitted]</div><div style="font-size:12.8px">[2016-07-09 15:27:49.556574] W [MSGID: 114031] [client-rpc-fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-0: remote operation failed [Operation not permitted]</div><div style="font-size:12.8px">[2016-07-09 15:27:49.556659] W [fuse-bridge.c:2227:fuse_readv_cbk] 0-glusterfs-fuse: 80: READ => -1 gfid=deb61291-5176-4b81-8315-3f1cf8e3534d fd=0x7f5224002f68 (Operation not permitted)</div><div style="font-size:12.8px">[2016-07-09 15:27:59.612477] W [MSGID: 114031] [client-rpc-fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-1: remote operation failed [Operation not permitted]</div><div style="font-size:12.8px">[2016-07-09 15:27:59.613700] W [MSGID: 114031] [client-rpc-fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-0: remote operation failed [Operation not permitted]</div><div><span style="font-size:12.8px">[2016-07-09 15:27:59.613781] W [fuse-bridge.c:2227:fuse_</span><span style="font-size:12.8px">readv_cbk] 0-glusterfs-fuse: 168: READ => -1 gfid=deb61291-5176-4b81-8315-</span><span style="font-size:12.8px">3f1cf8e3534d fd=0x7f5224002f68 (Operation not permitted)</span> </div><div><br></div><div>Another gluster user suggested that in 3.7.12/13 aio support was disabled and I may need to change disk caching method. So far I have just stayed at 3.7.11 until I have time to get a better test setup.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<br>
Thanks in advance, Frank<br>
<br>
<br>
<br>
<br>
<br>
______________________________________________________________________________<br>
BODDEN-KLINIKEN Ribnitz-Damgarten GmbH<br>
Sandhufe 2<br>
18311 Ribnitz-Damgarten<br>
<br>
Telefon: 03821-700-0<br>
Fax: 03821-700-240<br>
<br>
E-Mail: <a href="mailto:info@bodden-kliniken.de">info@bodden-kliniken.de</a> Internet: <a href="http://www.bodden-kliniken.de" rel="noreferrer" target="_blank">http://www.bodden-kliniken.de</a><br>
<br>
Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, Steuer-Nr.: 079/133/40188<br>
Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer: Dr. Falko Milski<br>
<br>
Der Inhalt dieser E-Mail ist ausschließlich für den bezeichneten Adressaten bestimmt. Wenn Sie nicht der vorge-<br>
sehene Adressat dieser E-Mail oder dessen Vertreter sein sollten, beachten Sie bitte, dass jede Form der Veröf-<br>
fentlichung, Vervielfältigung oder Weitergabe des Inhalts dieser E-Mail unzulässig ist. Wir bitten Sie, sofort den<br>
Absender zu informieren und die E-Mail zu löschen.<br>
<br>
<br>
Bodden-Kliniken Ribnitz-Damgarten GmbH 2016<br>
*** Virenfrei durch Kerio Mail Server und Sophos Antivirus ***<br>
<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a></blockquote></div><br></div></div>