<div dir="ltr">Hi Humble!<div><br></div><div>Thanks for the reply. The docs do not mention anything related to 3.6->3.7 upgrade that applies to my case. </div><div><br></div><div>I could resolve the issue in the meantime by doing the steps mentioned in the 3.7.1 release notes (<a href="https://gluster.readthedocs.org/en/latest/release-notes/3.7.1/">https://gluster.readthedocs.org/en/latest/release-notes/3.7.1/</a>). </div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div style="font-family:arial;font-size:small">Thanks,</div><div style="font-family:arial;font-size:small"><br></div><div style="font-family:arial;font-size:small">Andreas</div><div style="font-family:arial;font-size:small"><br></div></div></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On Thu, Aug 27, 2015 at 3:22 PM, Humble Devassy Chirammal <span dir="ltr"><<a href="mailto:humble.devassy@gmail.com" target="_blank">humble.devassy@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>Hi Andreas, <br><span class=""><br>><br>Is it even possible to perform a rolling upgrade? <div>><br></div><br><br></span>The GlusterFS upgrade process is documented @ <a href="https://gluster.readthedocs.org/en/latest/Upgrade-Guide/README/" target="_blank">https://gluster.readthedocs.org/en/latest/Upgrade-Guide/README/</a> <br></div><div><br><br></div></div><div class="gmail_extra"><br clear="all"><div><div><div dir="ltr"><div>--Humble<br></div><br></div></div></div>
<br><div class="gmail_quote"><div><div class="h5">On Thu, Aug 27, 2015 at 4:57 PM, Andreas Mather <span dir="ltr"><<a href="mailto:andreas@allaboutapps.at" target="_blank">andreas@allaboutapps.at</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><div class="h5"><div dir="ltr">Hi All!<div><br></div><div>I wanted to do a rolling upgrade of gluster from 3.6.3 to 3.7.3, but after the upgrade, the updated node won't connect.</div><div><br></div><div>The cluster has 4 nodes (vhost[1-4]) and 4 volumes (vol[1-4]) with 2 replicas each:<div>vol1: vhost1/brick1, vhost2/brick2</div><div>vol2: vhost2/brick1, vhost1/brick2</div><div>vol3: vhost3/brick1, vhost4/brick2</div><div>vol4: vhost4/brick1, vhost3/brick2</div></div><div><br></div><div>I'm trying to start the upgrade on vhost4. After restarting glusterd, peer status shows all other peers as disconnected, the log has repeated entries like this:</div><div><br></div><div><div>[2015-08-27 10:59:56.982254] E [MSGID: 106167] [glusterd-handshake.c:2078:__glusterd_peer_dump_version_cbk] 0-management: Error through RPC layer, retry again later</div><div>[2015-08-27 10:59:56.982335] E [rpc-clnt.c:362:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x196)[0x7f1a7a9579e6] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f1a7a7229be] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f1a7a722ace] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x9c)[0x7f1a7a72447c] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x48)[0x7f1a7a724c38] ))))) 0-management: forced unwinding frame type(GF-DUMP) op(NULL(2)) called at 2015-08-27 10:59:56.981550 (xid=0x2)</div><div>[2015-08-27 10:59:56.982346] W [rpc-clnt-ping.c:204:rpc_clnt_ping_cbk] 0-management: socket disconnected</div><div>[2015-08-27 10:59:56.982359] I [MSGID: 106004] [glusterd-handler.c:5051:__glusterd_peer_rpc_notify] 0-management: Peer <vhost3-int> (<72e2078d-1ed9-4cdd-aad2-c86e418746d1>), in state <Peer in Cluster>, has disconnected from glusterd.</div><div>[2015-08-27 10:59:56.982491] W [glusterd-locks.c:677:glusterd_mgmt_v3_unlock] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x196)[0x7f1a7a9579e6] (--> /usr/lib64/glusterfs/3.7.3/xlator/mgmt/glusterd.so(glusterd_mgmt_v3_unlock+0x541)[0x7f1a6f55ee91] (--> /usr/lib64/glusterfs/3.7.3/xlator/mgmt/glusterd.so(__glusterd_peer_rpc_notify+0x162)[0x7f1a6f4c6972] (--> /usr/lib64/glusterfs/3.7.3/xlator/mgmt/glusterd.so(glusterd_big_locked_notify+0x4c)[0x7f1a6f4bc90c] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x90)[0x7f1a7a724c80] ))))) 0-management: Lock for vol vol1 not held</div><div>[2015-08-27 10:59:56.982504] W [MSGID: 106118] [glusterd-handler.c:5073:__glusterd_peer_rpc_notify] 0-management: Lock not released for vol1</div><div>[2015-08-27 10:59:56.982608] W [glusterd-locks.c:677:glusterd_mgmt_v3_unlock] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x196)[0x7f1a7a9579e6] (--> /usr/lib64/glusterfs/3.7.3/xlator/mgmt/glusterd.so(glusterd_mgmt_v3_unlock+0x541)[0x7f1a6f55ee91] (--> /usr/lib64/glusterfs/3.7.3/xlator/mgmt/glusterd.so(__glusterd_peer_rpc_notify+0x162)[0x7f1a6f4c6972] (--> /usr/lib64/glusterfs/3.7.3/xlator/mgmt/glusterd.so(glusterd_big_locked_notify+0x4c)[0x7f1a6f4bc90c] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x90)[0x7f1a7a724c80] ))))) 0-management: Lock for vol vol2 not held</div><div>[2015-08-27 10:59:56.982618] W [MSGID: 106118] [glusterd-handler.c:5073:__glusterd_peer_rpc_notify] 0-management: Lock not released for vol2</div><div>[2015-08-27 10:59:56.982728] W [glusterd-locks.c:677:glusterd_mgmt_v3_unlock] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x196)[0x7f1a7a9579e6] (--> /usr/lib64/glusterfs/3.7.3/xlator/mgmt/glusterd.so(glusterd_mgmt_v3_unlock+0x541)[0x7f1a6f55ee91] (--> /usr/lib64/glusterfs/3.7.3/xlator/mgmt/glusterd.so(__glusterd_peer_rpc_notify+0x162)[0x7f1a6f4c6972] (--> /usr/lib64/glusterfs/3.7.3/xlator/mgmt/glusterd.so(glusterd_big_locked_notify+0x4c)[0x7f1a6f4bc90c] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x90)[0x7f1a7a724c80] ))))) 0-management: Lock for vol vol3 not held</div><div>[2015-08-27 10:59:56.982739] W [MSGID: 106118] [glusterd-handler.c:5073:__glusterd_peer_rpc_notify] 0-management: Lock not released for vol3</div><div>[2015-08-27 10:59:56.982844] W [glusterd-locks.c:677:glusterd_mgmt_v3_unlock] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x196)[0x7f1a7a9579e6] (--> /usr/lib64/glusterfs/3.7.3/xlator/mgmt/glusterd.so(glusterd_mgmt_v3_unlock+0x541)[0x7f1a6f55ee91] (--> /usr/lib64/glusterfs/3.7.3/xlator/mgmt/glusterd.so(__glusterd_peer_rpc_notify+0x162)[0x7f1a6f4c6972] (--> /usr/lib64/glusterfs/3.7.3/xlator/mgmt/glusterd.so(glusterd_big_locked_notify+0x4c)[0x7f1a6f4bc90c] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x90)[0x7f1a7a724c80] ))))) 0-management: Lock for vol vol4 not held</div><div>[2015-08-27 10:59:56.982858] W [MSGID: 106118] [glusterd-handler.c:5073:__glusterd_peer_rpc_notify] 0-management: Lock not released for vol4</div><div>[2015-08-27 10:59:56.982881] W [socket.c:642:__socket_rwv] 0-management: readv on <a href="http://192.168.92.2:24007" target="_blank">192.168.92.2:24007</a> failed (Connection reset by peer)</div><div>[2015-08-27 10:59:56.982974] E [rpc-clnt.c:362:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x196)[0x7f1a7a9579e6] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f1a7a7229be] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f1a7a722ace] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x9c)[0x7f1a7a72447c] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x48)[0x7f1a7a724c38] ))))) 0-management: forced unwinding frame type(GLUSTERD-DUMP) op(DUMP(1)) called at 2015-08-27 10:59:56.981566 (xid=0x1)</div></div><div><br></div><div><br></div><div>Any ideas? Is it even possible to perform a rolling upgrade? </div><div><br></div><div>Thanks for any help!</div><span><font color="#888888"><div><br></div><div>Andreas </div></font></span></div>
<br></div></div>_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br></blockquote></div><br></div>
</blockquote></div><br></div></div>