<p dir="ltr">Rebalance is broken in 3.7.1. It will be fixed in 3.7.2. Sorry for the inconvenience.</p>
<p dir="ltr">Regards,<br>
Atin</p>
<p dir="ltr">Sent from Samsung Galaxy S4</p>
<div class="gmail_quote">On 9 Jun 2015 18:34, "Jonhnny Weslley" <<a href="mailto:jw@jonhnnyweslley.net">jw@jonhnnyweslley.net</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi guys,<div><br></div><div>I trying to create a pool of 4 nodes using centos7 and gluster 3.7 in a vagrant-based environment for test. First, I create and start a replicated volume using only 2 nodes (replication 2). After, I mount the volume using fuse and copy some files. Everything works fine. <br clear="all"><div><br></div><div>Then, I try to expand the volume previously created using the command:</div><div><br></div><div>sudo gluster volume add-brick jged 10.10.50.73:/home/vagrant/brick 10.10.50.74:/home/vagrant/brick force<br></div><div><br></div><div>And works again:</div><div><br></div><div>sudo gluster volume info<br></div><div><br></div><div><div>Volume Name: jged</div><div>Type: Distributed-Replicate</div><div>Volume ID: 862ab9b7-4753-4682-ba44-cbe481b1b7df</div><div>Status: Started</div><div>Number of Bricks: 2 x 2 = 4</div><div>Transport-type: tcp</div><div>Bricks:</div><div>Brick1: 10.10.50.71:/home/vagrant/brick</div><div>Brick2: 10.10.50.72:/home/vagrant/brick</div><div>Brick3: 10.10.50.73:/home/vagrant/brick</div><div>Brick4: 10.10.50.74:/home/vagrant/brick</div><div>Options Reconfigured:</div><div>performance.readdir-ahead: on</div></div><div><br></div><div><br></div><div>But when I try to rebalance the volume (sudo gluster volume rebalance jged start), the gluster's proccess in the node where the command was executed dies and dont start again after running 'systemctl start glusterd'. I look the log file (/var/log/glusterfs/etc-glusterfs-glusterd.vol.log) but I cant figure out what is wrong! :(</div><div><br></div><div>Follow the tail of the log file:</div><div><br></div><div><div>[2015-06-09 12:30:56.197802] I [MSGID: 100030] [glusterfsd.c:2294:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.7.1 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid)</div><div>[2015-06-09 12:30:56.207596] I [glusterd.c:1282:init] 0-management: Maximum allowed open file descriptors set to 65536</div><div>[2015-06-09 12:30:56.207653] I [glusterd.c:1327:init] 0-management: Using /var/lib/glusterd as working directory</div><div>[2015-06-09 12:30:56.211505] E [rpc-transport.c:291:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/3.7.1/rpc-transport/rdma.so: cannot open shared object file: No such file or directory</div><div>[2015-06-09 12:30:56.211521] W [rpc-transport.c:295:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine</div><div>[2015-06-09 12:30:56.211528] W [rpcsvc.c:1595:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed</div><div>[2015-06-09 12:30:56.211535] E [glusterd.c:1515:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport</div><div>[2015-06-09 12:30:56.213311] I [glusterd.c:413:glusterd_check_gsync_present] 0-glusterd: geo-replication module not installed in the system</div><div>[2015-06-09 12:30:56.213454] I [glusterd-store.c:1986:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30700</div><div>[2015-06-09 12:30:56.213523] I [glusterd.c:154:glusterd_uuid_init] 0-management: retrieved UUID: f264d968-5a14-459b-8f3b-569aa15c3ce2</div><div>[2015-06-09 12:30:56.213568] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600</div><div>[2015-06-09 12:30:56.213675] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600</div><div>[2015-06-09 12:30:56.213801] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600</div><div>[2015-06-09 12:30:56.213896] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600</div><div>[2015-06-09 12:30:56.213979] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600</div><div>[2015-06-09 12:30:56.214094] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600</div><div>[2015-06-09 12:30:56.987649] I [glusterd-handler.c:3387:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0</div><div>[2015-06-09 12:30:56.987711] I [glusterd-handler.c:3387:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0</div><div>[2015-06-09 12:30:56.987755] I [glusterd-handler.c:3387:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0</div><div>[2015-06-09 12:30:56.987801] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600</div><div>[2015-06-09 12:30:56.989874] W [socket.c:923:__socket_keepalive] 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 13, Invalid argument</div><div>[2015-06-09 12:30:56.989890] E [socket.c:3015:socket_connect] 0-management: Failed to set keep-alive: Invalid argument</div><div>[2015-06-09 12:30:56.990051] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600</div><div>[2015-06-09 12:30:56.992360] W [socket.c:923:__socket_keepalive] 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 14, Invalid argument</div><div>[2015-06-09 12:30:56.992419] E [socket.c:3015:socket_connect] 0-management: Failed to set keep-alive: Invalid argument</div><div>[2015-06-09 12:30:56.992629] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600</div><div>[2015-06-09 12:30:56.994163] W [socket.c:923:__socket_keepalive] 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 15, Invalid argument</div><div>[2015-06-09 12:30:56.994177] E [socket.c:3015:socket_connect] 0-management: Failed to set keep-alive: Invalid argument</div><div>Final graph:</div><div>+------------------------------------------------------------------------------+</div><div> 1: volume management</div><div> 2: type mgmt/glusterd</div><div> 3: option rpc-auth.auth-glusterfs on</div><div> 4: option rpc-auth.auth-unix on</div><div> 5: option rpc-auth.auth-null on</div><div> 6: option transport.socket.listen-backlog 128</div><div> 7: option ping-timeout 30</div><div> 8: option transport.socket.read-fail-log off</div><div> 9: option transport.socket.keepalive-interval 2</div><div> 10: option transport.socket.keepalive-time 10</div><div> 11: option transport-type rdma</div><div> 12: option working-directory /var/lib/glusterd</div><div> 13: end-volume</div><div> 14:</div><div>+------------------------------------------------------------------------------+</div><div>[2015-06-09 12:30:57.007380] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1</div><div>[2015-06-09 12:30:57.288783] I [glusterd-rpc-ops.c:464:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 41707504-c217-4b7a-8f3a-7150bd040d0a, host: 10.10.50.74, port: 0</div><div>[2015-06-09 12:30:57.292480] I [glusterd-handler.c:2587:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 41707504-c217-4b7a-8f3a-7150bd040d0a</div><div>[2015-06-09 12:30:57.292542] I [glusterd-handler.c:2630:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend</div><div>[2015-06-09 12:30:57.293810] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped</div><div>[2015-06-09 12:30:57.303585] W [socket.c:3059:socket_connect] 0-nfs: Ignore failed connection attempt on , (No such file or directory)</div><div>[2015-06-09 12:30:57.304172] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: glustershd already stopped</div><div>[2015-06-09 12:30:57.324624] W [socket.c:3059:socket_connect] 0-glustershd: Ignore failed connection attempt on , (No such file or directory)</div><div>[2015-06-09 12:30:57.324721] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped</div><div>[2015-06-09 12:30:57.324767] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped</div><div>[2015-06-09 12:30:57.343225] I [rpc-clnt.c:972:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600</div><div>pending frames:</div><div>frame : type(0) op(0)</div><div>frame : type(0) op(0)</div><div>frame : type(0) op(0)</div><div>patchset: git://<a href="http://git.gluster.com/glusterfs.git" target="_blank">git.gluster.com/glusterfs.git</a></div><div>signal received: 6</div><div>time of crash:</div><div>2015-06-09 12:30:57</div><div>configuration details:</div><div>argp 1</div><div>backtrace 1</div><div>dlfcn 1</div><div>libpthread 1</div><div>llistxattr 1</div><div>setfsid 1</div><div>spinlock 1</div><div>epoll.h 1</div><div>xattr.h 1</div><div>st_atim.tv_nsec 1</div><div>package-string: glusterfs 3.7.1</div><div>/lib64/libglusterfs.so.0(_gf_msg_backtrace_nomem+0xb2)[0x7f3f46ca2d92]</div><div>/lib64/libglusterfs.so.0(gf_print_trace+0x32d)[0x7f3f46cbd9ed]</div><div>/lib64/libc.so.6(+0x35a00)[0x7f3f45696a00]</div><div>/lib64/libc.so.6(gsignal+0x39)[0x7f3f45696989]</div><div>/lib64/libc.so.6(abort+0x148)[0x7f3f45698098]</div><div>/lib64/libc.so.6(+0x76197)[0x7f3f456d7197]</div><div>/lib64/libc.so.6(__fortify_fail+0x37)[0x7f3f4576ecb7]</div><div>/lib64/libc.so.6(+0x10be80)[0x7f3f4576ce80]</div><div>/lib64/libc.so.6(+0x10b5bb)[0x7f3f4576c5bb]</div><div>/lib64/libc.so.6(__snprintf_chk+0x78)[0x7f3f4576c4d8]</div><div>/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_volume_defrag_restart+0x191)[0x7f3f3bb37931]</div><div>/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_restart_rebalance+0x82)[0x7f3f3bb3daa2]</div><div>/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_spawn_daemons+0x4f)[0x7f3f3bb3db1f]</div><div>/lib64/libglusterfs.so.0(synctask_wrap+0x12)[0x7f3f46cdd482]</div><div>/lib64/libc.so.6(+0x47570)[0x7f3f456a8570]</div><div>---------</div><div><br></div></div><div><br></div>-- <br><div>Jonhnny Weslley<br><a href="http://blog.jonhnnyweslley.net" target="_blank">http://blog.jonhnnyweslley.net</a></div>
</div></div>
<br>_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-devel" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
<br></blockquote></div>