<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">I’ve rebuilt the cluster again, making a fresh installation. And now the error is different.<div class=""><br class=""><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo;" class="">MASTER NODE&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; MASTER VOL&nbsp; &nbsp; MASTER BRICK&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; SLAVE USER&nbsp; &nbsp; SLAVE&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; SLAVE NODE&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; STATUS &nbsp; &nbsp; CRAWL STATUS&nbsp; &nbsp; LAST_SYNCED &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;</div><div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo;" class="">-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------</div><div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><a href="http://master-host01.me.com" class="">master-host01.me.com</a>&nbsp; &nbsp; geotest &nbsp; &nbsp; &nbsp; /gpool/brick03/geotest&nbsp; &nbsp; guser &nbsp; &nbsp; &nbsp; &nbsp; guser@slave-host01::geotestdr&nbsp; &nbsp; N/A &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class="Apple-tab-span" style="white-space:pre">        </span>Faulty &nbsp; &nbsp; N/A &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; N/A &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;</div><div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><a href="http://master-host02.me.com" class="">master-host02.me.com</a>&nbsp; &nbsp; geotest &nbsp; &nbsp; &nbsp; /gpool/brick03/geotest&nbsp; &nbsp; guser &nbsp; &nbsp; &nbsp; &nbsp; guser@slave-host01::geotestdr&nbsp; &nbsp; slave-host01&nbsp; &nbsp; <span class="Apple-tab-span" style="white-space:pre">                </span>Passive&nbsp; &nbsp; N/A &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; N/A &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;</div><div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><a href="http://master-host03.me.com" class="">master-host03.me.com</a>&nbsp; &nbsp; geotest &nbsp; &nbsp; &nbsp; /gpool/brick03/geotest&nbsp; &nbsp; guser &nbsp; &nbsp; &nbsp; &nbsp; guser@slave-host01::geotestdr&nbsp; &nbsp; slave-host03&nbsp; &nbsp; <span class="Apple-tab-span" style="white-space:pre">                </span>Passive&nbsp; &nbsp; N/A &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; N/A &nbsp; &nbsp;</div><div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><br class=""></div><div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><br class=""></div><div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><br class=""></div><div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><br class=""></div><div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><br class=""></div><div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><br class=""></div><div style="margin: 0px; line-height: normal;" class=""><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:31.326898] I [monitor(monitor):221:monitor] Monitor: ------------------------------------------------------------</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:31.327461] I [monitor(monitor):222:monitor] Monitor: starting gsyncd worker</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:31.544631] I [gsyncd(/gpool/brick03/geotest):649:main_i] &lt;top&gt;: syncing: <a href="gluster://localhost:geotest" class="">gluster://localhost:geotest</a> -&gt; <a href="ssh://guser@slave-host02:gluster://localhost:geotestdr" class="">ssh://guser@slave-host02:gluster://localhost:geotestdr</a></div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:31.547542] I [changelogagent(agent):75:__init__] ChangelogAgent: Agent listining...</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:31.830554] E [syncdutils(/gpool/brick03/geotest):252:log_raise_exception] &lt;top&gt;: connection to peer is broken</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:31.831017] W [syncdutils(/gpool/brick03/geotest):256:log_raise_exception] &lt;top&gt;: !!!!!!!!!!!!!</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:31.831258] W [syncdutils(/gpool/brick03/geotest):257:log_raise_exception] &lt;top&gt;: !!! getting "No such file or directory" errors is most likely due to MISCONFIGURATION, please consult <a href="https://access.redhat.com/site/documentation/en-US/Red_Hat_Storage/2.1/html/Administration_Guide/chap-User_Guide-Geo_Rep-Preparation-Settingup_Environment.html" class="">https://access.redhat.com/site/documentation/en-US/Red_Hat_Storage/2.1/html/Administration_Guide/chap-User_Guide-Geo_Rep-Preparation-Settingup_Environment.html</a></div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:31.831502] W [syncdutils(/gpool/brick03/geotest):265:log_raise_exception] &lt;top&gt;: !!!!!!!!!!!!!</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:31.836395] E [resource(/gpool/brick03/geotest):222:errlog] Popen: command "ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -oControlMaster=auto -S /tmp/gsyncd-aux-ssh-SfXvbB/de372ce5774b5d259c58c5c9522ffc8f.sock guser@slave-host02 /nonexistent/gsyncd --session-owner ec473e17-b933-4bf7-9eed-4c393f7aaf5d -N --listen --timeout 120 <a href="gluster://localhost:geotestdr" class="">gluster://localhost:geotestdr</a>" returned with 127, saying:</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:31.836694] E [resource(/gpool/brick03/geotest):226:logerr] Popen: ssh&gt; bash: /nonexistent/gsyncd: No such file or directory</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:31.837193] I [syncdutils(/gpool/brick03/geotest):220:finalize] &lt;top&gt;: exiting.</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:31.840569] I [repce(agent):92:service_loop] RepceServer: terminating on reaching EOF.</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:31.840993] I [syncdutils(agent):220:finalize] &lt;top&gt;: exiting.</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:31.840742] I [monitor(monitor):274:monitor] Monitor: worker(/gpool/brick03/geotest) died before establishing connection</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:42.130866] I [monitor(monitor):221:monitor] Monitor: ------------------------------------------------------------</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:42.131448] I [monitor(monitor):222:monitor] Monitor: starting gsyncd worker</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:42.348165] I [gsyncd(/gpool/brick03/geotest):649:main_i] &lt;top&gt;: syncing: <a href="gluster://localhost:geotest" class="">gluster://localhost:geotest</a> -&gt; <a href="ssh://guser@slave-host02:gluster://localhost:geotestdr" class="">ssh://guser@slave-host02:gluster://localhost:geotestdr</a></div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:42.349118] I [changelogagent(agent):75:__init__] ChangelogAgent: Agent listining...</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:42.653141] E [syncdutils(/gpool/brick03/geotest):252:log_raise_exception] &lt;top&gt;: connection to peer is broken</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:42.653656] W [syncdutils(/gpool/brick03/geotest):256:log_raise_exception] &lt;top&gt;: !!!!!!!!!!!!!</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:42.653898] W [syncdutils(/gpool/brick03/geotest):257:log_raise_exception] &lt;top&gt;: !!! getting "No such file or directory" errors is most likely due to MISCONFIGURATION, please consult <a href="https://access.redhat.com/site/documentation/en-US/Red_Hat_Storage/2.1/html/Administration_Guide/chap-User_Guide-Geo_Rep-Preparation-Settingup_Environment.html" class="">https://access.redhat.com/site/documentation/en-US/Red_Hat_Storage/2.1/html/Administration_Guide/chap-User_Guide-Geo_Rep-Preparation-Settingup_Environment.html</a></div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:42.654129] W [syncdutils(/gpool/brick03/geotest):265:log_raise_exception] &lt;top&gt;: !!!!!!!!!!!!!</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:42.659329] E [resource(/gpool/brick03/geotest):222:errlog] Popen: command "ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -oControlMaster=auto -S /tmp/gsyncd-aux-ssh-6r8rxx/de372ce5774b5d259c58c5c9522ffc8f.sock guser@slave-host02 /nonexistent/gsyncd --session-owner ec473e17-b933-4bf7-9eed-4c393f7aaf5d -N --listen --timeout 120 <a href="gluster://localhost:geotestdr" class="">gluster://localhost:geotestdr</a>" returned with 127, saying:</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:42.659626] E [resource(/gpool/brick03/geotest):226:logerr] Popen: ssh&gt; bash: /nonexistent/gsyncd: No such file or directory</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:42.660140] I [syncdutils(/gpool/brick03/geotest):220:finalize] &lt;top&gt;: exiting.</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:42.662802] I [repce(agent):92:service_loop] RepceServer: terminating on reaching EOF.</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:42.663197] I [syncdutils(agent):220:finalize] &lt;top&gt;: exiting.</div><div style="font-family: Menlo; font-size: 11px; margin: 0px; line-height: normal;" class="">[2016-03-30 22:09:42.663024] I [monitor(monitor):274:monitor] Monitor: worker(/gpool/brick03/geotest) died before establishing connection</div><div style="font-family: Menlo; font-size: 11px;" class=""><br class=""></div><div style="font-family: Menlo; font-size: 11px;" class=""><br class=""></div><div class="">—Bishoy</div><div style="font-family: Menlo; font-size: 11px;" class=""><br class=""></div></div><div><blockquote type="cite" class=""><div class="">On Mar 30, 2016, at 10:50 AM, Gmail &lt;<a href="mailto:b.s.mikhael@gmail.com" class="">b.s.mikhael@gmail.com</a>&gt; wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=windows-1252" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">I’ve tried changing the permissions to 777 on /var/log/glusterfs on all the slave nodes, but still no luck :(<div class=""><br class=""></div><div class="">here is the log from the master node where I created and started the geo-replication session.</div><div class=""><br class=""></div><div class=""><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:53.463150] I [monitor(monitor):221:monitor] Monitor: ------------------------------------------------------------</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:53.463669] I [monitor(monitor):222:monitor] Monitor: starting gsyncd worker</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:53.603774] I [changelogagent(agent):75:__init__] ChangelogAgent: Agent listining...</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:53.604080] I [gsyncd(/mnt/brick10/xfsvol2):649:main_i] &lt;top&gt;: syncing:&nbsp;<a href="gluster://localhost:xfsvol2" class="">gluster://localhost:xfsvol2</a>&nbsp;-&gt;&nbsp;<a href="ssh://guser@slave-host01:gluster://localhost:xfsvol2dr" class="">ssh://guser@slave-host01:gluster://localhost:xfsvol2dr</a></div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.210602] E [syncdutils(/mnt/brick10/xfsvol2):252:log_raise_exception] &lt;top&gt;: connection to peer is broken</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.211117] E [resource(/mnt/brick10/xfsvol2):222:errlog] Popen: command "ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -oControlMaster=auto -S /tmp/gsyncd-aux-ssh-evONxc/3bda60dc6e900c0833fed4e4fdfbd480.sock guser@slave-host01 /nonexistent/gsyncd --session-owner ef9ccae5-0def-4a47-9a96-881a1896755c -N --listen --timeout 120&nbsp;<a href="gluster://localhost:xfsvol2dr" class="">gluster://localhost:xfsvol2dr</a>" returned with 1, saying:</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.211376] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; [2016-03-30 17:14:53.933174] I [cli.c:720:main] 0-cli: Started running /usr/sbin/gluster with version 3.7.3</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.211631] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; [2016-03-30 17:14:53.933225] I [cli.c:608:cli_rpc_init] 0-cli: Connecting to remote glusterd at localhost</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.211828] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; [2016-03-30 17:14:54.074207] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.212017] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; [2016-03-30 17:14:54.074302] I [socket.c:2409:socket_event_handler] 0-transport: disconnecting now</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.212199] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; [2016-03-30 17:14:54.077207] I [cli-rpc-ops.c:6230:gf_cli_getwd_cbk] 0-cli: Received resp to getwd</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.212380] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; [2016-03-30 17:14:54.077269] I [input.c:36:cli_batch] 0-: Exiting with: 0</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.212584] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; ERROR:root:FAIL:&nbsp;</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.212774] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; Traceback (most recent call last):</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.212954] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 165, in main</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.213131] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; &nbsp; main_i()</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.213308] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 638, in main_i</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.213500] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; &nbsp; startup(go_daemon=go_daemon, log_file=log_file, label=label)</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.213690] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 144, in startup</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.213890] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; &nbsp; GLogger._gsyncd_loginit(**kw)</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.214068] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 107, in _gsyncd_loginit</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.214246] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; &nbsp; cls.setup(label=kw.get('label'), **lkw)</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.214422] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 79, in setup</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.214622] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; &nbsp; logging_handler = handlers.WatchedFileHandler(lprm['filename'])</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.214802] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; File "/usr/lib64/python2.6/logging/handlers.py", line 377, in __init__</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.214977] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; &nbsp; logging.FileHandler.__init__(self, filename, mode, encoding, delay)</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.215152] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; File "/usr/lib64/python2.6/logging/__init__.py", line 835, in __init__</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.215327] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; &nbsp; StreamHandler.__init__(self, self._open())</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.215523] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; File "/usr/lib64/python2.6/logging/__init__.py", line 854, in _open</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.215703] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; &nbsp; &nbsp; stream = open(self.baseFilename, self.mode)</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.215883] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; IOError: [Errno 13] Permission denied: '/var/log/glusterfs/geo-replication-slaves/mbr/ef9ccae5-0def-4a47-9a96-881a1896755c:gluster%3A%2F%2F127.0.0.1%3Axfsvol2dr.log'</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.216063] E [resource(/mnt/brick10/xfsvol2):226:logerr] Popen: ssh&gt; failed with IOError.</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.216500] I [syncdutils(/mnt/brick10/xfsvol2):220:finalize] &lt;top&gt;: exiting.</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.218672] I [repce(agent):92:service_loop] RepceServer: terminating on reaching EOF.</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.219063] I [syncdutils(agent):220:finalize] &lt;top&gt;: exiting.</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;">[2016-03-30 17:14:54.218930] I [monitor(monitor):274:monitor] Monitor: worker(/mnt/brick10/xfsvol2) died before establishing connection</div><div class="" style="margin: 0px; font-size: 11px; font-family: Menlo;"><br class=""></div><div class="" style="margin: 0px;"><b class="">—Bishoy</b></div><div class="" style="margin: 0px;"><b class=""><br class=""></b></div><div class=""><blockquote type="cite" class=""><div class="">On Mar 29, 2016, at 1:05 AM, Aravinda &lt;<a href="mailto:avishwan@redhat.com" class="">avishwan@redhat.com</a>&gt; wrote:</div><br class="Apple-interchange-newline"><div class="">
  
    <meta content="text/html; charset=windows-1252" http-equiv="Content-Type" class="">
  
  <div text="#000000" bgcolor="#FFFFFF" class="">
    Geo-replication command should be run as privileged user itself.<br class="">
    <br class="">
    gluster volume geo-replication &lt;MASTERVOL&gt;
    &lt;SLAVEUSER&gt;@&lt;SLAVEHOST&gt; start<br class="">
    <br class="">
    and then check the status, if it shows Faulty then please share the
    log files present in
    /var/log/glusterfs/geo-replication/&lt;MASTERVOL&gt;/*.log<br class="">
    <br class="">
    <pre class="moz-signature" cols="72">regards
Aravinda</pre>
    <div class="moz-cite-prefix">On 03/29/2016 12:51 PM, Gmail wrote:<br class="">
    </div>
    <blockquote cite="mid:387A76C3-2569-47BD-8105-5FF136EAFCD6@gmail.com" type="cite" class="">
      <meta http-equiv="Content-Type" content="text/html;
        charset=windows-1252" class="">
      I’ve been trying to setup geo-replication using Gluster 3.7.3 on
      OEL 6.5
      <div class="">It keeps giving me faulty session.</div>
      <div class="">I’ve tried to use root user instead, it works fine!</div>
      <div class=""><br class="">
      </div>
      <div class="">I’ve followed literally the documentation but no
        luck getting the unprivileged user working.</div>
      <div class=""><br class="">
      </div>
      <div class="">I’ve tried running&nbsp;<span style="font-family: Menlo;
          font-size: 11px;" class="">/usr/libexec/glusterfs/gsyncd </span>on
        the slave node using the&nbsp;unprivileged user, and that’s what I
        get.</div>
      <div class=""><font class="" face="Menlo"><span style="font-size:
            11px;" class=""><br class="">
          </span></font></div>
      <div class="">
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">/usr/libexec/glusterfs/gsyncd&nbsp; --session-owner
          ef9ccae5-0def-4a47-9a96-881a1896755c -N --listen --timeout
          120&nbsp;<a moz-do-not-send="true" href="gluster://localhost:vol01dr" class="">gluster://localhost:vol01dr</a></div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">[2016-03-29 00:52:49.058244] I [cli.c:720:main] 0-cli:
          Started running /usr/sbin/gluster with version 3.7.3</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">[2016-03-29 00:52:49.058297] I
          [cli.c:608:cli_rpc_init] 0-cli: Connecting to remote glusterd
          at localhost</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">[2016-03-29 00:52:49.174686] I [MSGID: 101190]
          [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll:
          Started thread with index 1</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">[2016-03-29 00:52:49.174768] I
          [socket.c:2409:socket_event_handler] 0-transport:
          disconnecting now</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">[2016-03-29 00:52:49.177482] I
          [cli-rpc-ops.c:6230:gf_cli_getwd_cbk] 0-cli: Received resp to
          getwd</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">[2016-03-29 00:52:49.177545] I [input.c:36:cli_batch]
          0-: Exiting with: 0</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">ERROR:root:FAIL:&nbsp;</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">Traceback (most recent call last):</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; File
          "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line
          165, in main</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; &nbsp; main_i()</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; File
          "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line
          638, in main_i</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; &nbsp; startup(go_daemon=go_daemon, log_file=log_file,
          label=label)</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; File
          "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line
          144, in startup</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; &nbsp; GLogger._gsyncd_loginit(**kw)</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; File
          "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line
          107, in _gsyncd_loginit</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; &nbsp; cls.setup(label=kw.get('label'), **lkw)</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; File
          "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 79,
          in setup</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; &nbsp; logging_handler =
          handlers.WatchedFileHandler(lprm['filename'])</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; File "/usr/lib64/python2.6/logging/handlers.py",
          line 377, in __init__</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; &nbsp; logging.FileHandler.__init__(self, filename, mode,
          encoding, delay)</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; File "/usr/lib64/python2.6/logging/__init__.py",
          line 835, in __init__</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; &nbsp; StreamHandler.__init__(self, self._open())</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; File "/usr/lib64/python2.6/logging/__init__.py",
          line 854, in _open</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">&nbsp; &nbsp; stream = open(self.baseFilename, self.mode)</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">IOError: [Errno 13] Permission denied:
'/var/log/glusterfs/geo-replication-slaves/mbr/ef9ccae5-0def-4a47-9a96-881a1896755c:gluster%3A%2F%2F127.0.0.1%3Avol01dr.log'</div>
        <div class="" style="margin: 0px; font-size: 11px; font-family:
          Menlo;">failed with IOError.</div>
      </div>
      <div class="" style="margin: 0px; font-size: 11px; font-family:
        Menlo;"><br class="">
      </div>
      <div class="" style="margin: 0px; font-size: 11px; font-family:
        Menlo;"><br class="">
      </div>
      <div class="">
        <div style="letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
          <div style="letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
            <div style="letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
              <div style="letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
                <div class=""><b class="">— Bishoy</b></div>
              </div>
            </div>
          </div>
        </div>
      </div>
      <br class="">
      <br class="">
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br class="">
      <pre wrap="" class="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a></pre>
    </blockquote>
    <br class="">
  </div>

</div></blockquote></div><br class=""></div></div></div></blockquote></div><br class=""></div></div></body></html>