<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Jul 27, 2016, at 4:54 PM, Pranith Kumar Karampuri <<a href="mailto:pkarampu@redhat.com" class="">pkarampu@redhat.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><br class="Apple-interchange-newline"><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_quote" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">On Wed, Jul 27, 2016 at 11:26 PM, Gmail<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:b.s.mikhael@gmail.com" target="_blank" class="">b.s.mikhael@gmail.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word;" class="">Hello All,<div class=""><br class=""></div><div class="">I keep getting the following errors accompanied by the bricks in the same replica group going offline when trying to sync files between two volumes located in two different geographic locations using Csync2.</div><div class=""><br class=""></div><div class=""><div class=""><font face="Menlo" color="#454545" class=""><span style="font-size: 11px;" class="">[2016-07-27 17:07:06.701575] E [MSGID: 113015] [posix.c:1011:posix_opendir] 0-gvol001-posix: opendir failed on /zfsvol/brick02/gvol001/.glusterfs/f2/28/f228f015-2bf4-43a3-bfa6-803f282f13af/man3 [Too many open files]</span></font></div><div class=""><font face="Menlo" color="#454545" class=""><span style="font-size: 11px;" class="">[2016-07-27 17:07:06.701643] E [MSGID: 115056] [server-rpc-fops.c:690:server_opendir_cbk] 0-gvol001-server: 46179928: OPENDIR <gfid:25c95c97-d5a4-415c-baed-b87b847b2ec1>/perl-5.18.1/cpan/Test-Harness/blib/man3 (10bc989e-395f-44ec-911e-746335f567a0) ==> (Too many open files) [Too many open files]</span></font></div><div class=""><font face="Menlo" color="#454545" class=""><span style="font-size: 11px;" class="">[2016-07-27 17:07:06.702574] E [MSGID: 113099] [posix-helpers.c:389:_posix_xattr_get_set] 0-gvol001-posix: Opening file /zfsvol/brick02/gvol001/.glusterfs/7f/af/7faf6eff-4036-4603-a974-7e95c8fdd568 failed [Too many open files]</span></font></div><div class=""><font face="Menlo" color="#454545" class=""><span style="font-size: 11px;" class="">[2016-07-27 17:07:06.709581] E [MSGID: 113099] [posix-helpers.c:389:_posix_xattr_get_set] 0-gvol001-posix: Opening file /zfsvol/brick02/gvol001/.glusterfs/dd/d2/ddd2c86e-c296-463c-9d03-2da596bb9f88 failed [Too many open files]</span></font></div></div></div></blockquote><div class=""><br class=""></div><div class="">1) May be there is a bug in the brick stack which forgets to close a file.<br class=""></div><div class="">2) May be Csync2 doesn't close the files after syncing.<br class=""></div><div class="">Could you let us know the version of gluster and also statedump of the bricks using<span class="Apple-converted-space"> </span><a href="https://gluster.readthedocs.io/en/latest/Troubleshooting/statedump/" class="">https://gluster.readthedocs.io/en/latest/Troubleshooting/statedump/</a><br class=""></div></div></div></blockquote>I’ve tried to collect the logs as per the documentation provided, but not all of the logs was successfully collected.</div><div><br class=""></div><div># gluster volume statedump gvol001<div class="">volume statedump: failed: Commit failed on <HOSTNAME>. Please check log file for details.</div><blockquote type="cite" class=""><div class=""><div class="gmail_quote" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div class=""><br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word;" class=""><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><b class="">—Bishoy</b></div></div><br class="">_______________________________________________<br class="">Gluster-users mailing list<br class=""><a href="mailto:Gluster-users@gluster.org" class="">Gluster-users@gluster.org</a><br class=""><a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank" class="">http://www.gluster.org/mailman/listinfo/gluster-users</a><br class=""></blockquote></div><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br clear="all" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">--<span class="Apple-converted-space"> </span></span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_signature" data-smartmail="gmail_signature" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div dir="ltr" class="">Pranith</div></div></div></blockquote></div><br class=""></body></html>