<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jul 29, 2016 at 8:43 AM, Gmail <span dir="ltr"><<a href="mailto:b.s.mikhael@gmail.com" target="_blank">b.s.mikhael@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">I’ve already done that in the previous e-mail.</div></blockquote><div><br></div><div>Those were state dump files, the command failed and was saying check logs for details, so was asking for logfiles as well.<br> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div><br><div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div><b>— Bishoy</b></div></div></div></div></div>
</div>
<br><div><blockquote type="cite"><div>On Jul 28, 2016, at 3:20 PM, Gmail <<a href="mailto:b.s.mikhael@gmail.com" target="_blank">b.s.mikhael@gmail.com</a>> wrote:</div><br><div><span><glusterdump.glusterd.dump.1469742056></span><span><glusterdump.glusterfs.dump.1469742040></span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><blockquote type="cite"><span class=""><div>On Jul 27, 2016, at 4:54 PM, Pranith Kumar Karampuri <<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>> wrote:</div><br></span><span class=""><div><br><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">On Wed, Jul 27, 2016 at 11:26 PM, Gmail<span> </span><span dir="ltr"><<a href="mailto:b.s.mikhael@gmail.com" target="_blank">b.s.mikhael@gmail.com</a>></span><span> </span>wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word">Hello All,<div><br></div><div>I keep getting the following errors accompanied by the bricks in the same replica group going offline when trying to sync files between two volumes located in two different geographic locations using Csync2.</div><div><br></div><div><div><font face="Menlo" color="#454545"><span style="font-size:11px">[2016-07-27 17:07:06.701575] E [MSGID: 113015] [posix.c:1011:posix_opendir] 0-gvol001-posix: opendir failed on /zfsvol/brick02/gvol001/.glusterfs/f2/28/f228f015-2bf4-43a3-bfa6-803f282f13af/man3 [Too many open files]</span></font></div><div><font face="Menlo" color="#454545"><span style="font-size:11px">[2016-07-27 17:07:06.701643] E [MSGID: 115056] [server-rpc-fops.c:690:server_opendir_cbk] 0-gvol001-server: 46179928: OPENDIR <gfid:25c95c97-d5a4-415c-baed-b87b847b2ec1>/perl-5.18.1/cpan/Test-Harness/blib/man3 (10bc989e-395f-44ec-911e-746335f567a0) ==> (Too many open files) [Too many open files]</span></font></div><div><font face="Menlo" color="#454545"><span style="font-size:11px">[2016-07-27 17:07:06.702574] E [MSGID: 113099] [posix-helpers.c:389:_posix_xattr_get_set] 0-gvol001-posix: Opening file /zfsvol/brick02/gvol001/.glusterfs/7f/af/7faf6eff-4036-4603-a974-7e95c8fdd568 failed [Too many open files]</span></font></div><div><font face="Menlo" color="#454545"><span style="font-size:11px">[2016-07-27 17:07:06.709581] E [MSGID: 113099] [posix-helpers.c:389:_posix_xattr_get_set] 0-gvol001-posix: Opening file /zfsvol/brick02/gvol001/.glusterfs/dd/d2/ddd2c86e-c296-463c-9d03-2da596bb9f88 failed [Too many open files]</span></font></div></div></div></blockquote><div><br></div><div>1) May be there is a bug in the brick stack which forgets to close a file.<br></div><div>2) May be Csync2 doesn't close the files after syncing.<br></div><div>Could you let us know the version of gluster and also statedump of the bricks using<span> </span><a href="https://gluster.readthedocs.io/en/latest/Troubleshooting/statedump/" target="_blank">https://gluster.readthedocs.io/en/latest/Troubleshooting/statedump/</a><br></div></div></div></span></blockquote>I’ve tried to collect the logs as per the documentation provided, but not all of the logs was successfully collected.</div><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><br></div><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><span class=""># gluster volume statedump gvol001<div>volume statedump: failed: Commit failed on <HOSTNAME>. Please check log file for details.</div></span><blockquote type="cite"><div><div class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word"><div><br></div><div><br></div><div><b>—Bishoy</b></div></div><span class=""><br>_______________________________________________<br>Gluster-users mailing list<br><a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br><a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br></span></blockquote></div><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" clear="all"><span class="HOEnZb"><font color="#888888"><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important">--<span> </span></span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div data-smartmail="gmail_signature" style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div dir="ltr">Pranith</div></div></font></span></div></blockquote></div></div></blockquote></div><br></div></div></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Pranith<br></div></div>
</div></div>