<HTML><HEAD>
<STYLE id=eMClientCss>
<![CDATA[blockquote.cite { margin-left: 5px; margin-right: 0px; padding-left: 10px; padding-right:0px; border-left: 1px solid #cccccc }
blockquote.cite2 {margin-left: 5px; margin-right: 0px; padding-left: 10px; padding-right:0px; border-left: 1px solid #cccccc; margin-top: 3px; padding-top: 0px; }
.plain pre, .plain tt { font-family: monospace; font-size: 100%; font-weight: normal; font-style: normal; white-space: pre-wrap; }
a img { border: 0px; }body {font-family: Times New Roman;font-size: 12pt;}
.plain pre, .plain tt {font-family: Times New Roman;font-size: 12pt;}
]]></STYLE>
<STYLE>#x5be7004997dc4a5aaafac920d7fd5cb2 BLOCKQUOTE.cite2
{MARGIN-TOP: 3px; PADDING-TOP: 0px; PADDING-LEFT: 10px; MARGIN-LEFT: 5px; BORDER-LEFT: #cccccc 1px solid; PADDING-RIGHT: 0px; MARGIN-RIGHT: 0px}
#x5be7004997dc4a5aaafac920d7fd5cb2 .plain PRE, #x5be7004997dc4a5aaafac920d7fd5cb2 .plain TT
{FONT-SIZE: 100%; FONT-FAMILY: monospace; WHITE-SPACE: pre-wrap; FONT-WEIGHT: normal; FONT-STYLE: normal}
#x5be7004997dc4a5aaafac920d7fd5cb2 A IMG
{BORDER-TOP: 0px; BORDER-RIGHT: 0px; BORDER-BOTTOM: 0px; BORDER-LEFT: 0px}
#x5be7004997dc4a5aaafac920d7fd5cb2 .plain PRE, #x5be7004997dc4a5aaafac920d7fd5cb2 .plain TT, #x5be7004997dc4a5aaafac920d7fd5cb2
{FONT-SIZE: 12pt; FONT-FAMILY: Times New Roman}
#x5be7004997dc4a5aaafac920d7fd5cb2 #x2c6bddfe6a90449b8a2838ae2a9ba8cc BLOCKQUOTE.cite2
{MARGIN-TOP: 3px; PADDING-TOP: 0px; PADDING-LEFT: 10px; MARGIN-LEFT: 5px; BORDER-LEFT: #cccccc 1px solid; PADDING-RIGHT: 0px; MARGIN-RIGHT: 0px}
#x5be7004997dc4a5aaafac920d7fd5cb2 #x2c6bddfe6a90449b8a2838ae2a9ba8cc .plain PRE, #x5be7004997dc4a5aaafac920d7fd5cb2 #x2c6bddfe6a90449b8a2838ae2a9ba8cc .plain TT
{FONT-SIZE: 100%; FONT-FAMILY: monospace; WHITE-SPACE: pre-wrap; FONT-WEIGHT: normal; FONT-STYLE: normal}
#x5be7004997dc4a5aaafac920d7fd5cb2 #x2c6bddfe6a90449b8a2838ae2a9ba8cc A IMG
{BORDER-TOP: 0px; BORDER-RIGHT: 0px; BORDER-BOTTOM: 0px; BORDER-LEFT: 0px}
#x5be7004997dc4a5aaafac920d7fd5cb2 #x2c6bddfe6a90449b8a2838ae2a9ba8cc, #x5be7004997dc4a5aaafac920d7fd5cb2 #x2c6bddfe6a90449b8a2838ae2a9ba8cc .plain PRE, #x5be7004997dc4a5aaafac920d7fd5cb2 #x2c6bddfe6a90449b8a2838ae2a9ba8cc .plain TT
{FONT-SIZE: 12pt; FONT-FAMILY: Times New Roman}
</STYLE>
</HEAD>
<BODY scroll=auto class>
<DIV>I changed the logging to "error" to get rid of these messages as I was wondering if this was part of the problem. It didn't change the performance.</DIV>
<DIV>Also, I get these same errors both before and after the reboot. I only see the slowdown after the reboot. </DIV>
<DIV> </DIV>
<DIV>I have SELinux disabled. Not sure about ACL. Don't think I can turn ACL off on XFS.</DIV>
<DIV> </DIV>
<DIV>I am happy to post results of strace. Do I just do 'strace tar -xPf boost.tar >& strace.log'?</DIV>
<DIV> </DIV>
<DIV>David</DIV>
<DIV> </DIV>
<DIV> </DIV>
<DIV>------ Original Message ------</DIV>
<DIV>From: "Jeff Darcy" <<A href="mailto:jdarcy@redhat.com">jdarcy@redhat.com</A>></DIV>
<DIV>To: "David Robinson" <<A href="mailto:drobinson@corvidtec.com">drobinson@corvidtec.com</A>></DIV>
<DIV>Cc: <A href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</A>; "Gluster Devel" <<A href="mailto:gluster-devel@gluster.org">gluster-devel@gluster.org</A>></DIV>
<DIV>Sent: 8/18/2015 4:49:40 PM</DIV>
<DIV>Subject: Re: [Gluster-users] gluster small file performance</DIV>
<DIV> </DIV>
<DIV id=x5be7004997dc4a5aaafac920d7fd5cb2>
<BLOCKQUOTE class=cite2 cite=300443767.13398623.1439930980719.JavaMail.zimbra@redhat.com type="cite">
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif; COLOR: #000000">
<BLOCKQUOTE style="FONT-SIZE: 12pt; TEXT-DECORATION: none; FONT-FAMILY: Helvetica,Arial,sans-serif; FONT-WEIGHT: normal; COLOR: #000; FONT-STYLE: normal; PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #1010ff 2px solid">
<DIV id=x2c6bddfe6a90449b8a2838ae2a9ba8cc class=plain>
<DIV>Note: The log files attached have the "No data available" messages parsed out to reduce the file size. There were an enormous amount of these. One of my colleagues submitted something to the message board about these errors in 3.7.3.</DIV>
<BLOCKQUOTE style="MARGIN-RIGHT: 0px" dir=ltr>
<DIV><SPAN style="FONT-SIZE: small" data-mce-style="font-size: small;" size="2"><EM>[2015-08-17 17:03:37.270219] W [fuse-bridge.c:1230:fuse_err_cbk] 0-glusterfs-fuse: 6643: REMOVEXATTR() /boost_1_57_0/boost/accumulators/accumulators.hpp => -1 (No data available)<BR>[2015-08-17 17:03:37.271004] W [fuse-bridge.c:1230:fuse_err_cbk] 0-glusterfs-fuse: 6646: REMOVEXATTR() /boost_1_57_0/boost/accumulators/accumulators.hpp => -1 (No data available)<BR>[2015-08-17 17:03:37.271663] W [fuse-bridge.c:1230:fuse_err_cbk] 0-glusterfs-fuse: 6648: REMOVEXATTR() /boost_1_57_0/boost/accumulators/accumulators.hpp => -1 (No data available)<BR>[2015-08-17 17:03:37.274273] W [fuse-bridge.c:1230:fuse_err_cbk] 0-glusterfs-fuse: 6662: REMOVEXATTR() /boost_1_57_0/boost/accumulators/accumulators_fwd.hpp => -1 (No data available)</EM></SPAN></DIV></BLOCKQUOTE></DIV></BLOCKQUOTE>
<DIV><BR></DIV>
<DIV>I can't help but wonder how much these are affecting your performance. That's a lot of extra messages, and even more effort to log the failures. When I run your tests myself, I don't see any of these and I don't see a performance drop-off either. Maybe something ACL- or SELinux-related? It would be extra-helpful to get a stack trace for just one of these, to see where they're coming from.</DIV>
<DIV><BR></DIV>
<DIV><BR></DIV>
<STYLE><![CDATA[#x2c6bddfe6a90449b8a2838ae2a9ba8cc BLOCKQUOTE.cite
{PADDING-LEFT: 10px; MARGIN-LEFT: 5px; BORDER-LEFT: #cccccc 1px solid; PADDING-RIGHT: 0px; MARGIN-RIGHT: 0px}
#x2c6bddfe6a90449b8a2838ae2a9ba8cc BLOCKQUOTE.cite2
{MARGIN-TOP: 3px; PADDING-TOP: 0px; PADDING-LEFT: 10px; MARGIN-LEFT: 5px; BORDER-LEFT: #cccccc 1px solid; PADDING-RIGHT: 0px; MARGIN-RIGHT: 0px}
#x2c6bddfe6a90449b8a2838ae2a9ba8cc .plain PRE, #x2c6bddfe6a90449b8a2838ae2a9ba8cc .plain TT
{FONT-SIZE: 100%; FONT-FAMILY: monospace; WHITE-SPACE: pre-wrap; FONT-WEIGHT: normal; FONT-STYLE: normal}
#x2c6bddfe6a90449b8a2838ae2a9ba8cc A IMG
{BORDER-TOP: 0px; BORDER-RIGHT: 0px; BORDER-BOTTOM: 0px; BORDER-LEFT: 0px}
#x2c6bddfe6a90449b8a2838ae2a9ba8cc #x6c555ae0819a4526b0bbad1bc21c76d6, #x2c6bddfe6a90449b8a2838ae2a9ba8cc #xa2e15f847053498c9f73f30e6507a826, #x2c6bddfe6a90449b8a2838ae2a9ba8cc .plain PRE, #x2c6bddfe6a90449b8a2838ae2a9ba8cc .plain TT, #x2c6bddfe6a90449b8a2838ae2a9ba8cc
{FONT-SIZE: 12pt; FONT-FAMILY: Times New Roman}
]]></STYLE>
</DIV></BLOCKQUOTE></DIV></BODY></HTML>