<div dir="ltr"><div><div><div><div><div><div>I am seeing a pause when the .t runs that seem to last close to how much ever time we put in EXPECT_WITHIN<br><br>[2016-09-01 03:24:21.852744] I [common.c:1134:pl_does_monkey_want_stuck_lock] 0-patchy-locks: stuck lock<br>[2016-09-01 03:24:21.852775] W [inodelk.c:659:pl_inode_setlk] 0-patchy-locks: MONKEY LOCKING (forcing stuck lock)! at 2016-09-01 03:24:21<br>[2016-09-01 03:24:21.852792] I [server-rpc-fops.c:317:server_finodelk_cbk] 0-patchy-server: replied<br>[2016-09-01 03:24:21.861937] I [server-rpc-fops.c:5682:server3_3_inodelk] 0-patchy-server: inbound<br>[2016-09-01 03:24:21.862318] I [server-rpc-fops.c:278:server_inodelk_cbk] 0-patchy-server: replied<br>[2016-09-01 03:24:21.862627] I [server-rpc-fops.c:5682:server3_3_inodelk] 0-patchy-server: inbound <<---- No I/O after this.<br>[2016-09-01 03:27:19.6N]:++++++++++ G_LOG:tests/features/lock_revocation.t: TEST: 52 append_to_file /mnt/glusterfs/1/testfile ++++++++++<br>[2016-09-01 03:27:19.871044] I [server-rpc-fops.c:5772:server3_3_finodelk] 0-patchy-server: inbound<br>[2016-09-01 03:27:19.871280] I [clear.c:219:clrlk_clear_inodelk] 0-patchy-locks: 2<br>[2016-09-01 03:27:19.871307] I [clear.c:273:clrlk_clear_inodelk] 0-patchy-locks: released_granted<br>[2016-09-01 03:27:19.871330] I [server-rpc-fops.c:278:server_inodelk_cbk] 0-patchy-server: replied<br>[2016-09-01 03:27:19.871389] W [inodelk.c:228:__inodelk_prune_stale] 0-patchy-locks: Lock revocation [reason: age; gfid: 3ccca736-ba89-4f8c-ba17-f6cdbcd0e3c3; domain: patchy-replicate-0; age: 178 sec] - Inode lock revoked: 0 granted & 1 blocked locks cleared<br><br></div>We can prevent the hang with adding $CLI volume stop $V0, but the test would fail. When that happens, the following error is printed on the console from perfused<br><br>perfused: perfuse_node_inactive: perfuse_node_fsync failed error = 57: Resource temporarily unavailable <<--- I wonder if this comes because INODELK fop fails with EAGAIN.<br><br></div>I am also seeing a weird behaviour where it says it is releasing granted locks but prints that it released 1 blocked lock. <br><br></div>+Manu<br></div>I think there are 2 things going on here. 1) There is a hang, I am still guessing it is gluster issue until proven otherwise.<br></div>2) I got to figure out why the counters are showing wrong information from the information printed in the logs. I kept going through the code, it seems fine. It should have printed that it released 1 granted lock & 0 blocked locks. But it prints it in reverse.<br><br></div>If you do git diff on <span style="font-weight:normal"><font size="2"><a href="http://nbslave72.cloud.gluster.org">nbslave72.cloud.gluster.org</a>, you can see the changes I made. Could you please help?<br></font></span><div><div><div><div><div><div><br></div></div></div></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Aug 28, 2016 at 7:36 AM, Atin Mukherjee <span dir="ltr"><<a href="mailto:amukherj@redhat.com" target="_blank">amukherj@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">This is still bothering us a lot and looks like there is a genuine issue in the code which is making the the process to be hung/deadlocked<span></span>?<div><br></div><div>Raghavendra T - any more findings?<div><div class="h5"><br><br>On Friday 19 August 2016, Atin Mukherjee <<a href="mailto:amukherj@redhat.com" target="_blank">amukherj@redhat.com</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><a href="https://bugzilla.redhat.com/show_bug.cgi?id=1368421" target="_blank">https://bugzilla.redhat.com/sh<wbr>ow_bug.cgi?id=1368421</a><br><br></div><div>NetBSD regressions are getting aborted very frequently. Apart from the infra issue related to connectivity (Nigel has started looking into it), lock_revocation.t is getting hung in such instances which is causing run to be aborted after 300 minutes. This has already started impacting the patches to get in which eventually impacts the upcoming release cycles.<br><br></div><div>I'd request the feature owner/maintainer to have a look at it asap.<br></div><div><div data-smartmail="gmail_signature"><div dir="ltr"><div><br></div>--Atin<br></div></div>
</div></div>
</blockquote></div></div></div><span class="HOEnZb"><font color="#888888"><br><br>-- <br>--Atin<br>
</font></span><br>______________________________<wbr>_________________<br>
maintainers mailing list<br>
<a href="mailto:maintainers@gluster.org">maintainers@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/maintainers" rel="noreferrer" target="_blank">http://www.gluster.org/<wbr>mailman/listinfo/maintainers</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Pranith<br></div></div>
</div>