<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jul 25, 2016 at 5:37 PM, Atin Mukherjee <span dir="ltr"><<a href="mailto:amukherj@redhat.com" target="_blank">amukherj@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Mon, Jul 25, 2016 at 4:34 PM, Avra Sengupta <span dir="ltr"><<a href="mailto:asengupt@redhat.com" target="_blank">asengupt@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>The crux of the problem is that as of
today, brick processes on restart try to reuse the old port they
were using (assuming that no other process will be using it, and
not consulting pmap_registry_alloc() before using it). With a
recent change, pmap_registry_alloc (), reassigns older ports that
were used, but are now free. Hence snapd now gets a port that was
previously used by a brick and tries to bind to it, whereas the
older brick process without consulting pmap table blindly tries to
connect to it, and hence we see this problem.<br>
<br>
Now coming to the fix, I feel brick process should not try to get
the older port and should just take a new port every time it comes
up. We will not run out of ports with this change coz, now pmap
allocates old ports again, and the previous port being used by the
brick process will eventually be reused. If anyone sees any
concern with this approach, please feel free to raise so now. <br></div></div></blockquote><div><br></div></span><div>Looks to be OK, but I'll think through it and get back to you by a day or two if I have any objections. <br></div></div></div></div></blockquote><div><br></div><div>If we are conservative about bricks not binding to a different port on a restart, I've an alternative approach here [1] . Neither it has a full fledged commit message nor a BZ. I've just put this up for your input?<br><br>[1] <a href="http://review.gluster.org/15005">http://review.gluster.org/15005</a><br> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div><div class="h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><div>
<br>
While awaiting feedback from you guys, I have sent this patch
(<a href="http://review.gluster.org/15001" target="_blank">http://review.gluster.org/15001</a>), which moves the said test case
to bad tests for now, and after we collectively reach to a
conclusion on the fix, we will remove this from bad test.<br>
<br>
Regards,<br>
Avra<div><div><br>
<br>
On 07/25/2016 02:33 PM, Avra Sengupta wrote:<br>
</div></div></div><div><div>
<blockquote type="cite">
<div>The failure suggests that the port
snapd is trying to bind to is already in use. But snapd has been
modified to use a new port everytime. I am looking into this.<br>
<br>
On 07/25/2016 02:23 PM, Nithya Balachandran wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">More failures:
<div><a href="https://build.gluster.org/job/rackspace-regression-2GB-triggered/22452/console" target="_blank">https://build.gluster.org/job/rackspace-regression-2GB-triggered/22452/console</a><br>
</div>
<div><br>
</div>
<div>I see these messages in the snapd.log:</div>
<div><br>
</div>
<div>
<div>[2016-07-22 05:31:52.482282] I
[rpcsvc.c:2199:rpcsvc_set_outstanding_rpc_limit]
0-rpc-service: Configured rpc.outstanding-rpc-limit with
value 64</div>
<div>[2016-07-22 05:31:52.482352] W [MSGID: 101002]
[options.c:954:xl_opt_validate] 0-patchy-server: option
'listen-port' is deprecated, preferred is
'transport.socket.listen-port', continuing with correction</div>
<div>[2016-07-22 05:31:52.482436] E
[socket.c:771:__socket_server_bind] 0-tcp.patchy-server:
binding to failed: Address already in use</div>
<div>[2016-07-22 05:31:52.482447] E
[socket.c:774:__socket_server_bind] 0-tcp.patchy-server:
Port is already in use</div>
<div>[2016-07-22 05:31:52.482459] W
[rpcsvc.c:1630:rpcsvc_create_listener] 0-rpc-service:
listening on transport failed</div>
<div>[2016-07-22 05:31:52.482469] W [MSGID: 115045]
[server.c:1061:init] 0-patchy-server: creation of listener
failed</div>
<div>[2016-07-22 05:31:52.482481] E [MSGID: 101019]
[xlator.c:433:xlator_init] 0-patchy-server: Initialization
of volume 'patchy-server' failed, review your volfile
again</div>
<div>[2016-07-22 05:31:52.482491] E [MSGID: 101066]
[graph.c:324:glusterfs_graph_init] 0-patchy-server:
initializing translator failed</div>
<div>[2016-07-22 05:31:52.482499] E [MSGID: 101176]
[graph.c:670:glusterfs_graph_activate] 0-graph: init
failed</div>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Jul 25, 2016 at 12:00 PM,
Ashish Pandey <span dir="ltr"><<a href="mailto:aspandey@redhat.com" target="_blank">aspandey@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div style="font-family:times new roman,new york,times,serif;font-size:12pt;color:rgb(0,0,0)">
<div>Hi,<br>
</div>
<div><br>
</div>
<div>Following test has failed 3 times in last two
days -</div>
<div><br>
</div>
<div>./tests/bugs/snapshot/bug-1316437.t <br>
</div>
<div><a href="https://build.gluster.org/job/rackspace-regression-2GB-triggered/22445/consoleFull" target="_blank">https://build.gluster.org/job/rackspace-regression-2GB-triggered/22445/consoleFull</a></div>
<div><a href="https://build.gluster.org/job/rackspace-regression-2GB-triggered/22445/consoleFull" target="_blank">https://build.gluster.org/job/rackspace-regression-2GB-triggered/22445/consoleFull</a></div>
<div><a href="https://build.gluster.org/job/rackspace-regression-2GB-triggered/22470/consoleFull" target="_blank">https://build.gluster.org/job/rackspace-regression-2GB-triggered/22470/consoleFull</a></div>
<div><br>
</div>
<div>Please take a look at it and check if it spurious
failure or not.<span><font color="#888888"><br>
</font></span></div>
<span><font color="#888888">
<div><br>
</div>
<div>Ashish<br>
</div>
</font></span></div>
</div>
<br>
_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</blockquote>
<br>
</div></div></div>
</blockquote></div></div></div><span class=""><font color="#888888"><br><br clear="all"><br>-- <br><div data-smartmail="gmail_signature"><div dir="ltr"><div><br></div>--Atin<br></div></div>
</font></span></div></div>
</blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><br></div>--Atin<br></div></div>
</div></div>