<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body bgcolor="#FFFFFF">
That makes sense, but now that the brick is missing and the data is not replicated, how don I get the brick back in to the trusted pool?<br>
<br>
<div>-------- Original message --------</div>
<div>From: Subrata Ghosh </div>
<div>Date:02/14/2015 9:55 PM (GMT-08:00) </div>
<div>To: Thomas Holkenbrink , &quot;'gluster-users@gluster.org'&quot; </div>
<div>Subject: Re: [Gluster-users] Gluster replicate-brick issues (Distrubuted-Replica)
</div>
<div><br>
</div>
<div>
<p>Please try to use &quot;commit force&quot; option and see whether it works as gluster&nbsp; suggested the other only replace-brick command has been depreciated.<br>
<br>
</p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">[root@Server1 ~]# gluster volume replace-brick $volname $from $to start</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">All replace-brick commands except commit force are deprecated. Do you want to continue? (y/n) y</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">volume replace-brick: success: replace-brick started successfully</span></p>
<span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">ID: 0062d555-e7eb-4ebe-a264-7e0baf6e7546<br>
<br>
</span>
<p><span style="font-size:8pt">You could try following&nbsp; options , used to resolve our replace-brcik scenario&nbsp; looks like close to your case&nbsp; &quot;gluster heal info&quot; has some issues. That I requested for a clarification fewdays back.</span></p>
<p><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;"></span></p>
<span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;<br>
</span>
<p># gluster volume replace-bricks $vol_name $old_brick $new_brick commit force</p>
<p># gluster volume heal $vol_name full</p>
#gluster volume $vol_name info &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;--&gt; this shows that the Numer of files=1, even though the file is already healed.<br>
<br>
<br>
Regards,<br>
Subrata<br>
<br>
<div class="moz-cite-prefix">On 02/14/2015 02:28 PM, Thomas Holkenbrink wrote:<br>
</div>
<blockquote type="cite"><style>
<!--
@font-face
        {font-family:"Cambria Math"}
@font-face
        {font-family:Calibri}
@font-face
        {font-family:Tahoma}
p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0in;
        margin-bottom:.0001pt;
        font-size:11.0pt;
        font-family:"Calibri","sans-serif"}
a:link, span.MsoHyperlink
        {color:blue;
        text-decoration:underline}
a:visited, span.MsoHyperlinkFollowed
        {color:purple;
        text-decoration:underline}
pre
        {margin:0in;
        margin-bottom:.0001pt;
        font-size:10.0pt;
        font-family:"Courier New"}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
        {margin:0in;
        margin-bottom:.0001pt;
        font-size:8.0pt;
        font-family:"Tahoma","sans-serif"}
span.EmailStyle17
        {font-family:"Calibri","sans-serif";
        color:windowtext}
span.BalloonTextChar
        {font-family:"Tahoma","sans-serif"}
span.HTMLPreformattedChar
        {font-family:"Courier New"}
.MsoChpDefault
        {font-family:"Calibri","sans-serif"}
@page WordSection1
        {margin:1.0in 1.0in 1.0in 1.0in}
div.WordSection1
        {}
-->
</style>
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#002060">We have tried to migrate a Brick from one server to another using the following commands. &nbsp;&nbsp;But the data is NOT being replicated… and the BRICK is not showing up anymore.</span></p>
<p class="MsoNormal"><span style="color:#002060">Gluster still appears to be working but the Bricks are not balanced and I need to add the other Brick for Server3 that I don’t want to do until after Server1:Brick2 gets replicated.</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">This is the command to create the Original Volume:</span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="">[root@Server1 ~]# </span>
<span style="">gluster volume create Storage1 replica 2 transport tcp Server1:/exp/br01/brick1 Server2:/exp/br01/brick1 Server1:/exp/br02/brick2 Server2:/exp/br02/brick2</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt; color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt; color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">This is the Current configuration BEFORE the migration.. Server3 has been Peer Probed successfully but that has been it</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">[root@Server1 ~]# gluster --version</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">glusterfs 3.6.2 built on Jan 22 2015 12:58:11</span><span style="color:black"></span></p>
<p class="MsoNormal"><span style="color:black">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">[root@Server1 ~]# gluster volume status</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Status of volume: Storage1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Gluster process&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Port&nbsp;&nbsp;&nbsp; Online&nbsp; Pid</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">------------------------------------------------------------------------------</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Brick Server1:/exp/br01/brick1&nbsp; 49152&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2167</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Brick Server2:/exp/br01/brick1&nbsp; 49152&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2192</span></p>
<p class="MsoNormal" style="margin-left:.5in"><b><span style="">Brick Server1:/exp/br02/brick2&nbsp; 49153&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2172&nbsp;&nbsp; &lt;--- this is the one that goes missing</span></b></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Brick Server2:/exp/br02/brick2&nbsp; 49153&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2193</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">NFS Server on localhost&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;2049&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2181</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Self-heal Daemon on localhost&nbsp;&nbsp; N/A&nbsp;&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2186</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">NFS Server on Server2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2049&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2205</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Self-heal Daemon on Server2&nbsp;&nbsp;&nbsp;&nbsp; N/A&nbsp;&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2210</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">NFS Server on Server3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2049&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 6015</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Self-heal Daemon on Server3&nbsp;&nbsp;&nbsp;&nbsp; N/A&nbsp;&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 6016</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Task Status of Volume Storage1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">------------------------------------------------------------------------------</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">There are no active volume tasks</span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="">[root@Server1 ~]# gluster volume info</span></p>
<p class="MsoNormal"><span style="">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Volume Name: Storage1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Type: Distributed-Replicate</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Volume ID: 9616ce42-48bd-4fe3-883f-decd6c4fcd00</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Status: Started</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Number of Bricks: 2 x 2 = 4</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Transport-type: tcp</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Bricks:</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick1: Server1:/exp/br01/brick1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick2: Server2:/exp/br01/brick1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick3: Server1:/exp/br02/brick2</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick4: Server2:/exp/br02/brick2</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Options Reconfigured:</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">diagnostics.brick-log-level: WARNING</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">diagnostics.client-log-level: WARNING</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">cluster.entry-self-heal: off</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">cluster.data-self-heal: off</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">cluster.metadata-self-heal: off</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.cache-size: 1024MB</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.cache-max-file-size: 2MB</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.cache-refresh-timeout: 1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.stat-prefetch: off</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.read-ahead: on</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.quick-read: off</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.write-behind-window-size: 4MB</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.flush-behind: on</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.write-behind: on</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.io-thread-count: 32</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.io-cache: on</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">network.ping-timeout: 2</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">nfs.addr-namelookup: off</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.strict-write-ordering: on</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">[root@Server1 ~]#</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">So we start the Migration of the Brick to the new server using the replace Brick command</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">[root@Server1 ~]# volname=Storage1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">[root@Server1 ~]# from=Server1:/exp/br02/brick2</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">[root@Server1 ~]# to=Server3:/exp/br02/brick2</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">[root@Server1 ~]# gluster volume replace-brick $volname $from $to start</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">All replace-brick commands except commit force are deprecated. Do you want to continue? (y/n) y</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">volume replace-brick: success: replace-brick started successfully</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">ID: 0062d555-e7eb-4ebe-a264-7e0baf6e7546</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">[root@Server1 ~]# gluster volume replace-brick $volname $from $to status</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">All replace-brick commands except commit force are deprecated. Do you want to continue? (y/n) y</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">volume replace-brick: success: Number of files migrated = 281&nbsp;&nbsp; Migration complete</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">At this point everything seems to be in order with no outstanding issues.</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">[root@Server1 ~]# gluster volume status</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Status of volume: Storage1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Gluster process&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Port&nbsp;&nbsp;&nbsp; Online&nbsp; Pid</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">------------------------------------------------------------------------------</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Brick Server1:/exp/br01/brick1&nbsp; 49152&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2167</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Brick Server2:/exp/br01/brick1&nbsp; 49152&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2192</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Brick Server1:/exp/br02/brick2&nbsp; 49153&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 27557</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Brick Server2:/exp/br02/brick2&nbsp; 49153&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2193</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">NFS Server on localhost&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2049&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 27562</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Self-heal Daemon on localhost&nbsp;&nbsp; N/A&nbsp;&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2186</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">NFS Server on Server2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2049&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2205</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Self-heal Daemon on Server2&nbsp;&nbsp;&nbsp;&nbsp; N/A&nbsp;&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2210</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">NFS Server on Server3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2049&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 6015</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Self-heal Daemon on Server3&nbsp;&nbsp;&nbsp;&nbsp; N/A&nbsp;&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 6016</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Task Status of Volume Storage1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">------------------------------------------------------------------------------</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Task&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : Replace brick</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">ID&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : 0062d555-e7eb-4ebe-a264-7e0baf6e7546</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Source Brick&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : Server1:/exp/br02/brick2</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Destination Brick&nbsp;&nbsp;&nbsp; : Server3:/exp/br02/brick2</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Status&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : completed</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">The volume reports that the replace Brick command completed.. so the next step is to commit the change</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">[root@Server1 ~]# gluster volume replace-brick $volname $from $to commit</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">All replace-brick commands except commit force are deprecated. Do you want to continue? (y/n) y</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">volume replace-brick: success: replace-brick commit successful</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">At this point when I take a look at the status I see that the OLD brick is now missing (Server1:/exp/br02/brick2) AND I don’t see the new Brick… WTF… panic!</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">[root@Server1 ~]# gluster volume status</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Status of volume: Storage1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Gluster process&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Port&nbsp;&nbsp;&nbsp; Online&nbsp; Pid</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">------------------------------------------------------------------------------</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Brick Server1:/exp/br01/brick1&nbsp; 49152&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2167</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Brick Server2:/exp/br01/brick1&nbsp; 49152&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2192</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Brick Server2:/exp/br02/brick2&nbsp; 49153&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2193</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">NFS Server on localhost&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2049&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 28906</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Self-heal Daemon on localhost&nbsp;&nbsp; N/A&nbsp;&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 28911</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">NFS Server on Server2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2049&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2205</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Self-heal Daemon on Server2&nbsp;&nbsp;&nbsp;&nbsp; N/A&nbsp;&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2210</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">NFS Server on Server3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2049&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 6015</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Self-heal Daemon on Server3&nbsp;&nbsp;&nbsp;&nbsp; N/A&nbsp;&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 6016</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">Task Status of Volume Storage1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">------------------------------------------------------------------------------</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">There are no active volume tasks</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">After the commit on Server1 it does not have the Tasks listed anymore… yet server2 and server3 see this</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">[root@Server2 ~]# gluster volume status</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Status of volume: Storage1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Gluster process&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Port&nbsp;&nbsp;&nbsp; Online&nbsp; Pid</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">------------------------------------------------------------------------------</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick Server1:/exp/br01/brick1&nbsp; 49152&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2167</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick Server2:/exp/br01/brick1&nbsp; 49152&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2192</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick Server2:/exp/br02/brick2&nbsp; 49153&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2193</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">NFS Server on localhost&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2049&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2205</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Self-heal Daemon on localhost&nbsp;&nbsp; N/A&nbsp;&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2210</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">NFS Server on 10.45.16.17&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2049&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 28906</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Self-heal Daemon on 10.45.16.17 N/A&nbsp;&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 28911</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">NFS Server on server3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2049&nbsp;&nbsp;&nbsp; Y&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 6015</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Self-heal Daemon on server3&nbsp;&nbsp;&nbsp;&nbsp; N/A&nbsp;&nbsp;&nbsp;&nbsp; Y&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;6016</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Task Status of Volume Storage1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">------------------------------------------------------------------------------</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Task&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : Replace brick</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">ID&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : 0062d555-e7eb-4ebe-a264-7e0baf6e7546</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Source Brick&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : Server1:/exp/br02/brick2</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Destination Brick&nbsp;&nbsp;&nbsp; : server3:/exp/br02/brick2</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Status&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : completed</span></p>
<p class="MsoNormal"><span style="font-size:8.0pt; font-family:&quot;Courier New&quot;">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">If I navigate the brick on Server3 the brick is NOT empty.. but missing A LOT!&nbsp; It’s like the replace brick stopped… and never restarted again.</span></p>
<p class="MsoNormal"><span style="color:#002060">The replace brick reported back </span>
“Number of files migrated = 281&nbsp;&nbsp; Migration complete” <span style="color:#002060">
but when I look on Server3 Brick I get:</span></p>
<p class="MsoNormal"><span style="">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; [root@Server3 brick2]# find . -type f -print | wc -l</span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="">16</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">I’m missing 265 files.. (they still exist on the OLD brick.. but how can I move it?)</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">If I try to add the old brick back with another brick on the new server as such</span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="">[root@Server1 ~]# gluster volume add-brick Storage1 Server1:/exp/br02/brick2 Server3:/exp/br01/brick1</span><span style=""></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="">volume add-brick: failed: /exp/br02/brick2 is already part of a volume</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">Im fearfull of running:</span></p>
<p class="MsoNormal"><span style="">[root@Server1 ~]# setfattr -n trusted.glusterfs.volume-id -v 0x$(grep volume-id /var/lib/glusterd/vols/$volname/info | cut -d= -f2 | sed 's/-//g')
</span><span style="">/exp/br02/brick2</span><span style=""></span></p>
<p class="MsoNormal"><span style="color:#002060">Although it should allow me to add the brick</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">Gluster Heal info returns:</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">[root@Server2 ~]# gluster volume heal Storage1 info</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick Server1:/exp/br01/brick1/</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Number of entries: 0</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick Server2:/exp/br01/brick1/</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Number of entries: 0</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick Server1:/exp/br02/brick2</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Status: Transport endpoint is not connected</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick Server2:/exp/br02/brick2/</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Number of entries: 0</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">I have restarted glusterd numerous times.</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">at this time I’m not sure where to go from here… I know that the Server1:/exp/br02/brick2 still has all the data.. and Server3:/exp/br01/brick1 is not complete</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">How do I actually get the brick to replicate?</span></p>
<p class="MsoNormal"><span style="color:#002060">How can I add Server1:/exp/br02/brick2 back into the trusted pool if I can’t replicate it, or re-add it?</span></p>
<p class="MsoNormal"><span style="color:#002060">How can I fix this to get it back into a replicated state between the three servers?</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">Thomas</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">----DATA----</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">Gluster volume info at this point</span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="">[root@Server1 ~]# gluster volume info</span></p>
<p class="MsoNormal"><span style="">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Volume Name: Storage1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Type: Distributed-Replicate</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Volume ID: 9616ce42-48bd-4fe3-883f-decd6c4fcd00</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Status: Started</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Number of Bricks: 2 x 2 = 4</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Transport-type: tcp</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Bricks:</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick1: Server1:/exp/br01/brick1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick2: Server2:/exp/br01/brick1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick3: server3:/exp/br02/brick2</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick4: Server2:/exp/br02/brick2</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Options Reconfigured:</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">diagnostics.brick-log-level: WARNING</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">diagnostics.client-log-level: WARNING</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">cluster.entry-self-heal: off</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">cluster.data-self-heal: off</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">cluster.metadata-self-heal: off</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.cache-size: 1024MB</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.cache-max-file-size: 2MB</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.cache-refresh-timeout: 1</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.stat-prefetch: off</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.read-ahead: on</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.quick-read: off</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.write-behind-window-size: 4MB</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.flush-behind: on</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.write-behind: on</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.io-thread-count: 32</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.io-cache: on</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">network.ping-timeout: 2</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">nfs.addr-namelookup: off</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">performance.strict-write-ordering: on</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">[root@Server1 ~]#</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">[root@server3 brick2]# gluster volume heal Storage1 info</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick Server1:/exp/br01/brick1/</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Number of entries: 0</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick Server2:/exp/br01/brick1/</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Number of entries: 0</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick Server3:/exp/br02/brick2/</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Number of entries: 0</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">&nbsp;</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Brick Server2:/exp/br02/</span><span style="">brick2/</span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="">Number of entries: 0</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="color:#002060">Gluster LOG ( there are a few errors but I’m not sure how to decipher them)</span></p>
<p class="MsoNormal"><span style="color:#002060">&nbsp;</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:29:19.862809] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick Server1:/exp/br02/brick2 has disconnected from glusterd.</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:29:19.862836] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/7565ec897c6454bd3e2f4800250a7221.socket failed (Invalid argument)</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:29:19.862853] I [MSGID: 106006] [glusterd-handler.c:4257:__glusterd_nodesvc_rpc_notify] 0-management: nfs has disconnected from glusterd.</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:29:19.953762] I [glusterd-pmap.c:227:pmap_registry_bind] 0-pmap: adding brick /exp/br02/brick2 on port 49153</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:31:12.977450] I [glusterd-replace-brick.c:99:__glusterd_handle_replace_brick] 0-management: Received replace brick req</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:31:12.977495] I [glusterd-replace-brick.c:154:__glusterd_handle_replace_brick] 0-management: Received replace brick status request</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:31:13.048852] I [glusterd-replace-brick.c:1412:rb_update_srcbrick_port] 0-: adding src-brick port no</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:31:19.588380] I [glusterd-replace-brick.c:99:__glusterd_handle_replace_brick] 0-management: Received replace brick req</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:31:19.588422] I [glusterd-replace-brick.c:154:__glusterd_handle_replace_brick] 0-management: Received replace brick status request</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:31:19.661101] I [glusterd-replace-brick.c:1412:rb_update_srcbrick_port] 0-: adding src-brick port no</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:31:45.115355] W [glusterd-op-sm.c:4021:glusterd_op_modify_op_ctx] 0-management: op_ctx modification failed</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:31:45.118597] I [glusterd-handler.c:3803:__glusterd_handle_status_volume] 0-management: Received status volume req for volume Storage1</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:10.956357] I [glusterd-replace-brick.c:99:__glusterd_handle_replace_brick] 0-management: Received replace brick req</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:10.956385] I [glusterd-replace-brick.c:154:__glusterd_handle_replace_brick] 0-management: Received replace brick commit request</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:11.028472] I [glusterd-replace-brick.c:1412:rb_update_srcbrick_port] 0-: adding src-brick port no</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:12.122552] I [glusterd-utils.c:6276:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV3 successfully</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:12.131836] I [glusterd-utils.c:6281:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV1 successfully</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:12.141107] I [glusterd-utils.c:6286:glusterd_nfs_pmap_deregister] 0-: De-registered NFSV3 successfully</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:12.150375] I [glusterd-utils.c:6291:glusterd_nfs_pmap_deregister] 0-: De-registered NLM v4 successfully</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:12.159630] I [glusterd-utils.c:6296:glusterd_nfs_pmap_deregister] 0-: De-registered NLM v1 successfully</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:12.168889] I [glusterd-utils.c:6301:glusterd_nfs_pmap_deregister] 0-: De-registered ACL v3 successfully</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.254689] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.254799] W [socket.c:2992:socket_connect] 0-management: Ignore failed connection attempt on , (No such file or directory)</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.257790] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.257908] W [socket.c:2992:socket_connect] 0-management: Ignore failed connection attempt on , (No such file or directory)</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.258031] W [socket.c:611:__socket_rwv] 0-socket.management: writev on 127.0.0.1:1019 failed (Broken pipe)</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.258111] W [socket.c:611:__socket_rwv] 0-socket.management: writev on 127.0.0.1:1021 failed (Broken pipe)</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.258130] W [socket.c:611:__socket_rwv] 0-socket.management: writev on 10.45.16.17:1018 failed (Broken pipe)</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.711948] I [mem-pool.c:545:mem_pool_destroy] 0-management: size=588 max=0 total=0</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.711967] I [mem-pool.c:545:mem_pool_destroy] 0-management: size=124 max=0 total=0</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.712008] I [mem-pool.c:545:mem_pool_destroy] 0-management: size=588 max=0 total=0</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.712021] I [mem-pool.c:545:mem_pool_destroy] 0-management: size=124 max=0 total=0</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.731311] I [mem-pool.c:545:mem_pool_destroy] 0-management: size=588 max=0 total=0</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.731326] I [mem-pool.c:545:mem_pool_destroy] 0-management: size=124 max=0 total=0</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.731356] I [glusterd-pmap.c:271:pmap_registry_remove] 0-pmap: removing brick /exp/br02/brick2 on port 49153</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.823129] I [socket.c:2344:socket_event_handler] 0-transport: disconnecting now</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.840668] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/7565ec897c6454bd3e2f4800250a7221.socket failed (Invalid argument)</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.840693] I [MSGID: 106006] [glusterd-handler.c:4257:__glusterd_nodesvc_rpc_notify] 0-management: nfs has disconnected from glusterd.</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.840712] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/ac4c043d3c6a2e5159c86e8c75c51829.socket failed (Invalid argument)</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:13.840728] I [MSGID: 106006] [glusterd-handler.c:4257:__glusterd_nodesvc_rpc_notify] 0-management: glustershd has disconnected from glusterd.</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:14.729667] E [glusterd-rpc-ops.c:1169:__glusterd_commit_op_cbk] 0-management: Received commit RJT from uuid: 294aa603-ec24-44b9-864b-0fe743faa8d9</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:14.743623] E [glusterd-rpc-ops.c:1169:__glusterd_commit_op_cbk] 0-management: Received commit RJT from uuid: 92aabaf4-4b6c-48da-82b6-c465aff2ec6d</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:18.762975] W [glusterd-op-sm.c:4021:glusterd_op_modify_op_ctx] 0-management: op_ctx modification failed</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:18.764552] I [glusterd-handler.c:3803:__glusterd_handle_status_volume] 0-management: Received status volume req for volume Storage1</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:18.769051] E [glusterd-utils.c:9955:glusterd_volume_status_aggregate_tasks_status] 0-management: Local tasks count (0) and remote tasks count (1) do not match. Not aggregating tasks status.</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:18.769070] E [glusterd-syncop.c:961:_gd_syncop_commit_op_cbk] 0-management: Failed to aggregate response from&nbsp; node/brick</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:18.771095] E [glusterd-utils.c:9955:glusterd_volume_status_aggregate_tasks_status] 0-management: Local tasks count (0) and remote tasks count (1) do not match. Not aggregating tasks status.</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:18.771108] E [glusterd-syncop.c:961:_gd_syncop_commit_op_cbk] 0-management: Failed to aggregate response from&nbsp; node/brick</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:48.570796] W [glusterd-op-sm.c:4021:glusterd_op_modify_op_ctx] 0-management: op_ctx modification failed</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:48.572352] I [glusterd-handler.c:3803:__glusterd_handle_status_volume] 0-management: Received status volume req for volume Storage1</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:48.576899] E [glusterd-utils.c:9955:glusterd_volume_status_aggregate_tasks_status] 0-management: Local tasks count (0) and remote tasks count (1) do not match. Not aggregating tasks status.</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:48.576918] E [glusterd-syncop.c:961:_gd_syncop_commit_op_cbk] 0-management: Failed to aggregate response from&nbsp; node/brick</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:48.578982] E [glusterd-utils.c:9955:glusterd_volume_status_aggregate_tasks_status] 0-management: Local tasks count (0) and remote tasks count (1) do not match. Not aggregating tasks status.</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:32:48.579001] E [glusterd-syncop.c:961:_gd_syncop_commit_op_cbk] 0-management: Failed to aggregate response from&nbsp; node/brick</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:36:57.840738] W [glusterd-op-sm.c:4021:glusterd_op_modify_op_ctx] 0-management: op_ctx modification failed</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:36:57.842370] I [glusterd-handler.c:3803:__glusterd_handle_status_volume] 0-management: Received status volume req for volume Storage1</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:36:57.846919] E [glusterd-utils.c:9955:glusterd_volume_status_aggregate_tasks_status] 0-management: Local tasks count (0) and remote tasks count (1) do not match. Not aggregating tasks status.</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:36:57.846941] E [glusterd-syncop.c:961:_gd_syncop_commit_op_cbk] 0-management: Failed to aggregate response from&nbsp; node/brick</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:36:57.849026] E [glusterd-utils.c:9955:glusterd_volume_status_aggregate_tasks_status] 0-management: Local tasks count (0) and remote tasks count (1) do not match. Not aggregating tasks status.</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:36:57.849046] E [glusterd-syncop.c:961:_gd_syncop_commit_op_cbk] 0-management: Failed to aggregate response from&nbsp; node/brick</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:37:20.208081] W [glusterd-op-sm.c:4021:glusterd_op_modify_op_ctx] 0-management: op_ctx modification failed</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:37:20.211279] I [glusterd-handler.c:3803:__glusterd_handle_status_volume] 0-management: Received status volume req for volume Storage1</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:37:20.215792] E [glusterd-utils.c:9955:glusterd_volume_status_aggregate_tasks_status] 0-management: Local tasks count (0) and remote tasks count (1) do not match. Not aggregating tasks status.</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:37:20.215809] E [glusterd-syncop.c:961:_gd_syncop_commit_op_cbk] 0-management: Failed to aggregate response from&nbsp; node/brick</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:37:20.216295] E [glusterd-utils.c:9955:glusterd_volume_status_aggregate_tasks_status] 0-management: Local tasks count (0) and remote tasks count (1) do not match. Not aggregating tasks status.</span></p>
<p class="MsoNormal"><span style="">[2015-02-14 06:37:20.216308] E [glusterd-syncop.c:961:_gd_syncop_commit_op_cbk] 0-management: Failed to aggregate response from&nbsp; node/brick</span></p>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset> <br>
<pre>_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</div>
</body>
</html>