<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
Hi Ravi,</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
Thanks for the clarification. For one reason or another I thought there was no longer a way to shrink a gluster filesystem but it appears from testing you are correct. I generated 100 test files on a 2node/4 ebs brick distributed gluster volume, removed one
brick, committed and ended up having those files successfully migrated to the remaining 3 bricks. </div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<div>Volume Name: volume1</div>
<div>Type: Distribute</div>
<div>Volume ID: 1180f5ec-af82-4322-8fc2-1add01161442</div>
<div>Status: Started</div>
<div>Number of Bricks: 4</div>
<div>Transport-type: tcp</div>
<div>Bricks:</div>
<div>Brick1: gluster1.local:/export/xvdf/brick</div>
<div>Brick2: gluster1.local:/export/xvdg/brick</div>
<div>Brick3: gluster2.local:/export/xvdf/brick</div>
<div>Brick4: gluster2.local:/export/xvdg/brick</div>
<div>[root@ip-172-31-19-77 ~]# </div>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<span style="font-weight: bold;">Gluster1</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<div>[root@ip-172-31-19-77 ~]# ls /export/xvdf/brick/</div>
<div>1 13 24 26 33 47 48 5 51 53 54 58 61 62 64 65 70 72 73 8 9 91 94 96 97 98</div>
<div>[root@ip-172-31-19-77 ~]# ls /export/xvdg/brick/</div>
<div>11 18 19 20 23 27 28 32 35 38 44 46 49 63 68 69 7 78 79 82 85 86 88 93 99</div>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<div>[root@ip-172-31-19-77 ~]# ls /export/xvdf/brick/ | wc -l</div>
<div>26</div>
<div>[root@ip-172-31-19-77 ~]# ls /export/xvdg/brick/ | wc -l</div>
<div>25</div>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<b>Gluster2</b></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<div>[ec2-user@ip-172-31-19-78 ~]$ ls /export/xvdf/brick/</div>
<div>16 22 29 30 34 37 4 41 43 56 57 60 71 74 75 76 80 83 89 90 92</div>
<div>[ec2-user@ip-172-31-19-78 ~]$ ls /export/xvdg/brick/</div>
<div>10 12 15 2 25 31 39 42 50 55 6 67 81 87</div>
<div>100 14 17 21 3 36 40 45 52 59 66 77 84 95</div>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<div>[ec2-user@ip-172-31-19-78 ~]$ ls /export/xvdg/brick/ | wc -l</div>
<div>28</div>
<div>[ec2-user@ip-172-31-19-78 ~]$ ls /export/xvdf/brick/ | wc -l</div>
<div>21</div>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<b>Remove Brick Operation</b></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div>
<div><font face="Calibri,sans-serif">[root@ip-172-31-19-77 ~]# gluster volume remove-brick volume1 gluster1.local:/export/xvdf/brick start</font></div>
<div><font face="Calibri,sans-serif">volume remove-brick start: success</font></div>
<div><font face="Calibri,sans-serif">ID: b0a6decd-9130-4773-97f9-af108ff0bd77</font></div>
<div><font face="Calibri,sans-serif">[root@ip-172-31-19-77 ~]# gluster volume remove-brick volume1 gluster1.local:/export/xvdf/brick status</font></div>
<div><font face="Calibri,sans-serif"> Node Rebalanced-files size scanned failures skipped status run time in secs</font></div>
<div><font face="Calibri,sans-serif"> --------- ----------- ----------- ----------- ----------- ----------- ------------ --------------</font></div>
<div><font face="Calibri,sans-serif"> localhost 41 0Bytes 141 0 0 completed 1.00</font></div>
<div><font face="Calibri,sans-serif"><br>
</font></div>
<div><font face="Calibri,sans-serif">[root@ip-172-31-19-77 ~]# gluster volume remove-brick volume1 gluster1.local:/export/xvdf/brick commit </font></div>
<div><font face="Calibri,sans-serif">Removing brick(s) can result in data loss. Do you want to Continue? (y/n) y</font></div>
<div><font face="Calibri,sans-serif">volume remove-brick commit: success</font></div>
</div>
<div><font face="Calibri,sans-serif"><br>
</font></div>
<div><font face="Calibri,sans-serif">
<div>[root@ip-172-31-19-77 ~]# gluster volume info volume1</div>
<div> </div>
<div>Volume Name: volume1</div>
<div>Type: Distribute</div>
<div>Volume ID: 1180f5ec-af82-4322-8fc2-1add01161442</div>
<div>Status: Started</div>
<div>Number of Bricks: 3</div>
<div>Transport-type: tcp</div>
<div>Bricks:</div>
<div>Brick1: gluster1.local:/export/xvdg/brick</div>
<div>Brick2: gluster2.local:/export/xvdf/brick</div>
<div>Brick3: gluster2.local:/export/xvdg/brick</div>
</font></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
I checked the brick mount points and the overall gluster mount point and found that it did indeed migrate the data from the removed brick to the others and all files were accounted for. </div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
I checked the most recent documentation on github here <a href="https://github.com/gluster/glusterfs/blob/master/doc/admin-guide/en-US/markdown/admin_managing_volumes.md#shrinking-volumes">https://github.com/gluster/glusterfs/blob/master/doc/admin-guide/en-US/markdown/admin_managing_volumes.md#shrinking-volumes</a> and
found only examples for 'remove-brick force' (which obviously doesn't migrate any data) and 'replace-brick' (which is now deprecated but isn't noted as such in the docs). I'm left wondering why they don't have a non-forceful remove-brick operation documented
for people that simply want to remove a brick while migrating data to remaining bricks. </div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
Am I missing something or this common use case undocumented?</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
John</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<div id="MAC_OUTLOOK_SIGNATURE"></div>
</div>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<span id="OLK_SRC_BODY_SECTION" style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<div style="font-family:Calibri; font-size:12pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Ravishankar N<br>
<span style="font-weight:bold">Date: </span>Wednesday, April 1, 2015 at 9:38 AM<br>
<span style="font-weight:bold">To: </span>John Lilley, "<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>"<br>
<span style="font-weight:bold">Subject: </span>Re: [Gluster-users] Shrinking gluster filesystem in 3.6.2<br>
</div>
<div><br>
</div>
<div>
<div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<div class="moz-cite-prefix">On 03/31/2015 05:12 AM, Lilley, John F. wrote:<br>
</div>
<blockquote cite="mid:CDF0DE10-8DD3-41AD-99E8-8929417F557C@caltech.edu" type="cite">
<div>Hi,</div>
<div><br>
</div>
<div>I'd like to shrink my aws/ebs-based *distribute-only* gluster file system by migrating the data to other already existing, active and partially utilized bricks but found the 'replace-brick start' mentioned in the documentation is now deprecated. I see
that there has been some back and forth on the mailing list regarding migrating data using self-heal on a replicated system but not so much on a distribute-only file system. Can anyone tell me the blessed way of doing this in 3.6.2? Is there one?</div>
<div><br>
</div>
<div>To be clear, all of the ebs-based bricks are partially utilized at this point so I'd need a method to migrate the data first.</div>
<div><br>
</div>
</blockquote>
<br>
If I understand you correctly, you want to replace a brick in a distribute volume with one of lesser capacity. You could first add a new brick and then remove the existing brick with remove-brick start/status/commit sequence. Something like this:<br>
------------------------------------------------<br>
<tt>[root@tuxpad ~]# gluster volume info testvol</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Volume Name: testvol</tt><tt><br>
</tt><tt>Type: Distribute</tt><tt><br>
</tt><tt>Volume ID: a89aa154-885c-4e14-8d3a-b555733b11f1</tt><tt><br>
</tt><tt>Status: Started</tt><tt><br>
</tt><tt>Number of Bricks: 3</tt><tt><br>
</tt><tt>Transport-type: tcp</tt><tt><br>
</tt><tt>Bricks:</tt><tt><br>
</tt><tt>Brick1: 127.0.0.2:/home/ravi/bricks/brick1</tt><tt><br>
</tt><tt>Brick2: 127.0.0.2:/home/ravi/bricks/brick2</tt><tt><br>
</tt><tt>Brick3: 127.0.0.2:/home/ravi/bricks/brick3</tt><tt><br>
</tt><tt>[root@tuxpad ~]# </tt><tt><br>
</tt><tt>[root@tuxpad ~]# </tt><tt><br>
</tt><tt>[root@tuxpad ~]# gluster volume add-brick testvol 127.0.0.2:/home/ravi/bricks/brick{4..6}</tt><tt><br>
</tt><tt>volume add-brick: success</tt><tt><br>
</tt><tt>[root@tuxpad ~]# </tt><tt><br>
</tt><tt>[root@tuxpad ~]# gluster volume info testvol</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Volume Name: testvol</tt><tt><br>
</tt><tt>Type: Distribute</tt><tt><br>
</tt><tt>Volume ID: a89aa154-885c-4e14-8d3a-b555733b11f1</tt><tt><br>
</tt><tt>Status: Started</tt><tt><br>
</tt><tt>Number of Bricks: 6</tt><tt><br>
</tt><tt>Transport-type: tcp</tt><tt><br>
</tt><tt>Bricks:</tt><tt><br>
</tt><tt>Brick1: 127.0.0.2:/home/ravi/bricks/brick1</tt><tt><br>
</tt><tt>Brick2: 127.0.0.2:/home/ravi/bricks/brick2</tt><tt><br>
</tt><tt>Brick3: 127.0.0.2:/home/ravi/bricks/brick3</tt><tt><br>
</tt><tt>Brick4: 127.0.0.2:/home/ravi/bricks/brick4</tt><tt><br>
</tt><tt>Brick5: 127.0.0.2:/home/ravi/bricks/brick5</tt><tt><br>
</tt><tt>Brick6: 127.0.0.2:/home/ravi/bricks/brick6</tt><tt><br>
</tt><tt>[root@tuxpad ~]# </tt><tt><br>
</tt><tt>[root@tuxpad ~]# </tt><tt><br>
</tt><tt>[root@tuxpad ~]# </tt><tt><br>
</tt><tt>[root@tuxpad ~]# gluster v remove-brick testvol 127.0.0.2:/home/ravi/bricks/brick{1..3} start</tt><tt><br>
</tt><tt>volume remove-brick start: success</tt><tt><br>
</tt><tt>ID: d535675e-8362-4a44-a291-1e567a77531e</tt><tt><br>
</tt><tt>[root@tuxpad ~]# gluster v remove-brick testvol 127.0.0.2:/home/ravi/bricks/brick{1..3} status</tt><tt><br>
</tt><tt> Node Rebalanced-files size scanned failures skipped status run time in secs</tt><tt><br>
</tt><tt> --------- ----------- ----------- ----------- ----------- ----------- ------------ --------------</tt><tt><br>
</tt><tt> localhost 10 0Bytes 20 0 0 <b> completed </b> 0.00</tt><tt><br>
</tt><tt>[root@tuxpad ~]# </tt><tt><br>
</tt><tt>[root@tuxpad ~]# gluster v remove-brick testvol 127.0.0.2:/home/ravi/bricks/brick{1..3} commit</tt><tt><br>
</tt><tt>Removing brick(s) can result in data loss. Do you want to Continue? (y/n) y</tt><tt><br>
</tt><tt>volume remove-brick commit: success</tt><tt><br>
</tt><b><tt>Check the removed bricks to ensure all files are migrated.</tt></b><b><tt><br>
</tt></b><b><tt>If files with data are found on the brick path, copy them via a gluster mount point before re-purposing the removed brick.
</tt></b><tt><br>
</tt><tt>[root@tuxpad ~]# </tt><tt><br>
</tt><tt>[root@tuxpad ~]# gluster volume info testvol</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Volume Name: testvol</tt><tt><br>
</tt><tt>Type: Distribute</tt><tt><br>
</tt><tt>Volume ID: a89aa154-885c-4e14-8d3a-b555733b11f1</tt><tt><br>
</tt><tt>Status: Started</tt><tt><br>
</tt><tt>Number of Bricks: 3</tt><tt><br>
</tt><tt>Transport-type: tcp</tt><tt><br>
</tt><tt>Bricks:</tt><tt><br>
</tt><tt>Brick1: 127.0.0.2:/home/ravi/bricks/brick4</tt><tt><br>
</tt><tt>Brick2: 127.0.0.2:/home/ravi/bricks/brick5</tt><tt><br>
</tt><tt>Brick3: 127.0.0.2:/home/ravi/bricks/brick6</tt><tt><br>
</tt><tt>[root@tuxpad ~]# </tt><tt><br>
</tt>------------------------------------------------<br>
Hope this helps.<br>
Ravi<br>
<br>
<blockquote cite="mid:CDF0DE10-8DD3-41AD-99E8-8929417F557C@caltech.edu" type="cite">
<div></div>
<div>Thank You,</div>
<div>John</div>
<div></div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset> <br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</div>
</div>
</span>
</body>
</html>