<div dir="ltr">Current run is segfault free(yay!) so far:<div><br></div><div><div>[root@gqas001 ~]# gluster v rebalance testvol status</div><div> Node Rebalanced-files size scanned failures skipped status run time in secs</div><div> --------- ----------- ----------- ----------- ----------- ----------- ------------ --------------</div><div> localhost 893493 54.5GB 2692286 0 0 in progress 38643.00</div><div> <a href="http://gqas013.sbu.lab.eng.bos.redhat.com">gqas013.sbu.lab.eng.bos.redhat.com</a> 0 0Bytes 1 0 0 completed 26070.00</div><div> <a href="http://gqas011.sbu.lab.eng.bos.redhat.com">gqas011.sbu.lab.eng.bos.redhat.com</a> 0 0Bytes 0 0 0 failed 0.00</div><div> <a href="http://gqas014.sbu.lab.eng.bos.redhat.com">gqas014.sbu.lab.eng.bos.redhat.com</a> 0 0Bytes 0 0 0 failed 0.00</div><div> <a href="http://gqas016.sbu.lab.eng.bos.redhat.com">gqas016.sbu.lab.eng.bos.redhat.com</a> 892110 54.4GB 2692295 0 0 in progress 38643.00</div><div> <a href="http://gqas015.sbu.lab.eng.bos.redhat.com">gqas015.sbu.lab.eng.bos.redhat.com</a> 0 0Bytes 0 0 0 failed 0.00</div><div>volume rebalance: testvol: success: </div></div><div><br></div><div>The beseline ran for <span style="font-size:12.8000001907349px">98,500.00 seconds. This one is at </span>38,643.00(1/3 the number of seconds) with 54 GB transferred so far. The same data set last run transferred 81 GB so at 54 GBs we are 66% there. By my estimations we should run for ~10,000-20,000 more seconds which would give us a 40-50% improvement! Lets see how it finishes out :)</div><div><br></div><div>Any idea why I am getting the "failed" for three of them? This has been consistent across each run I have tried.</div><div><br></div><div>-b</div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, May 1, 2015 at 3:05 AM, Ravishankar N <span dir="ltr"><<a href="mailto:ravishankar@redhat.com" target="_blank">ravishankar@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
I sent a <a href="http://review.gluster.org/#/c/10478/" target="_blank">fix</a>
but abandoned it since Susant (CC'ed) has already sent one
<a href="http://review.gluster.org/#/c/10459/" target="_blank">http://review.gluster.org/#/c/10459/</a> <br>
I think it needs re-submission, but more review-eyes are welcome.<br>
-Ravi<div><div class="h5"><br>
<br>
<div>On 05/01/2015 12:18 PM, Benjamin Turner
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">There was a segfault on gqas001, have a look when
you get a sec:
<div><br>
</div>
<div>
<div>Core was generated by `/usr/sbin/glusterfs -s localhost
--volfile-id rebalance/testvol --xlator-option'.</div>
<div>Program terminated with signal 11, Segmentation fault.</div>
<div>#0 gf_defrag_get_entry (this=0x7f26f8011180,
defrag=0x7f26f8031ef0, loc=0x7f26f4dbbfd0,
migrate_data=0x7f2707874be8) at dht-rebalance.c:2032</div>
<div>2032<span style="white-space:pre-wrap"> </span>
GF_FREE (tmp_container->parent_loc);</div>
<div>(gdb) bt</div>
<div>#0 gf_defrag_get_entry (this=0x7f26f8011180,
defrag=0x7f26f8031ef0, loc=0x7f26f4dbbfd0,
migrate_data=0x7f2707874be8) at dht-rebalance.c:2032</div>
<div>#1 gf_defrag_process_dir (this=0x7f26f8011180,
defrag=0x7f26f8031ef0, loc=0x7f26f4dbbfd0,
migrate_data=0x7f2707874be8) at dht-rebalance.c:2207</div>
<div>#2 0x00007f26fdae1eb8 in gf_defrag_fix_layout
(this=0x7f26f8011180, defrag=0x7f26f8031ef0,
loc=0x7f26f4dbbfd0, fix_layout=0x7f2707874b5c,
migrate_data=0x7f2707874be8)</div>
<div> at dht-rebalance.c:2299</div>
<div>#3 0x00007f26fdae1f4b in gf_defrag_fix_layout
(this=0x7f26f8011180, defrag=0x7f26f8031ef0,
loc=0x7f26f4dbc200, fix_layout=0x7f2707874b5c,
migrate_data=0x7f2707874be8)</div>
<div> at dht-rebalance.c:2416</div>
<div>#4 0x00007f26fdae1f4b in gf_defrag_fix_layout
(this=0x7f26f8011180, defrag=0x7f26f8031ef0,
loc=0x7f26f4dbc430, fix_layout=0x7f2707874b5c,
migrate_data=0x7f2707874be8)</div>
<div> at dht-rebalance.c:2416</div>
<div>#5 0x00007f26fdae1f4b in gf_defrag_fix_layout
(this=0x7f26f8011180, defrag=0x7f26f8031ef0,
loc=0x7f26f4dbc660, fix_layout=0x7f2707874b5c,
migrate_data=0x7f2707874be8)</div>
<div> at dht-rebalance.c:2416</div>
<div>#6 0x00007f26fdae1f4b in gf_defrag_fix_layout
(this=0x7f26f8011180, defrag=0x7f26f8031ef0,
loc=0x7f26f4dbc890, fix_layout=0x7f2707874b5c,
migrate_data=0x7f2707874be8)</div>
<div> at dht-rebalance.c:2416</div>
<div>#7 0x00007f26fdae1f4b in gf_defrag_fix_layout
(this=0x7f26f8011180, defrag=0x7f26f8031ef0,
loc=0x7f26f4dbcac0, fix_layout=0x7f2707874b5c,
migrate_data=0x7f2707874be8)</div>
<div> at dht-rebalance.c:2416</div>
<div>#8 0x00007f26fdae1f4b in gf_defrag_fix_layout
(this=0x7f26f8011180, defrag=0x7f26f8031ef0,
loc=0x7f26f4dbccf0, fix_layout=0x7f2707874b5c,
migrate_data=0x7f2707874be8)</div>
<div> at dht-rebalance.c:2416</div>
<div>#9 0x00007f26fdae1f4b in gf_defrag_fix_layout
(this=0x7f26f8011180, defrag=0x7f26f8031ef0,
loc=0x7f26f4dbcf60, fix_layout=0x7f2707874b5c,
migrate_data=0x7f2707874be8)</div>
<div> at dht-rebalance.c:2416</div>
<div>#10 0x00007f26fdae2524 in gf_defrag_start_crawl
(data=0x7f26f8011180) at dht-rebalance.c:2599</div>
<div>#11 0x00007f2709024f62 in synctask_wrap
(old_task=<value optimized out>) at syncop.c:375</div>
<div>#12 0x0000003648c438f0 in ?? () from /lib64/<a href="http://libc-2.12.so" target="_blank">libc-2.12.so</a></div>
<div>#13 0x0000000000000000 in ?? ()</div>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, May 1, 2015 at 12:53 AM,
Benjamin Turner <span dir="ltr"><<a href="mailto:bennyturns@gmail.com" target="_blank">bennyturns@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Ok I have all my data created and I just
started the rebalance. One thing to not in the client log
I see the following spamming:
<div><br>
</div>
<div>
<div>[root@gqac006 ~]# cat
/var/log/glusterfs/gluster-mount-.log | wc -l</div>
<div>394042</div>
</div>
<div><br>
</div>
<div>
<div>[2015-05-01 00:47:55.591150] I [MSGID: 109036]
[dht-common.c:6478:dht_log_new_layout_for_dir_selfheal]
0-testvol-dht: Setting layout of /file_dstdir/<a href="http://gqac006.sbu.lab.eng.bos.redhat.com/thrd_05/d_001/d_000/d_004/d_006" target="_blank">gqac006.sbu.lab.eng.bos.redhat.com/thrd_05/d_001/d_000/d_004/d_006</a>
with [Subvol_name: testvol-replicate-0, Err: -1 ,
Start: 0 , Stop: 2141429669 ], [Subvol_name:
testvol-replicate-1, Err: -1 , Start: 2141429670 ,
Stop: 4294967295 ], </div>
<div>[2015-05-01 00:47:55.596147] I
[dht-selfheal.c:1587:dht_selfheal_layout_new_directory]
0-testvol-dht: chunk size = 0xffffffff / 19920276 =
0xd7</div>
<div>[2015-05-01 00:47:55.596177] I
[dht-selfheal.c:1626:dht_selfheal_layout_new_directory]
0-testvol-dht: assigning range size 0x7fa39fa6 to
testvol-replicate-1</div>
<div>[2015-05-01 00:47:55.596189] I
[dht-selfheal.c:1626:dht_selfheal_layout_new_directory]
0-testvol-dht: assigning range size 0x7fa39fa6 to
testvol-replicate-0</div>
<div>[2015-05-01 00:47:55.597081] I [MSGID: 109036]
[dht-common.c:6478:dht_log_new_layout_for_dir_selfheal]
0-testvol-dht: Setting layout of /file_dstdir/<a href="http://gqac006.sbu.lab.eng.bos.redhat.com/thrd_05/d_001/d_000/d_004/d_005" target="_blank">gqac006.sbu.lab.eng.bos.redhat.com/thrd_05/d_001/d_000/d_004/d_005</a>
with [Subvol_name: testvol-replicate-0, Err: -1 ,
Start: 2141429670 , Stop: 4294967295 ], [Subvol_name:
testvol-replicate-1, Err: -1 , Start: 0 , Stop:
2141429669 ], </div>
<div>[2015-05-01 00:47:55.601853] I
[dht-selfheal.c:1587:dht_selfheal_layout_new_directory]
0-testvol-dht: chunk size = 0xffffffff / 19920276 =
0xd7</div>
<div>[2015-05-01 00:47:55.601882] I
[dht-selfheal.c:1626:dht_selfheal_layout_new_directory]
0-testvol-dht: assigning range size 0x7fa39fa6 to
testvol-replicate-1</div>
<div>[2015-05-01 00:47:55.601895] I
[dht-selfheal.c:1626:dht_selfheal_layout_new_directory]
0-testvol-dht: assigning range size 0x7fa39fa6 to
testvol-replicate-0</div>
<div><br>
</div>
</div>
<div>Just to confirm the patch is
in, glusterfs-3.8dev-0.71.gita7f8482.el6.x86_64.
Correct?</div>
<div><br>
</div>
<div>Here is the info on the data set:</div>
<div><br>
</div>
<div>
<div>hosts in test : ['<a href="http://gqac006.sbu.lab.eng.bos.redhat.com" target="_blank">gqac006.sbu.lab.eng.bos.redhat.com</a>',
'<a href="http://gqas003.sbu.lab.eng.bos.redhat.com" target="_blank">gqas003.sbu.lab.eng.bos.redhat.com</a>']</div>
<div>top test directory(s) : ['/gluster-mount']</div>
<div>peration : create</div>
<div>files/thread : 500000</div>
<div>threads : 8</div>
<div>record size (KB, 0 = maximum) : 0</div>
<div>file size (KB) : 64</div>
<div>file size distribution : fixed</div>
<div>files per dir : 100</div>
<div>dirs per dir : 10</div>
</div>
<div>
<div>total threads = 16</div>
<div>total files = 7222600</div>
<div>total data = 440.833 GB</div>
<div> 90.28% of requested files processed, minimum is
70.00</div>
<div>8107.852862 sec elapsed time</div>
<div>890.815377 files/sec</div>
<div>890.815377 IOPS</div>
<div>55.675961 MB/sec</div>
</div>
<div><br>
</div>
<div>Here is the rebalance run after about 5 or so
minutes:</div>
<span>
<div><br>
</div>
<div>[root@gqas001 ~]# gluster v rebalance testvol
status</div>
<div> Node
Rebalanced-files size scanned
failures skipped status run
time in secs</div>
<div> ---------
----------- ----------- ----------- -----------
----------- ------------ --------------</div>
</span>
<div> localhost
32203 2.0GB 120858 0
5184 in progress 1294.00</div>
<div> <a href="http://gqas011.sbu.lab.eng.bos.redhat.com" target="_blank">gqas011.sbu.lab.eng.bos.redhat.com</a>
0 0Bytes 0
0 0 failed 0.00</div>
<div> <a href="http://gqas016.sbu.lab.eng.bos.redhat.com" target="_blank">gqas016.sbu.lab.eng.bos.redhat.com</a>
9364 585.2MB 53121
0 0 in progress 1294.00</div>
<div> <a href="http://gqas013.sbu.lab.eng.bos.redhat.com" target="_blank">gqas013.sbu.lab.eng.bos.redhat.com</a>
0 0Bytes 14750
0 0 in progress 1294.00</div>
<div> <a href="http://gqas014.sbu.lab.eng.bos.redhat.com" target="_blank">gqas014.sbu.lab.eng.bos.redhat.com</a>
0 0Bytes 0
0 0 failed 0.00</div>
<div> <a href="http://gqas015.sbu.lab.eng.bos.redhat.com" target="_blank">gqas015.sbu.lab.eng.bos.redhat.com</a>
0 0Bytes 196382
0 0 in progress 1294.00</div>
<div>volume rebalance: testvol: success: </div>
<div><br>
</div>
<div>The hostnames are there if you want to poke around.
I had a problem with one of the added systems being on a
different version of glusterfs so I had to update
everything to
glusterfs-3.8dev-0.99.git7d7b80e.el6.x86_64, remove the
bricks I just added, and add them back. Something may
have went wrong in that process but I thought I did
everything correctly. I'll start fresh tomorrow. I
figured I'd let this run over night.<span><font color="#888888"><br>
</font></span></div>
<span><font color="#888888">
<div><br>
</div>
<div>-b</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</font></span></div>
<div>
<div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Apr 29, 2015 at 9:48
PM, Benjamin Turner <span dir="ltr"><<a href="mailto:bennyturns@gmail.com" target="_blank">bennyturns@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Sweet! Here is the baseline:
<div><br>
</div>
<div>
<div>[root@gqas001 ~]# gluster v rebalance
testvol status</div>
<span>
<div>
Node Rebalanced-files size
scanned failures skipped
status run time in secs</div>
<div>
--------- ----------- -----------
----------- ----------- -----------
------------ --------------</div>
</span>
<div> localhost
1328575 81.1GB 9402953
0 0
completed 98500.00</div>
<div> <a href="http://gqas012.sbu.lab.eng.bos.redhat.com" target="_blank">gqas012.sbu.lab.eng.bos.redhat.com</a>
0 0Bytes 8000011
0 0
completed 51982.00</div>
<div> <a href="http://gqas003.sbu.lab.eng.bos.redhat.com" target="_blank">gqas003.sbu.lab.eng.bos.redhat.com</a>
0 0Bytes 8000011
0 0
completed 51982.00</div>
<div> <a href="http://gqas004.sbu.lab.eng.bos.redhat.com" target="_blank">gqas004.sbu.lab.eng.bos.redhat.com</a>
1326290 81.0GB 9708625
0 0
completed 98500.00</div>
<div> <a href="http://gqas013.sbu.lab.eng.bos.redhat.com" target="_blank">gqas013.sbu.lab.eng.bos.redhat.com</a>
0 0Bytes 8000011
0 0
completed 51982.00</div>
<div> <a href="http://gqas014.sbu.lab.eng.bos.redhat.com" target="_blank">gqas014.sbu.lab.eng.bos.redhat.com</a>
0 0Bytes 8000011
0 0
completed 51982.00</div>
<div>volume rebalance: testvol: success: </div>
</div>
<div><br>
</div>
<div>I'll have a run on the patch started
tomorrow.</div>
<span><font color="#888888">
<div><br>
</div>
<div>-b</div>
</font></span></div>
<div>
<div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Apr 29,
2015 at 12:51 PM, Nithya Balachandran <span dir="ltr"><<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span><br>
Doh my mistake, I thought it was
merged. I was just running with the<br>
upstream 3.7 daily. Can I use this
run as my baseline and then I can run<br>
next time on the patch to show the %
improvement? I'll wipe everything and<br>
try on the patch, any idea when it
will be merged?<br>
<br>
</span>Yes, it would be very useful to
have this run as the baseline. The patch
has just been merged in master. It
should be backported to 3.7 in a day or
so.<br>
<br>
Regards,<br>
Nithya<br>
<div>
<div><br>
<br>
> > > ><br>
> > > > ><br>
> > > > > On Wed, Apr
22, 2015 at 1:10 AM, Nithya
Balachandran<br>
> > > > > <<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>><br>
> > > > > wrote:<br>
> > > > ><br>
> > > > > > That
sounds great. Thanks.<br>
> > > > > ><br>
> > > > > >
Regards,<br>
> > > > > > Nithya<br>
> > > > > ><br>
> > > > > > -----
Original Message -----<br>
> > > > > > From:
"Benjamin Turner" <<a href="mailto:bennyturns@gmail.com" target="_blank">bennyturns@gmail.com</a>><br>
> > > > > > To:
"Nithya Balachandran" <<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>><br>
> > > > > > Cc:
"Susant Palai" <<a href="mailto:spalai@redhat.com" target="_blank">spalai@redhat.com</a>>,
"Gluster Devel" <<br>
> > > > > > <a href="mailto:gluster-devel@gluster.org" target="_blank">gluster-devel@gluster.org</a>><br>
> > > > > > Sent:
Wednesday, 22 April, 2015 12:14:14
AM<br>
> > > > > >
Subject: Re: [Gluster-devel]
Rebalance improvement design<br>
> > > > > ><br>
> > > > > > I am
setting up a test env now, I'll have
some feedback for you<br>
> this<br>
> > > > > > week.<br>
> > > > > ><br>
> > > > > > -b<br>
> > > > > ><br>
> > > > > > On
Tue, Apr 21, 2015 at 11:36 AM,
Nithya Balachandran<br>
> > > > > > <<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a><br>
> > > > > > ><br>
> > > > > > wrote:<br>
> > > > > ><br>
> > > > > > >
Hi Ben,<br>
> > > > > > ><br>
> > > > > > >
Did you get a chance to try this
out?<br>
> > > > > > ><br>
> > > > > > >
Regards,<br>
> > > > > > >
Nithya<br>
> > > > > > ><br>
> > > > > > >
----- Original Message -----<br>
> > > > > > >
From: "Susant Palai" <<a href="mailto:spalai@redhat.com" target="_blank">spalai@redhat.com</a>><br>
> > > > > > >
To: "Benjamin Turner" <<a href="mailto:bennyturns@gmail.com" target="_blank">bennyturns@gmail.com</a>><br>
> > > > > > >
Cc: "Gluster Devel" <<a href="mailto:gluster-devel@gluster.org" target="_blank">gluster-devel@gluster.org</a>><br>
> > > > > > >
Sent: Monday, April 13, 2015 9:55:07
AM<br>
> > > > > > >
Subject: Re: [Gluster-devel]
Rebalance improvement design<br>
> > > > > > ><br>
> > > > > > >
Hi Ben,<br>
> > > > > > >
Uploaded a new patch here:<br>
> <a href="http://review.gluster.org/#/c/9657/" target="_blank">http://review.gluster.org/#/c/9657/</a>.<br>
> > > > > > >
We<br>
> > > > > > >
can<br>
> > > > > > >
start perf test on it. :)<br>
> > > > > > ><br>
> > > > > > >
Susant<br>
> > > > > > ><br>
> > > > > > >
----- Original Message -----<br>
> > > > > > >
From: "Susant Palai" <<a href="mailto:spalai@redhat.com" target="_blank">spalai@redhat.com</a>><br>
> > > > > > >
To: "Benjamin Turner" <<a href="mailto:bennyturns@gmail.com" target="_blank">bennyturns@gmail.com</a>><br>
> > > > > > >
Cc: "Gluster Devel" <<a href="mailto:gluster-devel@gluster.org" target="_blank">gluster-devel@gluster.org</a>><br>
> > > > > > >
Sent: Thursday, 9 April, 2015
3:40:09 PM<br>
> > > > > > >
Subject: Re: [Gluster-devel]
Rebalance improvement design<br>
> > > > > > ><br>
> > > > > > >
Thanks Ben. RPM is not available and
I am planning to refresh<br>
> the<br>
> > > > > > >
patch<br>
> > > > > > in<br>
> > > > > > >
two days with some more regression
fixes. I think we can run<br>
> the<br>
> > > > > > >
tests<br>
> > > > > > post<br>
> > > > > > >
that. Any larger data-set will be
good(say 3 to 5 TB).<br>
> > > > > > ><br>
> > > > > > >
Thanks,<br>
> > > > > > >
Susant<br>
> > > > > > ><br>
> > > > > > >
----- Original Message -----<br>
> > > > > > >
From: "Benjamin Turner" <<a href="mailto:bennyturns@gmail.com" target="_blank">bennyturns@gmail.com</a>><br>
> > > > > > >
To: "Vijay Bellur" <<a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>><br>
> > > > > > >
Cc: "Susant Palai" <<a href="mailto:spalai@redhat.com" target="_blank">spalai@redhat.com</a>>,
"Gluster Devel" <<br>
> > > > > > > <a href="mailto:gluster-devel@gluster.org" target="_blank">gluster-devel@gluster.org</a>><br>
> > > > > > >
Sent: Thursday, 9 April, 2015
2:10:30 AM<br>
> > > > > > >
Subject: Re: [Gluster-devel]
Rebalance improvement design<br>
> > > > > > ><br>
> > > > > > ><br>
> > > > > > > I
have some rebalance perf regression
stuff I have been<br>
> working on,<br>
> > > > > > >
is<br>
> > > > > > >
there an RPM with these patches
anywhere so that I can try it<br>
> on my<br>
> > > > > > >
systems? If not I'll just build
from:<br>
> > > > > > ><br>
> > > > > > ><br>
> > > > > > >
git fetch git:// <a href="http://review.gluster.org/glusterfs" target="_blank">review.gluster.org/glusterfs</a><br>
> > > > > > >
refs/changes/57/9657/8<br>
> > > > > > >
&&<br>
> > > > > > >
git cherry-pick FETCH_HEAD<br>
> > > > > > ><br>
> > > > > > ><br>
> > > > > > ><br>
> > > > > > > I
will have _at_least_ 10TB of
storage, how many TBs of data<br>
> should<br>
> > > > > > > I<br>
> > > > > > >
run<br>
> > > > > > >
with?<br>
> > > > > > ><br>
> > > > > > ><br>
> > > > > > >
-b<br>
> > > > > > ><br>
> > > > > > ><br>
> > > > > > >
On Tue, Apr 7, 2015 at 9:07 AM,
Vijay Bellur <<br>
> <a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>
><br>
> > > > > > wrote:<br>
> > > > > > ><br>
> > > > > > ><br>
> > > > > > ><br>
> > > > > > ><br>
> > > > > > >
On 04/07/2015 03:08 PM, Susant Palai
wrote:<br>
> > > > > > ><br>
> > > > > > ><br>
> > > > > > >
Here is one test performed on a
300GB data set and around<br>
> 100%(1/2<br>
> > > > > > >
the<br>
> > > > > > >
time) improvement was seen.<br>
> > > > > > ><br>
> > > > > > >
[root@gprfs031 ~]# gluster v i<br>
> > > > > > ><br>
> > > > > > >
Volume Name: rbperf<br>
> > > > > > >
Type: Distribute<br>
> > > > > > >
Volume ID: 35562662-337e-4923-b862-
d0bbb0748003<br>
> > > > > > >
Status: Started<br>
> > > > > > >
Number of Bricks: 4<br>
> > > > > > >
Transport-type: tcp<br>
> > > > > > >
Bricks:<br>
> > > > > > >
Brick1: gprfs029-10ge:/bricks/
gprfs029/brick1<br>
> > > > > > >
Brick2: gprfs030-10ge:/bricks/
gprfs030/brick1<br>
> > > > > > >
Brick3: gprfs031-10ge:/bricks/
gprfs031/brick1<br>
> > > > > > >
Brick4: gprfs032-10ge:/bricks/
gprfs032/brick1<br>
> > > > > > ><br>
> > > > > > ><br>
> > > > > > >
Added server 32 and started
rebalance force.<br>
> > > > > > ><br>
> > > > > > >
Rebalance stat for new changes:<br>
> > > > > > >
[root@gprfs031 ~]# gluster v
rebalance rbperf status<br>
> > > > > > >
Node Rebalanced-files size scanned
failures skipped status run<br>
> time<br>
> > > > > > >
in<br>
> > > > > > secs<br>
> > > > > > >
--------- ----------- -----------
----------- -----------<br>
> > > > > > >
-----------<br>
> > > > > > >
------------ --------------<br>
> > > > > > >
localhost 74639 36.1GB 297319 0 0
completed 1743.00<br>
> > > > > > >
172.17.40.30 67512 33.5GB 269187 0 0
completed 1395.00<br>
> > > > > > >
gprfs029-10ge 79095 38.8GB 284105 0
0 completed 1559.00<br>
> > > > > > >
gprfs032-10ge 0 0Bytes 0 0 0
completed 402.00<br>
> > > > > > >
volume rebalance: rbperf: success:<br>
> > > > > > ><br>
> > > > > > >
Rebalance stat for old model:<br>
> > > > > > >
[root@gprfs031 ~]# gluster v
rebalance rbperf status<br>
> > > > > > >
Node Rebalanced-files size scanned
failures skipped status run<br>
> time<br>
> > > > > > >
in<br>
> > > > > > secs<br>
> > > > > > >
--------- ----------- -----------
----------- -----------<br>
> > > > > > >
-----------<br>
> > > > > > >
------------ --------------<br>
> > > > > > >
localhost 86493 42.0GB 634302 0 0
completed 3329.00<br>
> > > > > > >
gprfs029-10ge 94115 46.2GB 687852 0
0 completed 3328.00<br>
> > > > > > >
gprfs030-10ge 74314 35.9GB 651943 0
0 completed 3072.00<br>
> > > > > > >
gprfs032-10ge 0 0Bytes 594166 0 0
completed 1943.00<br>
> > > > > > >
volume rebalance: rbperf: success:<br>
> > > > > > ><br>
> > > > > > ><br>
> > > > > > >
This is interesting. Thanks for
sharing & well done! Maybe we<br>
> > > > > > >
should<br>
> > > > > > >
attempt a much larger data set and
see how we fare there :).<br>
> > > > > > ><br>
> > > > > > >
Regards,<br>
> > > > > > ><br>
> > > > > > ><br>
> > > > > > >
Vijay<br>
> > > > > > ><br>
> > > > > > ><br>
> > > > > > >
______________________________
_________________<br>
> > > > > > >
Gluster-devel mailing list<br>
> > > > > > > <a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
> > > > > > > <a href="http://www.gluster.org/" target="_blank">http://www.gluster.org/</a>
mailman/listinfo/gluster-devel<br>
> > > > > > ><br>
> > > > > > >
_______________________________________________<br>
> > > > > > >
Gluster-devel mailing list<br>
> > > > > > > <a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
> > > > > > > <a href="http://www.gluster.org/mailman/listinfo/gluster-devel" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
> > > > > > >
_______________________________________________<br>
> > > > > > >
Gluster-devel mailing list<br>
> > > > > > > <a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
> > > > > > > <a href="http://www.gluster.org/mailman/listinfo/gluster-devel" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
> > > > > > ><br>
> > > > > ><br>
> > > > ><br>
> > > >
_______________________________________________<br>
> > > > Gluster-devel
mailing list<br>
> > > > <a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
> > > > <a href="http://www.gluster.org/mailman/listinfo/gluster-devel" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
> > > ><br>
> > >
_______________________________________________<br>
> > > Gluster-devel mailing
list<br>
> > > <a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
> > > <a href="http://www.gluster.org/mailman/listinfo/gluster-devel" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
> > ><br>
> ><br>
><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Gluster-devel mailing list
<a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a>
<a href="http://www.gluster.org/mailman/listinfo/gluster-devel" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-devel</a>
</pre>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br></div>