<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<br>
<div class="moz-cite-prefix">On 10/26/2016 02:12 PM, Gandalf
Corvotempesta wrote:<br>
</div>
<blockquote
cite="mid:CAJH6TXhf_yUXs91Fk9Vd=cjgsdpUvO00oCgTc94GbzkczTdSQw@mail.gmail.com"
type="cite">
<pre wrap="">2016-10-26 23:07 GMT+02:00 Joe Julian <a class="moz-txt-link-rfc2396E" href="mailto:joe@julianfamily.org"><joe@julianfamily.org></a>:
</pre>
<blockquote type="cite">
<pre wrap="">And yes, they can fail, but 20TB is small enough to heal pretty quickly.
</pre>
</blockquote>
<pre wrap="">
20TB small enough to build quickly? On which network? Gluster doesn't
have a dedicated cluster network, if the cluster is being hevily
accessed, the healing will slow down everything else (or everything
else will slow down the healing)</pre>
</blockquote>
<br>
Quickly = MTTR is within tolerances to continue to meet SLA. It's
just math.<br>
<br>
As for a dedicated heal network, split-horizon dns handles that just
fine. Clients resolve a server's hostname to the "eth1" (for
example) address and the servers themselves resolve the same
hostname to the "eth0" address. We played with bonding but decided
against the complexity.<br>
<br>
<blockquote
cite="mid:CAJH6TXhf_yUXs91Fk9Vd=cjgsdpUvO00oCgTc94GbzkczTdSQw@mail.gmail.com"
type="cite">
<pre wrap="">
Anyway, you can heal quickly, but I still prefere to have data safe on
each node. If you start with 3 server at once, probably each disk is
coming from the same batch, thus a massive disks failure is easy to
get.</pre>
</blockquote>
<br>
There's preference and there's engineering to meet requirements. If
your SLA is 5 nines and you engineer 6 nines, you may realize that
the difference between a 99.99993% uptime and a 99.99997% uptime
isn't worth the added expense of doing replication <i>and</i>
raid-1.<br>
<br>
<blockquote
cite="mid:CAJH6TXhf_yUXs91Fk9Vd=cjgsdpUvO00oCgTc94GbzkczTdSQw@mail.gmail.com"
type="cite">
<pre wrap="">
If you loose only 2 disks, one for each server, from the same replica
group, you are game over. With RAID6, you have to loose 5 disks from
the same replica group.</pre>
</blockquote>
<br>
I never loose my drives. They're always firmly attached. :P<br>
<br>
With 300 drives, 60 bricks, replica 3 (across 3 racks), I have a six
nines availability for any one replica subvolume. If you really want
to fudge the numbers, the reliability for any given file is not
worth calculating in that volume. The odds of all three bricks
failing for any 1 file among 20 distribute subvolumes is
statistically infinitesimal.<br>
<br>
<blockquote
cite="mid:CAJH6TXhf_yUXs91Fk9Vd=cjgsdpUvO00oCgTc94GbzkczTdSQw@mail.gmail.com"
type="cite">
<pre wrap="">
In my environment, I can create 4 RAID-0 on each server (3 disks on
each RAID0), or 2 RAID-6 with 6 disks each, or 1 RAID-6 with 12 disks
or 1 RAID-7 with 12 disks (RAID-7 with less than 12 disks is
non-sense)
I don't know which one is better.
</pre>
</blockquote>
<br>
Just do the reliability calculations and engineer a storage system
to meet (exceed) your obligations within the available budget.
<a class="moz-txt-link-freetext" href="http://www.eventhelix.com/realtimemantra/faulthandling/system_reliability_availability.htm">http://www.eventhelix.com/realtimemantra/faulthandling/system_reliability_availability.htm</a><br>
<br>
</body>
</html>