<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
On 02/12/2016 12:08 PM, Mike Stump wrote:<br>
<blockquote
cite="mid:75D8D91A-895C-46E5-8A42-4E154159E907@comcast.net"
type="cite">
Ok. I’m a new user, I want to make an array with 10 machines. I
want to be able to able to suffer the loss of any one machine. I
don’t mind wasting 50% of the disk space to do this. I don’t want
to suffer split brain. I want the array to support both read and
write access to data.
How do I achieve that?
</blockquote>
<br>
What is your acceptable annual downtime (typically outlined in an
SLA or OLA)? That's a bit of information you should have when you're
engineering a system.<br>
<br>
Split-brain happens when your replication has been partitioned and
writes have occurred in such a way that no valid copy can be
discerned. For the sake of example, we're going to use a very simple
file entitled "file.txt" with the contents of "The quick brown fox
jumped over the lazy yellow dog." It exists on a replicated volume
with no protection on a network where a server and client are in the
west wing, and the replica server and another client are in the east
wing. Somewhere in the middle, someone pulls the plug on the router.
The west client can see the west server and the east client can see
the east server.<br>
<br>
The west client updates file.txt changing the word "brown" to "red".
The east client updates the same file.txt and changes the word
"brown" to "white".<br>
<br>
The router recovers and the two servers try to synchronize any files
that were changed. They both had changes to file.txt. Which one was
right?<br>
<br>
There's no way to determine that from the information given. That's
split-brain.<br>
<br>
How can you combat split brain? <br>
<br>
One solution is quorum. Have enough replica that comparisons can be
made. If two servers are in the west and only one in the east and
they have the ability to determine quorum, the east server will not
allow writes during the network split. It can tell that it's not
safe because if they all three voted on which change was right, the
two in the west would win and data would be lost. The two in the
west see that one server is lost, but they still have quorum. They
allow the data to remain available, knowing that the out-of-quorum
server is safe from changes.<br>
<br>
Gluster has the ability to have a minimally participating quorum
participant called an arbiter. Let's make the west client an
arbiter. The net split happens. Only the two replica exist, one in
west and the other in east. The arbiter can see the west server but
not the east. The east server can see neither the west server nor
the arbiter. The east loses quorum but the west, seeing the arbiter,
does still have quorum and remains available with the safe
understanding that the east server, not having quorum, will not
accept writes.<br>
<br>
So with your 10 servers you could have a "replica 3 arbiter 1"
volume with one of the replica being an arbiter. It would only use
space for file names and metadata, but no actual data. If I were
doing it, I would probably do it as so:<br>
<br>
gluster volume create myvol replica 3 arbiter 1 server1:/brick1
server2:/brick1 server3:/arbiter \<br>
server3:/brick1 server4:/brick1 server5:/arbiter etc.<br>
<br>
Notice how there's both a data directory (/brick1) and an arbiter
directory (/arbiter) on bricks 3,5,7... which allows the data
"waste" that you're asking for while <i>mostly</i> allowing the
availability you seek. I say mostly because if your network
partitions, something's got to give or you will lose data. There's
absolutely no way for disconnected systems to coordinate binary
changes to each other with today's technology. <br>
<br>
Perhaps, one day, we will have quantum tunneling networks with
superimposed particles able to teleport data without the need of
networks, but that's not today. When that <i>is</i> available, I
expect rainbows and unicorns to be available as well.<br>
</body>
</html>