<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<br>
<br>
<div class="moz-cite-prefix">On 10/11/2015 03:53 AM, Lindsay
Mathieson wrote:<br>
</div>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>I'm unclear as to the distinction between the two,
but my guess:<br>
<br>
</div>
<div>- Quorum is only relevant to replica volumes<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
Client quorum is applicable only to replica volumes. Server quorum
is applicable to all volumes.<br>
<br>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>- It is p[referable to have a odd number of
replicas and a minimum of 3 <br>
</div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
Right.<br>
<br>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>Client Quorum;<br>
</div>
<div>- Client is the fuse client or gfapi driver<br>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
True. Also in the gluster nfs server when you use nfs mounts. Client
quorum is implemented in the AFR translator. So it comes into play
wherever this xlator is loaded.<br>
<br>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>- Quorum check is performed by the client <br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
More specifically the AFR xlator.<br>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>- if the check fails, the client will fail writes<br>
</div>
</div>
</div>
</blockquote>
<br>
Right. AFR will fail the writes with EROFS.<br>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div> * is there a timeout involved or does it fail
immediately?<br>
</div>
</div>
</div>
</blockquote>
<br>
If an ongoing write did not succeed on the necessary number of
bricks due to brick disconnect, that FOP will fail with EROFS. Also,
subsequent writes will also fail with the same error until quorum is
restored (i.e. client connects to the bricks again). So yes, it is
pretty much immediate.<br>
<br>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>- Quorum is determined by how many active bricks the
client can see<br>
</div>
<div> * As determined by quorum-type and/or quorum-count<br>
</div>
- Bricks remain up<br>
</div>
</div>
</blockquote>
True. Unless the brick process actually went down.<br>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">- Datastore remains up<br>
</div>
</blockquote>
Not sure if you are referring to other bricks, in which case yes.<br>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div><br>
</div>
<div>Server Quorum:<br>
</div>
<div>- Server is the brick process (glusterd?)<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
Not glusterd but the brick process i.e glusterfsd. <br>
The implementation of server quorum is in glusterd though.<br>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>- enabled with server-quorum-type=server<br>
</div>
<div>- Quorum check is performed by the server<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
Right.<br>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>- Quorum is determined by how many active bricks
the server can see<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
It's more like how many peers the glusterd on each node can see.
Have a look at "How to Test" section in
<a class="moz-txt-link-freetext" href="http://www.gluster.org/community/documentation/index.php/Features/Server-quorum">http://www.gluster.org/community/documentation/index.php/Features/Server-quorum</a><br>
<br>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>- If quorum fails<br>
* brick is brought down<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
Correct. By glusterd.<br>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div> * datastore remains up<br>
<br>
<br>
</div>
<div>In both cases bricks which remain part of a quorum
can still be written to, whereas bricks which are
isolated are readonly, or down altogether and will be
healed once quorums returns. In theory this will
prevent split brain problems.<br>
</div>
<div><br clear="all">
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
Yes.<br>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>
<div>Question:<br>
</div>
<div>- Which is better, server or client quorums?<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
You can still end up in split-brain of the files stored in the
volume if sever quorum is enabled. Sever quorum is more useful to
avoid conflicts in volume configuration since it also disallows
volume set commands, peer probe etc when not in quorum. Client
quorum is better if you want to avoid split-brains of files present
in the volume.<br>
<br>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>
<div>- Can you safely enable both? recommended?<br>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
IMHO, client-quorum is enough. In case of dist-rep volumes, it acts
on only those replica sets where quorum is not met making only that
replica pair EROFS. Server quorum outright kills the bricks, not
even allowing read access. But yes, you can enable both. <br>
<br>
<blockquote
cite="mid:CAEMkAmGEDNTiUbmtEuzMMS97-yhdR9cNhO0_zpBaoL=e8x4BQQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>
<div>thanks,<br>
</div>
<div>-- <br>
<div class="gmail_signature">Lindsay</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</body>
</html>