<html>
  <head>
    <meta content="text/html; charset=windows-1252"
      http-equiv="Content-Type">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <br>
    <br>
    <div class="moz-cite-prefix">On 09/03/2015 12:13 PM, Krutika
      Dhananjay wrote:<br>
    </div>
    <blockquote
      cite="mid:550072460.12279311.1441262638234.JavaMail.zimbra@redhat.com"
      type="cite">
      <div style="font-family: garamond,new york,times,serif; font-size:
        12pt; color: #000000">
        <div><br>
        </div>
        <div><br>
        </div>
        <hr id="zwchr">
        <blockquote style="border-left:2px solid
#1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From:
          </b>"Shyam" <a class="moz-txt-link-rfc2396E" href="mailto:srangana@redhat.com">&lt;srangana@redhat.com&gt;</a><br>
          <b>To: </b>"Krutika Dhananjay" <a class="moz-txt-link-rfc2396E" href="mailto:kdhananj@redhat.com">&lt;kdhananj@redhat.com&gt;</a><br>
          <b>Cc: </b>"Aravinda" <a class="moz-txt-link-rfc2396E" href="mailto:avishwan@redhat.com">&lt;avishwan@redhat.com&gt;</a>, "Gluster
          Devel" <a class="moz-txt-link-rfc2396E" href="mailto:gluster-devel@gluster.org">&lt;gluster-devel@gluster.org&gt;</a><br>
          <b>Sent: </b>Wednesday, September 2, 2015 11:13:55 PM<br>
          <b>Subject: </b>Re: [Gluster-devel] Gluster Sharding and
          Geo-replication<br>
          <div><br>
          </div>
          On 09/02/2015 10:47 AM, Krutika Dhananjay wrote:<br>
          &gt;<br>
          &gt;<br>
          &gt;
          ------------------------------------------------------------------------<br>
          &gt;<br>
          &gt;     *From: *"Shyam" <a class="moz-txt-link-rfc2396E" href="mailto:srangana@redhat.com">&lt;srangana@redhat.com&gt;</a><br>
          &gt;     *To: *"Aravinda" <a class="moz-txt-link-rfc2396E" href="mailto:avishwan@redhat.com">&lt;avishwan@redhat.com&gt;</a>,
          "Gluster Devel"<br>
          &gt;     <a class="moz-txt-link-rfc2396E" href="mailto:gluster-devel@gluster.org">&lt;gluster-devel@gluster.org&gt;</a><br>
          &gt;     *Sent: *Wednesday, September 2, 2015 8:09:55 PM<br>
          &gt;     *Subject: *Re: [Gluster-devel] Gluster Sharding and
          Geo-replication<br>
          &gt;<br>
          &gt;     On 09/02/2015 03:12 AM, Aravinda wrote:<br>
          &gt;      &gt; Geo-replication and Sharding Team today
          discussed about the approach<br>
          &gt;      &gt; to make Sharding aware Geo-replication. Details
          are as below<br>
          &gt;      &gt;<br>
          &gt;      &gt; Participants: Aravinda, Kotresh, Krutika, Rahul
          Hinduja, Vijay Bellur<br>
          &gt;      &gt;<br>
          &gt;      &gt; - Both Master and Slave Volumes should be
          Sharded Volumes with same<br>
          &gt;      &gt;    configurations.<br>
          &gt;<br>
          &gt;     If I am not mistaken, geo-rep supports replicating to
          a non-gluster<br>
          &gt;     local FS at the slave end. Is this correct? If so,
          would this<br>
          &gt;     limitation<br>
          &gt;     not make that problematic?<br>
          &gt;<br>
          &gt;     When you state *same configuration*, I assume you
          mean the sharding<br>
          &gt;     configuration, not the volume graph, right?<br>
          &gt;<br>
          &gt; That is correct. The only requirement is for the slave to
          have shard<br>
          &gt; translator (for, someone needs to present aggregated view
          of the file to<br>
          &gt; the READers on the slave).<br>
          &gt; Also the shard-block-size needs to be kept same between
          master and<br>
          &gt; slave. Rest of the configuration (like the number of
          subvols of DHT/AFR)<br>
          &gt; can vary across master and slave.<br>
          <div><br>
          </div>
          Do we need to have the sharded block size the same? As I
          assume the file <br>
          carries an xattr that contains the size it is sharded with <br>
          (trusted.glusterfs.shard.block-size), so if this is synced
          across, it <br>
          would do. If this is true, what it would mean is that "a
          sharded volume <br>
          needs a shard supported slave to ge-rep to".<br>
        </blockquote>
        <div>Yep. Even I feel it should probably not be necessary to
          enforce same-shard-size-everywhere as long as shard translator
          on the slave takes care not to further "shard" the individual
          shards gsyncD would write to, on the slave volume.<br>
        </div>
        <div>This is especially true if different files/images/vdisks on
          the master volume are associated with different block sizes.<br>
        </div>
        <div>This logic has to be built into the shard translator based
          on parameters (client-pid, parent directory of the file being
          written to).<br>
        </div>
        <div>What this means is that shard-block-size attribute on the
          slave would essentially be a don't-care parameter. I need to
          give all this some more thought though.<br>
        </div>
      </div>
    </blockquote>
    <br>
    <br>
    I think this may help with coping with changes in shard block size
    configuration in master. Otherwise, once user changes the shard
    block size in master, the slave will be affected.<br>
    Are there any other shard volume options that if changed on master,
    would affect slave? How do we ensure master and slave are in sync
    w.r.t the shard configuration?<br>
    <br>
    <blockquote
      cite="mid:550072460.12279311.1441262638234.JavaMail.zimbra@redhat.com"
      type="cite">
      <div style="font-family: garamond,new york,times,serif; font-size:
        12pt; color: #000000">
        <div><br>
        </div>
        <div>-Krutika<br>
        </div>
        <div><br>
        </div>
        <blockquote style="border-left:2px solid
#1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;">&gt;<br>
          &gt; -Krutika<br>
          &gt;<br>
          &gt;<br>
          &gt;<br>
          &gt;      &gt; - In Changelog record changes related to
          Sharded files also. Just<br>
          &gt;     like<br>
          &gt;      &gt;    any regular files.<br>
          &gt;      &gt; - Sharding should allow Geo-rep to
          list/read/write Sharding internal<br>
          &gt;      &gt;    Xattrs if Client PID is gsyncd(-1)<br>
          &gt;      &gt; - Sharding should allow read/write of Sharded
          files(that is in<br>
          &gt;     .shards<br>
          &gt;      &gt;    directory) if Client PID is GSYNCD<br>
          &gt;      &gt; - Sharding should return actual file instead of
          returning the<br>
          &gt;      &gt;    aggregated content when the Main file is
          requested(Client PID<br>
          &gt;      &gt;    GSYNCD)<br>
          &gt;      &gt;<br>
          &gt;      &gt; For example, a file f1 is created with GFID G1.<br>
          &gt;      &gt;<br>
          &gt;      &gt; When the file grows it gets sharded into
          chunks(say 5 chunks).<br>
          &gt;      &gt;<br>
          &gt;      &gt;      f1   G1<br>
          &gt;      &gt;      .shards/G1.1   G2<br>
          &gt;      &gt;      .shards/G1.2   G3<br>
          &gt;      &gt;      .shards/G1.3   G4<br>
          &gt;      &gt;      .shards/G1.4   G5<br>
          &gt;      &gt;<br>
          &gt;      &gt; In Changelog, this is recorded as 5 different
          files as below<br>
          &gt;      &gt;<br>
          &gt;      &gt;      CREATE G1 f1<br>
          &gt;      &gt;      DATA G1<br>
          &gt;      &gt;      META G1<br>
          &gt;      &gt;      CREATE G2 PGS/G1.1<br>
          &gt;      &gt;      DATA G2<br>
          &gt;      &gt;      META G1<br>
          &gt;      &gt;      CREATE G3 PGS/G1.2<br>
          &gt;      &gt;      DATA G3<br>
          &gt;      &gt;      META G1<br>
          &gt;      &gt;      CREATE G4 PGS/G1.3<br>
          &gt;      &gt;      DATA G4<br>
          &gt;      &gt;      META G1<br>
          &gt;      &gt;      CREATE G5 PGS/G1.4<br>
          &gt;      &gt;      DATA G5<br>
          &gt;      &gt;      META G1<br>
          &gt;      &gt;<br>
          &gt;      &gt; Where PGS is GFID of .shards directory.<br>
          &gt;      &gt;<br>
          &gt;      &gt; Geo-rep will create these files independently
          in Slave Volume and<br>
          &gt;      &gt; syncs Xattrs of G1. Data can be read only when
          all the chunks are<br>
          &gt;      &gt; synced to Slave Volume. Data can be read
          partially if main/first file<br>
          &gt;      &gt; and some of the chunks synced to Slave.<br>
          &gt;      &gt;<br>
          &gt;      &gt; Please add if I missed anything. C &amp; S
          Welcome.<br>
          &gt;      &gt;<br>
          &gt;      &gt; regards<br>
          &gt;      &gt; Aravinda<br>
          &gt;      &gt;<br>
          &gt;      &gt; On 08/11/2015 04:36 PM, Aravinda wrote:<br>
          &gt;      &gt;&gt; Hi,<br>
          &gt;      &gt;&gt;<br>
          &gt;      &gt;&gt; We are thinking different approaches to add
          support in<br>
          &gt;     Geo-replication<br>
          &gt;      &gt;&gt; for Sharded Gluster Volumes[1]<br>
          &gt;      &gt;&gt;<br>
          &gt;      &gt;&gt; *Approach 1: Geo-rep: Sync Full file*<br>
          &gt;      &gt;&gt;    - In Changelog only record main file
          details in the same brick<br>
          &gt;      &gt;&gt; where it is created<br>
          &gt;      &gt;&gt;    - Record as DATA in Changelog whenever
          any addition/changes<br>
          &gt;     to the<br>
          &gt;      &gt;&gt; sharded file<br>
          &gt;      &gt;&gt;    - Geo-rep rsync will do checksum as a
          full file from mount and<br>
          &gt;      &gt;&gt; syncs as new file<br>
          &gt;      &gt;&gt;    - Slave side sharding is managed by
          Slave Volume<br>
          &gt;      &gt;&gt; *Approach 2: Geo-rep: Sync sharded file
          separately*<br>
          &gt;      &gt;&gt;    - Geo-rep rsync will do checksum for
          sharded files only<br>
          &gt;      &gt;&gt;    - Geo-rep syncs each sharded files
          independently as new files<br>
          &gt;      &gt;&gt;    - [UNKNOWN] Sync internal xattrs(file
          size and block count)<br>
          &gt;     in the<br>
          &gt;      &gt;&gt; main sharded file to Slave Volume to
          maintain the same state as<br>
          &gt;     in Master.<br>
          &gt;      &gt;&gt;    - Sharding translator to allow file
          creation under .shards<br>
          &gt;     dir for<br>
          &gt;      &gt;&gt; gsyncd. that is Parent GFID is .shards
          directory<br>
          &gt;      &gt;&gt;    - If sharded files are modified during
          Geo-rep run may end up<br>
          &gt;     stale<br>
          &gt;      &gt;&gt; data in Slave.<br>
          &gt;      &gt;&gt;    - Files on Slave Volume may not be
          readable unless all sharded<br>
          &gt;      &gt;&gt; files sync to Slave(Each bricks in Master
          independently sync<br>
          &gt;     files to<br>
          &gt;      &gt;&gt; slave)<br>
          &gt;      &gt;&gt;<br>
          &gt;      &gt;&gt; First approach looks more clean, but we
          have to analize the Rsync<br>
          &gt;      &gt;&gt; checksum performance on big files(Sharded
          in backend, accessed<br>
          &gt;     as one<br>
          &gt;      &gt;&gt; big file from rsync)<br>
          &gt;      &gt;&gt;<br>
          &gt;      &gt;&gt; Let us know your thoughts. Thanks<br>
          &gt;      &gt;&gt;<br>
          &gt;      &gt;&gt; Ref:<br>
          &gt;      &gt;&gt; [1]<br>
          &gt;      &gt;&gt;<br>
          &gt;    
<a class="moz-txt-link-freetext" href="http://www.gluster.org/community/documentation/index.php/Features/sharding-xlator">http://www.gluster.org/community/documentation/index.php/Features/sharding-xlator</a><br>
          &gt;      &gt;&gt; --<br>
          &gt;      &gt;&gt; regards<br>
          &gt;      &gt;&gt; Aravinda<br>
          &gt;      &gt;&gt;<br>
          &gt;      &gt;&gt;<br>
          &gt;      &gt;&gt;
          _______________________________________________<br>
          &gt;      &gt;&gt; Gluster-devel mailing list<br>
          &gt;      &gt;&gt; <a class="moz-txt-link-abbreviated" href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
          &gt;      &gt;&gt;
          <a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-devel">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
          &gt;      &gt;<br>
          &gt;      &gt;<br>
          &gt;      &gt;<br>
          &gt;      &gt; _______________________________________________<br>
          &gt;      &gt; Gluster-devel mailing list<br>
          &gt;      &gt; <a class="moz-txt-link-abbreviated" href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
          &gt;      &gt;
          <a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-devel">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
          &gt;      &gt;<br>
          &gt;     _______________________________________________<br>
          &gt;     Gluster-devel mailing list<br>
          &gt;     <a class="moz-txt-link-abbreviated" href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
          &gt;     <a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-devel">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
          &gt;<br>
          &gt;<br>
        </blockquote>
        <div><br>
        </div>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
Gluster-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-devel">http://www.gluster.org/mailman/listinfo/gluster-devel</a>
</pre>
    </blockquote>
    <br>
  </body>
</html>