<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<tt>Geo-replication and Sharding Team today discussed about the
approach</tt><tt><br>
</tt><tt>to make Sharding aware Geo-replication. Details are as
below</tt><tt><br>
</tt><tt><br>
</tt><tt>Participants: Aravinda, Kotresh, Krutika, Rahul Hinduja,
Vijay Bellur</tt><tt><br>
</tt><tt><br>
</tt><tt>- Both Master and Slave Volumes should be Sharded Volumes
with same</tt><tt><br>
</tt><tt> configurations.</tt><tt><br>
</tt><tt>- In Changelog record changes related to Sharded files
also. Just like</tt><tt><br>
</tt><tt> any regular files.</tt><tt><br>
</tt><tt>- Sharding should allow Geo-rep to list/read/write Sharding
internal</tt><tt><br>
</tt><tt> Xattrs if Client PID is gsyncd(-1)</tt><tt><br>
</tt><tt>- Sharding should allow read/write of Sharded files(that is
in .shards</tt><tt><br>
</tt><tt> directory) if Client PID is GSYNCD</tt><tt><br>
</tt><tt>- Sharding should return actual file instead of returning
the</tt><tt><br>
</tt><tt> aggregated content when the Main file is requested(Client
PID</tt><tt><br>
</tt><tt> GSYNCD)</tt><tt><br>
</tt><tt><br>
</tt><tt>For example, a file f1 is created with GFID G1.</tt><tt><br>
</tt><tt><br>
</tt><tt>When the file grows it gets sharded into chunks(say 5
chunks).</tt><tt><br>
</tt><tt><br>
</tt><tt> f1 G1</tt><tt><br>
</tt><tt> .shards/G1.1 G2</tt><tt><br>
</tt><tt> .shards/G1.2 G3</tt><tt><br>
</tt><tt> .shards/G1.3 G4</tt><tt><br>
</tt><tt> .shards/G1.4 G5</tt><tt><br>
</tt><tt><br>
</tt><tt>In Changelog, this is recorded as 5 different files as
below</tt><tt><br>
</tt><tt><br>
</tt><tt> CREATE G1 f1</tt><tt><br>
</tt><tt> DATA G1</tt><tt><br>
</tt><tt> META G1</tt><tt><br>
</tt><tt> CREATE G2 PGS/G1.1</tt><tt><br>
</tt><tt> DATA G2</tt><tt><br>
</tt><tt> META G1</tt><tt><br>
</tt><tt> CREATE G3 PGS/G1.2</tt><tt><br>
</tt><tt> DATA G3</tt><tt><br>
</tt><tt> META G1</tt><tt><br>
</tt><tt> CREATE G4 PGS/G1.3</tt><tt><br>
</tt><tt> DATA G4</tt><tt><br>
</tt><tt> META G1</tt><tt><br>
</tt><tt> CREATE G5 PGS/G1.4</tt><tt><br>
</tt><tt> DATA G5</tt><tt><br>
</tt><tt> META G1</tt><tt><br>
</tt><tt><br>
</tt><tt>Where PGS is GFID of .shards directory.</tt><tt><br>
</tt><tt><br>
</tt><tt>Geo-rep will create these files independently in Slave
Volume and</tt><tt><br>
</tt><tt>syncs Xattrs of G1. Data can be read only when all the
chunks are</tt><tt><br>
</tt><tt>synced to Slave Volume. Data can be read partially if
main/first file</tt><tt><br>
</tt><tt>and some of the chunks synced to Slave.</tt><tt><br>
</tt><tt><br>
</tt><tt>Please add if I missed anything. C & S Welcome.<br>
<br>
</tt>
<pre class="moz-signature" cols="72">regards
Aravinda
</pre>
<div class="moz-cite-prefix">On 08/11/2015 04:36 PM, Aravinda wrote:<br>
</div>
<blockquote cite="mid:55C9D73C.7020704@redhat.com" type="cite">
<meta http-equiv="content-type" content="text/html;
charset=windows-1252">
Hi,<br>
<br>
We are thinking different approaches to add support in
Geo-replication for Sharded Gluster Volumes[1]<br>
<br>
<div class="ace-line" id="magicdomid731"><span
class="author-g-6mby7anhycxlsw8t b"><b>Approach 1: Geo-rep:
Sync Full file</b></span></div>
<div class="ace-line" id="magicdomid132"><span
class="author-g-6mby7anhycxlsw8t"> - In Changelog only
record main file details in the same brick where it is created<br>
</span></div>
<div class="ace-line" id="magicdomid223"><span
class="author-g-6mby7anhycxlsw8t"> - Record as DATA in
Changelog whenever any addition/changes to the sharded file</span></div>
<div class="ace-line" id="magicdomid321"><span
class="author-g-6mby7anhycxlsw8t"> - Geo-rep rsync will do
checksum as a full file from mount and syncs as new file</span></div>
<div class="ace-line" id="magicdomid443"><span
class="author-g-6mby7anhycxlsw8t"> - Slave side sharding is
managed by Slave Volume</span></div>
<div class="ace-line" id="magicdomid446"><span
class="author-g-6mby7anhycxlsw8t"> </span></div>
<div class="ace-line" id="magicdomid750"><span
class="author-g-6mby7anhycxlsw8t b"><b>Approach 2: Geo-rep:
Sync sharded file separately</b></span></div>
<div class="ace-line" id="magicdomid552"><span
class="author-g-6mby7anhycxlsw8t"> - Geo-rep rsync will do
checksum for sharded files only</span></div>
<div class="ace-line" id="magicdomid613"><span
class="author-g-6mby7anhycxlsw8t"> - Geo-rep syncs each
sharded files independently as new files</span></div>
<div class="ace-line" id="magicdomid937"><span
class="author-g-6mby7anhycxlsw8t"> - [UNKNOWN] Sync internal
xattrs(file size and block count) in the main sharded file to
Slave Volume to maintain the same state as in Master.</span></div>
<div class="ace-line" id="magicdomid829"><span
class="author-g-6mby7anhycxlsw8t"> - Sharding translator to
allow file creation under .shards dir for gsyncd. that is
Parent GFID is .shards directory<br>
</span></div>
<div class="ace-line" id="magicdomid938"><span
class="author-g-6mby7anhycxlsw8t"> - If sharded files are
modified during Geo-rep run may end up stale data in Slave.<br>
- Files on Slave Volume may not be readable unless all
sharded files sync to Slave(Each bricks in Master
independently sync files to slave)<br>
</span></div>
<br>
First approach looks more clean, but we have to analize the Rsync
checksum performance on big files(Sharded in backend, accessed as
one big file from rsync)<br>
<br>
Let us know your thoughts. Thanks<br>
<br>
Ref:<br>
[1]
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://www.gluster.org/community/documentation/index.php/Features/sharding-xlator">http://www.gluster.org/community/documentation/index.php/Features/sharding-xlator</a><br>
<pre class="moz-signature" cols="72">--
regards
Aravinda</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-devel">http://www.gluster.org/mailman/listinfo/gluster-devel</a>
</pre>
</blockquote>
<br>
</body>
</html>