<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Cambria;
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
code
{mso-style-priority:99;
font-family:"Courier New";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier New";}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle24
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:852382096;
mso-list-type:hybrid;
mso-list-template-ids:318007156 53746050 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-start-at:2;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:63.6pt;
text-indent:-.25in;
font-family:"Calibri","sans-serif";
mso-fareast-font-family:Calibri;
mso-bidi-font-family:"Times New Roman";}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:99.6pt;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:135.6pt;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:171.6pt;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:207.6pt;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:243.6pt;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:279.6pt;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:315.6pt;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:351.6pt;
text-indent:-.25in;
font-family:Wingdings;}
@list l1
{mso-list-id:1560282229;
mso-list-type:hybrid;
mso-list-template-ids:-760292678 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l1:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">Hi Sussant,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Extremely sorry for the belated reply. Thanks for your input. We will try having rebalance then create a set of small files with a random pattern generation and check where it falls in by DM_TYPE DHT.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I have one short query:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">We were also thinking to use (in case we need )
<b>Translators/cluster/unify. </b>Along with <b>NUFA or ALU scheduler</b>. However we noticed
<b>Translators/cluster/unify is became obsolete/legacy , in that case automatically ALU, NUFA,rr scheduler also became obsolete ?<o:p></o:p></b></span></p>
<p class="MsoNormal"><b><span style="color:#1F497D"><o:p> </o:p></span></b></p>
<p class="MsoNormal"><b><span style="color:#1F497D">Is the Translators/cluster/unify<o:p></o:p></span></b></p>
<p class="MsoNormal"><b><span style="color:#1F497D"><o:p> </o:p></span></b></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Subrata Ghosh
<br>
<b>Sent:</b> Thursday, May 21, 2015 4:26 PM<br>
<b>To:</b> gluster-devel@gluster.org; 'gluster-users@gluster.org'<br>
<b>Cc:</b> Nobin Mathew; 'Susant Palai'; 'Vijay Bellur'<br>
<b>Subject:</b> Regarding the issues gluster DHT and Layouts of bricks<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hi All,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Could you please guide us to solve the following DHT and brick layout problem we are dealing with ? Questions are marked
<b>bold.</b><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Problem statement :<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l1 level1 lfo2"><![if !supportLists]><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>We have a requirement to achieve maximum write and read performance and we have to meet some committed performance metrics.<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in"><o:p> </o:p></p>
<p class="MsoNormal"><b> Our goal is to place each file into different bricks to get optimal performance and also observer the nature of the throughput , hence need to have a mechanism to generate different hash using
</b>gluster <code><span style="font-size:10.0pt">glusterfs.gf_dm_hashfn</span></code><b>,
<o:p></o:p></b></p>
<p class="MsoNormal" style="margin-left:.5in"><b>(assuming number of files are : N , Number of Bricks :N) to place spate bricks.<o:p></o:p></b></p>
<p class="MsoNormal" style="margin-left:.5in"><b><o:p> </o:p></b></p>
<p class="MsoListParagraph" style="margin-left:63.6pt;text-indent:-.25in;mso-list:l0 level1 lfo4">
<![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]><b>How to make sure each file has different hash and falls to different bricks ?
<o:p></o:p></b></p>
<p class="MsoListParagraph" style="margin-left:63.6pt"><b><o:p> </o:p></b></p>
<p class="MsoListParagraph" style="margin-left:63.6pt;text-indent:-.25in;mso-list:l0 level1 lfo4">
<![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]><b>Other way to put the question if I know the range of the brick layout or more precisely if I know the hex value of the desired hash ( so that it will be placed desired brick) that we need to generate from Davis-Meyer algorithm
used in gluster, Can we create a file name such that, that also solve our problem to some extent?<o:p></o:p></b></p>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l1 level1 lfo2"><![if !supportLists]><code><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman"">
</span></span></span></code><![endif]>We tried to experiment to see how a file in gluster is decided to be placed in a particular brick following
<b>gluster </b><code><b><span style="font-size:10.0pt">glusterfs.gf_dm_hashfn</span></b></code><code><span style="font-size:10.0pt">
</span></code><code><span style="font-size:10.0pt;font-family:"Cambria","serif"">and took some idea from
</span></code><code><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></code></p>
<p class="MsoNormal" style="margin-left:.25in"><code><span style="font-size:10.0pt;font-family:"Cambria","serif""> some articles like
<b><a href="http://gluster.readthedocs.org/en/latest/Features/dht/">http://gluster.readthedocs.org/en/latest/Features/dht/</a></b> ,
<b><a href="https://joejulian.name/blog/dht-misses-are-expensive/">https://joejulian.name/blog/dht-misses-are-expensive/</a></b> page which describes
</span></code>layout for that brick<code><span style="font-size:10.0pt;font-family:"Cambria","serif""> and
</span></code>calculate a hash for the file. <o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:.25in"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:.25in"> To minimize collisions or generating different hash in such way to place each file in different bricks ( file 1 => brick A, file 2 => Brick B, file 3=> Brick C, file 4 => brick D)<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in"><o:p> </o:p></p>
<p class="MsoNormal"> We use kind of similar script to get the hash value for a file<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" style="margin-left:49.5pt">
<tbody>
<tr>
<td width="413" style="width:247.5pt;padding:0in 0in 0in 0in">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";color:#558ED5">def</span><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#558ED5">
</span><span style="font-size:10.0pt;font-family:"Courier New";color:#558ED5">gf_dm_hashfn(filename):</span><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#558ED5"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";color:#558ED5"> return</span><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#558ED5">
</span><span style="font-size:10.0pt;font-family:"Courier New";color:#558ED5">ctypes.c_uint32(glusterfs.gf_dm_hashfn(</span><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#558ED5"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";color:#558ED5"> filename,</span><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#558ED5"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";color:#558ED5"> len(filendame)))</span><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#558ED5"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#558ED5"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";color:#558ED5">if</span><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#558ED5">
</span><span style="font-size:10.0pt;font-family:"Courier New";color:#558ED5">__name__ ==</span><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#558ED5">
</span><span style="font-size:10.0pt;font-family:"Courier New";color:#558ED5">"__main__":</span><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#558ED5"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";color:#558ED5"> print</span><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#558ED5">
</span><span style="font-size:10.0pt;font-family:"Courier New";color:#558ED5">hex(gf_dm_hashfn(sys.argv[1]).value)</span><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#558ED5"><o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Times New Roman","serif"">We can then calculate the hash for a filename:<o:p></o:p></span></p>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0">
<tbody>
<tr>
<td style="padding:0in 0in 0in 0in">
<p class="MsoNormal"><span style="font-family:"Courier New";color:#558ED5"># python gf_dm_hash.py file1
</span><span style="font-family:"Times New Roman","serif";color:#558ED5"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#558ED5">0x99d1b6fL</span><span style="font-family:"Times New Roman","serif""><o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Extended attribute is fetch to check the range and try to match the above generated hash value.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><code><span style="font-size:10.0pt;color:#558ED5">getfattr -n trusted.glusterfs.dht -e hex file1<o:p></o:p></span></code></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:.25in"><o:p> </o:p></p>
<p class="MsoNormal"> However we are not able to exactly follow till this point , how the hash value matched to one of the layout assignments, to yield what we call a
<em><span style="font-family:"Calibri","sans-serif"">hashed location. <o:p></o:p></span></em></p>
<p class="MsoNormal"><em><span style="font-family:"Calibri","sans-serif""><o:p> </o:p></span></em></p>
<p class="MsoListParagraph" style="margin-left:63.6pt;text-indent:-.25in;mso-list:l0 level1 lfo4">
<![if !supportLists]><code><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span></span></code><![endif]><em><b><span style="font-family:"Calibri","sans-serif";font-style:normal">My question is if I know the range of brick lay out ( say </span></b></em><b>0xc0000000 to 0xffffffff, is range select a hash 0xc0070000<em><span style="font-family:"Calibri","sans-serif";font-style:normal">
) where to be placed the next file can we generate the name ( kind of reverse of </span></em>gluster
</b><code><b><span style="font-size:10.0pt">glusterfs.gf_dm_hashfn) ?</span></b></code><code><b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></b></code></p>
<p class="MsoNormal"><em><span style="font-family:"Calibri","sans-serif";font-style:normal"><o:p> </o:p></span></em></p>
<p class="MsoNormal"><em><span style="font-family:"Calibri","sans-serif";font-style:normal">PS : Susant : Can you throw some light or suggest a method we are trying to solve.<o:p></o:p></span></em></p>
<p class="MsoNormal"><em><span style="font-family:"Calibri","sans-serif";font-style:normal"><o:p> </o:p></span></em></p>
<p class="MsoNormal"><em><span style="font-family:"Calibri","sans-serif";font-style:normal">Thanks for your time.<o:p></o:p></span></em></p>
<p class="MsoNormal"><em><span style="font-family:"Calibri","sans-serif";font-style:normal"><o:p> </o:p></span></em></p>
<p class="MsoNormal"><em><span style="font-family:"Calibri","sans-serif";font-style:normal"><o:p> </o:p></span></em></p>
<p class="MsoNormal"><em><span style="font-family:"Calibri","sans-serif";font-style:normal">Best Regards,<o:p></o:p></span></em></p>
<p class="MsoNormal"><em><span style="font-family:"Calibri","sans-serif";font-style:normal">Subrata Ghosh<o:p></o:p></span></em></p>
<p class="MsoNormal"><em><span style="font-family:"Calibri","sans-serif";font-style:normal"><o:p> </o:p></span></em></p>
<p class="MsoNormal" style="margin-left:.25in"><em><span style="font-family:"Calibri","sans-serif""><o:p> </o:p></span></em></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="font-size:10.0pt;font-family:"Cambria","serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>