<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 12/08/2016 06:53 PM, Atin Mukherjee
wrote:<br>
</div>
<blockquote
cite="mid:CAGNCGH1vqHMP=+2ZgR66fVCTwMMYbpk0C2SEr0pLdjZszGSsCw@mail.gmail.com"
type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Dec 8, 2016 at 6:44 PM, Miloš
Čučulović - MDPI <span dir="ltr"><<a
moz-do-not-send="true" href="mailto:cuculovic@mdpi.com"
target="_blank">cuculovic@mdpi.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Ah,
damn! I found the issue. On the storage server, the
storage2 IP address was wrong, I inversed two digits in
the /etc/hosts file, sorry for that :(<br>
<br>
I was able to add the brick now, I started the heal, but
still no data transfer visible.<br>
</blockquote>
</div>
</div>
</div>
</blockquote>
1. Are the files getting created on the new brick though?<br>
2. Can you provide the output of `getfattr -d -m . -e hex
/data/data-cluster` on both bricks?<br>
3. Is it possible to attach gdb to the self-heal daemon on the
original (old) brick and get a backtrace?<br>
<tt> `gdb -p <pid of self-heal daemon on the orignal
brick>`</tt><tt><br>
</tt><tt> thread apply all bt</tt><tt> -->share this output<br>
quit gdb.<br>
</tt><br>
<br>
-Ravi<br>
<blockquote
cite="mid:CAGNCGH1vqHMP=+2ZgR66fVCTwMMYbpk0C2SEr0pLdjZszGSsCw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>@Ravi/Pranith - can you help here?<br>
<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
By doing gluster volume status, I have<span class=""><br>
<br>
Status of volume: storage<br>
Gluster process TCP Port RDMA
Port Online Pid<br>
------------------------------<wbr>------------------------------<wbr>------------------<br>
</span>
Brick storage2:/data/data-cluster 49152 0
Y 23101<br>
Brick storage:/data/data-cluster 49152 0
Y 30773<br>
Self-heal Daemon on localhost N/A N/A
Y 30050<br>
Self-heal Daemon on storage N/A N/A
Y 30792<br>
<br>
<br>
Any idea?<br>
<br>
On storage I have:<br>
Number of Peers: 1<br>
<br>
Hostname: 195.65.194.217<br>
Uuid: 7c988af2-9f76-4843-8e6f-d94866<wbr>d57bb0<br>
State: Peer in Cluster (Connected)<span class=""><br>
<br>
<br>
- Kindest regards,<br>
<br>
Milos Cuculovic<br>
IT Manager<br>
<br>
---<br>
MDPI AG<br>
Postfach, CH-4020 Basel, Switzerland<br>
Office: St. Alban-Anlage 66, 4052 Basel, Switzerland<br>
Tel. +41 61 683 77 35<br>
Fax +41 61 302 89 18<br>
Email: <a moz-do-not-send="true"
href="mailto:cuculovic@mdpi.com" target="_blank">cuculovic@mdpi.com</a><br>
Skype: milos.cuculovic.mdpi<br>
<br>
</span><span class="">
On 08.12.2016 13:55, Atin Mukherjee wrote:<br>
</span>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"><span
class="">
Can you resend the attachment as zip? I am unable to
extract the<br>
content? We shouldn't have 0 info file. What does
gluster peer status<br>
output say?<br>
<br>
On Thu, Dec 8, 2016 at 4:51 PM, Miloš Čučulović - MDPI<br>
</span><span class="">
<<a moz-do-not-send="true"
href="mailto:cuculovic@mdpi.com" target="_blank">cuculovic@mdpi.com</a>
<mailto:<a moz-do-not-send="true"
href="mailto:cuculovic@mdpi.com" target="_blank">cuculovic@mdpi.com</a>>>
wrote:<br>
<br>
I hope you received my last email Atin, thank you!<br>
<br>
- Kindest regards,<br>
<br>
Milos Cuculovic<br>
IT Manager<br>
<br>
---<br>
MDPI AG<br>
Postfach, CH-4020 Basel, Switzerland<br>
Office: St. Alban-Anlage 66, 4052 Basel,
Switzerland<br>
Tel. +41 61 683 77 35<br>
Fax +41 61 302 89 18<br>
</span>
Email: <a moz-do-not-send="true"
href="mailto:cuculovic@mdpi.com" target="_blank">cuculovic@mdpi.com</a>
<mailto:<a moz-do-not-send="true"
href="mailto:cuculovic@mdpi.com" target="_blank">cuculovic@mdpi.com</a>><span
class=""><br>
Skype: milos.cuculovic.mdpi<br>
<br>
On 08.12.2016 10:28, Atin Mukherjee wrote:<br>
<br>
<br>
---------- Forwarded message ----------<br>
From: *Atin Mukherjee* <<a
moz-do-not-send="true"
href="mailto:amukherj@redhat.com" target="_blank">amukherj@redhat.com</a><br>
</span><span class="">
<mailto:<a moz-do-not-send="true"
href="mailto:amukherj@redhat.com" target="_blank">amukherj@redhat.com</a>>
<mailto:<a moz-do-not-send="true"
href="mailto:amukherj@redhat.com" target="_blank">amukherj@redhat.com</a><br>
<mailto:<a moz-do-not-send="true"
href="mailto:amukherj@redhat.com" target="_blank">amukherj@redhat.com</a>>>><br>
Date: Thu, Dec 8, 2016 at 11:56 AM<br>
Subject: Re: [Gluster-users] Replica brick not
working<br>
To: Ravishankar N <<a
moz-do-not-send="true"
href="mailto:ravishankar@redhat.com" target="_blank">ravishankar@redhat.com</a><br>
</span><span class="">
<mailto:<a moz-do-not-send="true"
href="mailto:ravishankar@redhat.com" target="_blank">ravishankar@redhat.com</a><wbr>>
<mailto:<a moz-do-not-send="true"
href="mailto:ravishankar@redhat.com" target="_blank">ravishankar@redhat.com</a><br>
<mailto:<a moz-do-not-send="true"
href="mailto:ravishankar@redhat.com" target="_blank">ravishankar@redhat.com</a><wbr>>>><br>
Cc: Miloš Čučulović - MDPI <<a
moz-do-not-send="true"
href="mailto:cuculovic@mdpi.com" target="_blank">cuculovic@mdpi.com</a><br>
<mailto:<a moz-do-not-send="true"
href="mailto:cuculovic@mdpi.com" target="_blank">cuculovic@mdpi.com</a>><br>
</span>
<mailto:<a moz-do-not-send="true"
href="mailto:cuculovic@mdpi.com" target="_blank">cuculovic@mdpi.com</a>
<mailto:<a moz-do-not-send="true"
href="mailto:cuculovic@mdpi.com" target="_blank">cuculovic@mdpi.com</a>>>>,<span
class=""><br>
Pranith Kumar Karampuri<br>
<<a moz-do-not-send="true"
href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>
<mailto:<a moz-do-not-send="true"
href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>><br>
</span>
<mailto:<a moz-do-not-send="true"
href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>
<mailto:<a moz-do-not-send="true"
href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>>>><wbr>,<br>
gluster-users<br>
<<a moz-do-not-send="true"
href="mailto:gluster-users@gluster.org"
target="_blank">gluster-users@gluster.org</a>
<mailto:<a moz-do-not-send="true"
href="mailto:gluster-users@gluster.org"
target="_blank">gluster-users@gluster.<wbr>org</a>><br>
<mailto:<a moz-do-not-send="true"
href="mailto:gluster-users@gluster.org"
target="_blank">gluster-users@gluster.<wbr>org</a><span
class=""><br>
<mailto:<a moz-do-not-send="true"
href="mailto:gluster-users@gluster.org"
target="_blank">gluster-users@gluster.<wbr>org</a>>>><br>
<br>
<br>
<br>
<br>
On Thu, Dec 8, 2016 at 11:11 AM, Ravishankar N<br>
<<a moz-do-not-send="true"
href="mailto:ravishankar@redhat.com" target="_blank">ravishankar@redhat.com</a>
<mailto:<a moz-do-not-send="true"
href="mailto:ravishankar@redhat.com" target="_blank">ravishankar@redhat.com</a><wbr>><br>
</span>
<mailto:<a moz-do-not-send="true"
href="mailto:ravishankar@redhat.com" target="_blank">ravishankar@redhat.com</a>
<mailto:<a moz-do-not-send="true"
href="mailto:ravishankar@redhat.com" target="_blank">ravishankar@redhat.com</a><wbr>>>>
<div>
<div class="h5"><br>
wrote:<br>
<br>
On 12/08/2016 10:43 AM, Atin Mukherjee
wrote:<br>
<br>
>From the log snippet:<br>
<br>
[2016-12-07 09:15:35.677645] I
[MSGID: 106482]<br>
[glusterd-brick-ops.c:442:__gl<wbr>usterd_handle_add_brick]<br>
0-management: Received add brick req<br>
[2016-12-07 09:15:35.677708] I
[MSGID: 106062]<br>
[glusterd-brick-ops.c:494:__gl<wbr>usterd_handle_add_brick]<br>
0-management: replica-count is 2<br>
[2016-12-07 09:15:35.677735] E
[MSGID: 106291]<br>
[glusterd-brick-ops.c:614:__gl<wbr>usterd_handle_add_brick]<br>
0-management:<br>
<br>
The last log entry indicates that we
hit the code path in<br>
gd_addbr_validate_replica_coun<wbr>t
()<br>
<br>
if (replica_count ==<br>
volinfo->replica_count) {<br>
if
(!(total_bricks %<br>
volinfo->dist_leaf_count)) {<br>
ret
= 1;<br>
goto
out;<br>
}<br>
}<br>
<br>
<br>
It seems unlikely that this snippet was
hit because we print<br>
the E<br>
[MSGID: 106291] in the above message
only if ret==-1.<br>
gd_addbr_validate_replica_coun<wbr>t()
returns -1 and yet not<br>
populates<br>
err_str only when in volinfo->type
doesn't match any of the<br>
known<br>
volume types, so volinfo->type is
corrupted perhaps?<br>
<br>
<br>
You are right, I missed that ret is set to 1
here in the above<br>
snippet.<br>
<br>
@Milos - Can you please provide us the
volume info file from<br>
/var/lib/glusterd/vols/<volnam<wbr>e>/
from all the three nodes to<br>
continue<br>
the analysis?<br>
<br>
<br>
<br>
-Ravi<br>
<br>
@Pranith, Ravi - Milos was trying to
convert a dist (1 X 1)<br>
volume to a replicate (1 X 2) using
add brick and hit<br>
this issue<br>
where add-brick failed. The cluster
is operating with 3.7.6.<br>
Could you help on what scenario this
code path can be<br>
hit? One<br>
straight forward issue I see here is
missing err_str in<br>
this path.<br>
<br>
<br>
<br>
<br>
<br>
<br>
--<br>
<br>
~ Atin (atinm)<br>
<br>
<br>
<br>
--<br>
<br>
~ Atin (atinm)<br>
<br>
<br>
<br>
<br>
--<br>
<br>
~ Atin (atinm)<br>
</div>
</div>
</blockquote>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div class="gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr"><br>
</div>
<div>~ Atin (atinm)<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<p><br>
</p>
</body>
</html>