<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Hi,<br>
<br>
<br>
<div class="moz-cite-prefix">Dne 17.8.2015 v 06:34 Ravishankar N
napsal(a):<br>
</div>
<blockquote cite="mid:55D16443.1080601@redhat.com" type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<br>
<br>
<div class="moz-cite-prefix">On 08/16/2015 04:22 PM, Miloš Kozák
wrote:<br>
</div>
<blockquote cite="mid:55D06B55.2070200@lejmr.com" type="cite">Hi,
I have been running an glusterfs for a while, and everything
works just fine even after one node failure.. However, I went
for brick replacement due to my bricks were not thin-provisioned
and I wanted to use snapshots. In short, whole volume went down
due to heal daemon which tool all IO and all VMs running on top
of that volume started to be unresponsive. <br>
<br>
In short, I am rebuilding the volume from scratch. I created new
thinly provisioned bricks: <br>
lvs: <br>
brick_s3-sata-10k vg_s3-sata-10k Vwi-aotz
931,25g s3-sata-10k_pool 2,95 <br>
s3-sata-10k_pool vg_s3-sata-10k twi-a-tz
931,25g <br>
<br>
vgs: <br>
vg_s3-sata-10k 1 3 0 wz--n- 931,51g 148,00m <br>
<br>
df: <br>
/dev/mapper/vg_s3--sata--10k-brick_s3--sata--10k 976009600
28383480 947626120 3% /gfs/s3-sata-10k <br>
<br>
and mounted. When I uploaded two images onto it I found there
might be a problem. For the time being I run the volume in
replica 2 mode on top of two servers. The files were copied from
node1, and I think the files are OK on node1 only. However, the
volume heal indicates everything is OK. </blockquote>
Hi,<br>
If I understand you correctly, are you saying that when you
created a new replica 2 volume with thin LVs for the bricks ,
mounted the volume (fuse?) and copied 2 files (VM images?) via
this mount, the files were being written to only node1?<br>
</blockquote>
<br>
Mounted by fuse. <br>
I would not say written to only one node because both nodes contain
something, but I am quite suspicious about files on the node2.<br>
<br>
<blockquote cite="mid:55D16443.1080601@redhat.com" type="cite"> <br>
-Ravi<br>
<blockquote cite="mid:55D06B55.2070200@lejmr.com" type="cite">My
symptoms are as follows: <br>
<br>
df information from both servers: <br>
/dev/mapper/vg_s3--sata--10k-brick_s3--sata--10k 976009600
30754296 945255304 4% /gfs/s3-sata-10k <br>
/dev/mapper/vg_s3--sata--10k-brick_s3--sata--10k 976009600
28383480 947626120 3% /gfs/s3-sata-10k <br>
<br>
<br>
<br>
[root@nodef01i ~]# du /gfs/s3-sata-10k/ <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/indices/xattrop <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/indices <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/changelogs/htime <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/changelogs/csnap <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/changelogs <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/00/00 <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/00 <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/landfill <br>
20480004 /gfs/s3-sata-10k/fs/.glusterfs/84/26 <br>
20480004 /gfs/s3-sata-10k/fs/.glusterfs/84 <br>
10240000 /gfs/s3-sata-10k/fs/.glusterfs/d0/ff <br>
10240000 /gfs/s3-sata-10k/fs/.glusterfs/d0 <br>
30720008 /gfs/s3-sata-10k/fs/.glusterfs <br>
30720008 /gfs/s3-sata-10k/fs <br>
30720008 /gfs/s3-sata-10k/ <br>
<br>
[root@nodef02i ~]# du /gfs/s3-sata-10k/ <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/indices/xattrop <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/indices <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/changelogs/htime <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/changelogs/csnap <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/changelogs <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/00/00 <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/00 <br>
0 /gfs/s3-sata-10k/fs/.glusterfs/landfill <br>
18727172 /gfs/s3-sata-10k/fs/.glusterfs/84/26 <br>
18727172 /gfs/s3-sata-10k/fs/.glusterfs/84 <br>
9622016 /gfs/s3-sata-10k/fs/.glusterfs/d0/ff <br>
9622016 /gfs/s3-sata-10k/fs/.glusterfs/d0 <br>
28349192 /gfs/s3-sata-10k/fs/.glusterfs <br>
28349192 /gfs/s3-sata-10k/fs <br>
28349192 /gfs/s3-sata-10k/ <br>
<br>
<br>
[root@nodef01i ~]# du /gfs/s3-sata-10k/fs/* <br>
20480004 /gfs/s3-sata-10k/fs/f1607f25aa52f4fb6f98f20ef0f3f9d7
<br>
10240000 /gfs/s3-sata-10k/fs/3706a2cb0bb27ba5787b3c12388f4ebb
<br>
<br>
[root@nodef02i ~]# du /gfs/s3-sata-10k/fs/* <br>
18727172 /gfs/s3-sata-10k/fs/f1607f25aa52f4fb6f98f20ef0f3f9d7
<br>
9622016
/gfs/s3-sata-10k/fs/3706a2cb0bb27ba5787b3c12388f4ebb <br>
<br>
<br>
[root@nodef01i ~]# ll /gfs/s3-sata-10k/fs/ <br>
celkem 30720004 <br>
-rw-r----- 2 oneadmin oneadmin 20971520512 3. srp 23.53
f1607f25aa52f4fb6f98f20ef0f3f9d7 <br>
-rw-r----- 2 oneadmin oneadmin 10485760000 16. srp 11.23
3706a2cb0bb27ba5787b3c12388f4ebb <br>
<br>
[root@nodef02i ~]# ll /gfs/s3-sata-10k/fs/ <br>
celkem 28349188 <br>
-rw-r----- 2 oneadmin oneadmin 20971520512 3. srp 23.53
f1607f25aa52f4fb6f98f20ef0f3f9d7 <br>
-rw-r----- 2 oneadmin oneadmin 10485760000 16. srp 11.22
3706a2cb0bb27ba5787b3c12388f4ebb <br>
<br>
<br>
[root@nodef01i ~]# gluster volume heal ph-fs-0 info split-brain
<br>
Gathering list of split brain entries on volume ph-fs-0 has been
successful <br>
<br>
Brick 10.11.100.1:/gfs/s3-sata-10k/fs <br>
Number of entries: 0 <br>
<br>
Brick 10.11.100.2:/gfs/s3-sata-10k/fs <br>
Number of entries: 0 <br>
<br>
<br>
[root@nodef01i ~]# gluster volume heal ph-fs-0 info <br>
Brick nodef01i.czprg:/gfs/s3-sata-10k/fs/ <br>
Number of entries: 0 <br>
<br>
Brick nodef02i.czprg:/gfs/s3-sata-10k/fs/ <br>
Number of entries: 0 <br>
<br>
<br>
[root@nodef01i ~]# gluster volume status <br>
Status of volume: ph-fs-0 <br>
Gluster process Port Online Pid <br>
------------------------------------------------------------------------------
<br>
Brick 10.11.100.1:/gfs/s3-sata-10k/fs 49152 Y
3733 <br>
Brick 10.11.100.2:/gfs/s3-sata-10k/fs 49152 Y
64711 <br>
NFS Server on localhost 2049 Y 3747 <br>
Self-heal Daemon on localhost N/A Y 3752 <br>
NFS Server on 10.11.100.2 2049 Y 64725 <br>
Self-heal Daemon on 10.11.100.2 N/A Y 64730
<br>
<br>
Task Status of Volume ph-fs-0 <br>
------------------------------------------------------------------------------
<br>
There are no active volume tasks <br>
<br>
[root@nodef02i ~]# gluster volume status <br>
Status of volume: ph-fs-0 <br>
Gluster process Port Online Pid <br>
------------------------------------------------------------------------------
<br>
Brick 10.11.100.1:/gfs/s3-sata-10k/fs 49152 Y
3733 <br>
Brick 10.11.100.2:/gfs/s3-sata-10k/fs 49152 Y
64711 <br>
NFS Server on localhost 2049 Y 64725 <br>
Self-heal Daemon on localhost N/A Y 64730 <br>
NFS Server on 10.11.100.1 2049 Y 3747 <br>
Self-heal Daemon on 10.11.100.1 N/A Y 3752
<br>
<br>
Task Status of Volume ph-fs-0 <br>
------------------------------------------------------------------------------
<br>
There are no active volume tasks <br>
<br>
<br>
[root@nodef02i ~]# rpm -qa | grep gluster <br>
glusterfs-server-3.6.2-1.el6.x86_64 <br>
glusterfs-3.6.2-1.el6.x86_64 <br>
glusterfs-api-3.6.2-1.el6.x86_64 <br>
glusterfs-libs-3.6.2-1.el6.x86_64 <br>
glusterfs-cli-3.6.2-1.el6.x86_64 <br>
glusterfs-fuse-3.6.2-1.el6.x86_64 <br>
<br>
What other information should I provide? <br>
<br>
Thanks Milos <br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</blockquote>
<br>
</body>
</html>