<div dir="ltr"><span style="color:rgb(0,0,0);font-size:14px;line-height:26px;font-family:sans,helvetica"><span style="white-space:pre">un</span></span><span style="color:rgb(0,0,0);font-size:14px;line-height:26px;font-family:sans,helvetica;white-space:pre">subscribe gluster</span><br></div><div class="gmail_extra"><br><div class="gmail_quote">2015-03-17 20:00 GMT+08:00 <span dir="ltr"><<a href="mailto:gluster-users-request@gluster.org" target="_blank">gluster-users-request@gluster.org</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Send Gluster-users mailing list submissions to<br>
<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:gluster-users-request@gluster.org">gluster-users-request@gluster.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:gluster-users-owner@gluster.org">gluster-users-owner@gluster.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of Gluster-users digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Re: Issue with removing directories after disabling quotas<br>
(Vijaikumar M)<br>
2. glusterfs geo-replication - server with two interfaces -<br>
private IP advertised (Stefan Moravcik)<br>
3. adding a node (Pierre L?onard)<br>
4. Re: adding a node (JF Le Fill?tre)<br>
5. Re: adding a node (Pierre L?onard)<br>
6. Re: adding a node (JF Le Fill?tre)<br>
7. tune2fs exited with non-zero exit status<br>
(Osborne, Paul (<a href="mailto:paul.osborne@canterbury.ac.uk">paul.osborne@canterbury.ac.uk</a>))<br>
8. Re: adding a node (Pierre L?onard)<br>
9. Re: adding a node (JF Le Fill?tre)<br>
10. Re: adding a node (Pierre L?onard)<br>
11. Gluster 3.6 issue (F?lix de Lelelis)<br>
12. I/O error on replicated volume (Jonathan Heese)<br>
13. Re: [ovirt-users] Zombie disks failed to remove.. (Punit Dambiwal)<br>
14. Running a SQL database on Gluster on AWS (Jay Strauss)<br>
15. Re: I/O error on replicated volume (Ravishankar N)<br>
16. VM failed to start | Bad volume specification (Punit Dambiwal)<br>
17. Re: Working but some issues (Melkor Lord)<br>
18. Re: Working but some issues (Joe Julian)<br>
19. Bad volume specification | Connection timeout (Punit Dambiwal)<br>
20. Re: adding a node (Pierre L?onard)<br>
21. REMINDER: Gluster Community Bug Triage meeting today at 12:00<br>
UTC (Niels de Vos)<br>
22. Re: adding a node (JF Le Fill?tre)<br>
23. Re: adding a node (Pierre L?onard)<br>
24. Re: Working but some issues (Nico Schottelius)<br>
25. GlusterFS 3.4.7beta2 is now available for testing (Kaleb KEITHLEY)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Mon, 16 Mar 2015 17:30:41 +0530<br>
From: Vijaikumar M <<a href="mailto:vmallika@redhat.com">vmallika@redhat.com</a>><br>
To: JF Le Fill?tre <<a href="mailto:jean-francois.lefillatre@uni.lu">jean-francois.lefillatre@uni.lu</a>>,<br>
<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
Subject: Re: [Gluster-users] Issue with removing directories after<br>
disabling quotas<br>
Message-ID: <<a href="mailto:5506C5E9.2070204@redhat.com">5506C5E9.2070204@redhat.com</a>><br>
Content-Type: text/plain; charset=utf-8; format=flowed<br>
<br>
<br>
On Monday 16 March 2015 05:04 PM, JF Le Fill?tre wrote:<br>
> Hello Vijay,<br>
><br>
> Mmmm sorry, I jumped the gun and deleted by hand all the files present<br>
> on the bricks, so I can't see if there's any link anywhere...<br>
><br>
> It's not the first time I have seen this. Very early in my tests I had<br>
> another case of files still present on the bricks but not visible in<br>
> Gluster, which I solved the same way. At that point I chalked it up to<br>
> my limited knowledge of GlusterFS and I assumed that I had misconfigured it.<br>
><br>
> What do you suspect that it may be, and what do I have to look for if it<br>
> ever happens again?<br>
We wanted to check file attributes and xattrs on the bricks to see why<br>
some files where not deleted.<br>
Is this problem easily re-creatable? If yes, can you please provide the<br>
test-case?<br>
<br>
Thanks,<br>
Vijay<br>
<br>
<br>
> Thanks!<br>
> JF<br>
><br>
><br>
><br>
> On 16/03/15 12:25, Vijaikumar M wrote:<br>
>> Hi JF,<br>
>><br>
>> This may not be a quota issue. Can you please check if there are any<br>
>> linkto files exists on the brick?<br>
>> On node 'stor104/106', can we get the below output<br>
>><br>
>> #find /zfs/brick0/brick | xargs ls -l<br>
>> #find /zfs/brick1/brick | xargs ls -l<br>
>> #find /zfs/brick2/brick | xargs ls -l<br>
>><br>
>> #find /zfs/brick0/brick | xargs getfattr -d -m . -e hex<br>
>> #find /zfs/brick1/brick | xargs getfattr -d -m . -e hex<br>
>> #find /zfs/brick2/brick | xargs getfattr -d -m . -e hex<br>
>><br>
>><br>
>> Thanks,<br>
>> Vijay<br>
>><br>
>><br>
>><br>
>> On Monday 16 March 2015 04:18 PM, JF Le Fill?tre wrote:<br>
>>> Forgot to mention: the non-empty directories list files like this:<br>
>>><br>
>>> -?????????? ? ? ? ? ? Hal8723APhyReg.h<br>
>>> -?????????? ? ? ? ? ? Hal8723UHWImg_CE.h<br>
>>> -?????????? ? ? ? ? ? hal_com.h<br>
>>> -?????????? ? ? ? ? ? HalDMOutSrc8723A.h<br>
>>> -?????????? ? ? ? ? ? HalHWImg8723A_BB.h<br>
>>> -?????????? ? ? ? ? ? HalHWImg8723A_FW.h<br>
>>> -?????????? ? ? ? ? ? HalHWImg8723A_MAC.h<br>
>>> -?????????? ? ? ? ? ? HalHWImg8723A_RF.h<br>
>>> -?????????? ? ? ? ? ? hal_intf.h<br>
>>> -?????????? ? ? ? ? ? HalPwrSeqCmd.h<br>
>>> -?????????? ? ? ? ? ? ieee80211.h<br>
>>> -?????????? ? ? ? ? ? odm_debug.h<br>
>>><br>
>>> Thanks,<br>
>>> JF<br>
>>><br>
>>><br>
>>> On 16/03/15 11:45, JF Le Fill?tre wrote:<br>
>>>> Hello all,<br>
>>>><br>
>>>> So, another day another issue. I was trying to play with quotas on my<br>
>>>> volume:<br>
>>>><br>
>>>> ================================================================================<br>
>>>><br>
>>>> [root@stor104 ~]# gluster volume status<br>
>>>> Status of volume: live<br>
>>>> Gluster process Port Online Pid<br>
>>>> ------------------------------------------------------------------------------<br>
>>>><br>
>>>> Brick stor104:/zfs/brick0/brick 49167 Y 13446<br>
>>>> Brick stor104:/zfs/brick1/brick 49168 Y 13457<br>
>>>> Brick stor104:/zfs/brick2/brick 49169 Y 13468<br>
>>>> Brick stor106:/zfs/brick0/brick 49159 Y 14158<br>
>>>> Brick stor106:/zfs/brick1/brick 49160 Y 14169<br>
>>>> Brick stor106:/zfs/brick2/brick 49161 Y 14180<br>
>>>> NFS Server on localhost 2049 Y 13483<br>
>>>> Quota Daemon on localhost N/A Y 13490<br>
>>>> NFS Server on stor106 2049 Y 14195<br>
>>>> Quota Daemon on stor106 N/A Y 14202<br>
>>>> Task Status of Volume live<br>
>>>> ------------------------------------------------------------------------------<br>
>>>><br>
>>>> Task : Rebalance<br>
>>>> ID : 6bd03709-1f48-49a9-a215-d0a6e6f3ab1e<br>
>>>> Status : completed<br>
>>>> ================================================================================<br>
>>>><br>
>>>><br>
>>>><br>
>>>> Not sure if the "Quota Daemon on localhost -> N/A" is normal, but<br>
>>>> that's another topic.<br>
>>>><br>
>>>> While the quotas were enabled, I did some test copying a whole tree<br>
>>>> of small files (the Linux kernel sources) to the volume to see which<br>
>>>> performance I would get, and it was really low. So I decided to<br>
>>>> disable quotas again:<br>
>>>><br>
>>>><br>
>>>> ================================================================================<br>
>>>><br>
>>>> [root@stor104 ~]# gluster volume status<br>
>>>> Status of volume: live<br>
>>>> Gluster process Port Online Pid<br>
>>>> ------------------------------------------------------------------------------<br>
>>>><br>
>>>> Brick stor104:/zfs/brick0/brick 49167 Y 13754<br>
>>>> Brick stor104:/zfs/brick1/brick 49168 Y 13765<br>
>>>> Brick stor104:/zfs/brick2/brick 49169 Y 13776<br>
>>>> Brick stor106:/zfs/brick0/brick 49159 Y 14282<br>
>>>> Brick stor106:/zfs/brick1/brick 49160 Y 14293<br>
>>>> Brick stor106:/zfs/brick2/brick 49161 Y 14304<br>
>>>> NFS Server on localhost 2049 Y 13790<br>
>>>> NFS Server on stor106 2049 Y 14319<br>
>>>> Task Status of Volume live<br>
>>>> ------------------------------------------------------------------------------<br>
>>>><br>
>>>> Task : Rebalance<br>
>>>> ID : 6bd03709-1f48-49a9-a215-d0a6e6f3ab1e<br>
>>>> Status : completed<br>
>>>> ================================================================================<br>
>>>><br>
>>>><br>
>>>><br>
>>>> I remounted the volume from the client and tried deleting the<br>
>>>> directory containing the sources, which gave me a very long list of<br>
>>>> this:<br>
>>>><br>
>>>><br>
>>>> ================================================================================<br>
>>>><br>
>>>> rm: cannot remove<br>
>>>> ?/glusterfs/live/linux-3.18.7/tools/testing/selftests/ftrace/test.d/kprobe?:<br>
>>>> Directory not empty<br>
>>>> rm: cannot remove<br>
>>>> ?/glusterfs/live/linux-3.18.7/tools/testing/selftests/ptrace?:<br>
>>>> Directory not empty<br>
>>>> rm: cannot remove<br>
>>>> ?/glusterfs/live/linux-3.18.7/tools/testing/selftests/rcutorture/configs/rcu/v0.0?:<br>
>>>> Directory not empty<br>
>>>> rm: cannot remove<br>
>>>> ?/glusterfs/live/linux-3.18.7/tools/testing/selftests/rcutorture/configs/rcu/v3.5?:<br>
>>>> Directory not empty<br>
>>>> rm: cannot remove<br>
>>>> ?/glusterfs/live/linux-3.18.7/tools/testing/selftests/powerpc?:<br>
>>>> Directory not empty<br>
>>>> rm: cannot remove<br>
>>>> ?/glusterfs/live/linux-3.18.7/tools/perf/scripts/python/Perf-Trace-Util?:<br>
>>>> Directory not empty<br>
>>>> rm: cannot remove<br>
>>>> ?/glusterfs/live/linux-3.18.7/tools/perf/Documentation?: Directory<br>
>>>> not empty<br>
>>>> rm: cannot remove ?/glusterfs/live/linux-3.18.7/tools/perf/ui/tui?:<br>
>>>> Directory not empty<br>
>>>> rm: cannot remove<br>
>>>> ?/glusterfs/live/linux-3.18.7/tools/perf/util/include?: Directory not<br>
>>>> empty<br>
>>>> rm: cannot remove ?/glusterfs/live/linux-3.18.7/tools/lib?: Directory<br>
>>>> not empty<br>
>>>> rm: cannot remove ?/glusterfs/live/linux-3.18.7/tools/virtio?:<br>
>>>> Directory not empty<br>
>>>> rm: cannot remove ?/glusterfs/live/linux-3.18.7/virt/kvm?: Directory<br>
>>>> not empty<br>
>>>> ================================================================================<br>
>>>><br>
>>>><br>
>>>><br>
>>>> I did my homework on Google, yet the information I found was that the<br>
>>>> cause for this is that the contents of the bricks have been modified<br>
>>>> locally. It's definitely not the case here, I have *not* touched the<br>
>>>> contents of the bricks.<br>
>>>><br>
>>>> So my question is: is it possible that disabling the quotas may have<br>
>>>> had some side effects on the metadata of Gluster? If so, what can I<br>
>>>> do to force Gluster to rescan all local directories and "import"<br>
>>>> local files?<br>
>>>><br>
>>>> GlusterFS version: 3.6.2<br>
>>>><br>
>>>> The setup of my volume:<br>
>>>><br>
>>>><br>
>>>> ================================================================================<br>
>>>><br>
>>>> [root@stor104 ~]# gluster volume info<br>
>>>> Volume Name: live<br>
>>>> Type: Distribute<br>
>>>> Volume ID:<br>
>>>> Status: Started<br>
>>>> Number of Bricks: 6<br>
>>>> Transport-type: tcp<br>
>>>> Bricks:<br>
>>>> Brick1: stor104:/zfs/brick0/brick<br>
>>>> Brick2: stor104:/zfs/brick1/brick<br>
>>>> Brick3: stor104:/zfs/brick2/brick<br>
>>>> Brick4: stor106:/zfs/brick0/brick<br>
>>>> Brick5: stor106:/zfs/brick1/brick<br>
>>>> Brick6: stor106:/zfs/brick2/brick<br>
>>>> Options Reconfigured:<br>
>>>> features.quota: off<br>
>>>> performance.readdir-ahead: on<br>
>>>> nfs.volume-access: read-only<br>
>>>> cluster.data-self-heal-algorithm: full<br>
>>>> performance.strict-write-ordering: off<br>
>>>> performance.strict-o-direct: off<br>
>>>> performance.force-readdirp: off<br>
>>>> performance.write-behind-window-size: 4MB<br>
>>>> performance.io-thread-count: 32<br>
>>>> performance.flush-behind: on<br>
>>>> performance.client-io-threads: on<br>
>>>> performance.cache-size: 32GB<br>
>>>> performance.cache-refresh-timeout: 60<br>
>>>> performance.cache-max-file-size: 4MB<br>
>>>> nfs.disable: off<br>
>>>> cluster.eager-lock: on<br>
>>>> cluster.min-free-disk: 1%<br>
>>>> server.allow-insecure: on<br>
>>>> diagnostics.client-log-level: ERROR<br>
>>>> diagnostics.brick-log-level: ERROR<br>
>>>> ================================================================================<br>
>>>><br>
>>>><br>
>>>><br>
>>>> It is mounted from the client with those fstab options:<br>
>>>><br>
>>>> ================================================================================<br>
>>>><br>
>>>> stor104:live<br>
>>>> defaults,backupvolfile-server=stor106,direct-io-mode=disable,noauto<br>
>>>> ================================================================================<br>
>>>><br>
>>>><br>
>>>> Attached are the log files from stor104<br>
>>>><br>
>>>> Thanks a lot for any help!<br>
>>>> JF<br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> _______________________________________________<br>
>>>> Gluster-users mailing list<br>
>>>> <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
>>>> <a href="http://www.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
>>>><br>
<br>
<br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Mon, 16 Mar 2015 13:10:48 +0100<br>
From: Stefan Moravcik <<a href="mailto:smoravcik@newsweaver.com">smoravcik@newsweaver.com</a>><br>
To: <a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
Subject: [Gluster-users] glusterfs geo-replication - server with two<br>
interfaces - private IP advertised<br>
Message-ID: <<a href="mailto:D51E5B92-87A6-4213-AFBD-04C3207E6E34@newsweaver.com">D51E5B92-87A6-4213-AFBD-04C3207E6E34@newsweaver.com</a>><br>
Content-Type: text/plain; charset="iso-8859-1"<br>
<br>
I have been trying to setup a geo replication with glusterfs servers. Everything worked as expected in my test environment, on my staging environment, but then i tried the production and got stuck.<br>
<br>
Let say I have<br>
<br>
gluster fs server is on public ip 1.1.1.1<br>
<br>
gluster fs slave is on public 2.2.2.2, but this IP is on interface eth1 The eth0 on gluster fs slave server is 192.168.0.1.<br>
<br>
So when i start the command on 1.1.1.1 (firewall and ssh keys are set properly)<br>
<br>
gluster volume geo-replication vol0 2.2.2.2::vol0 create push-pem<br>
I get an error.<br>
<br>
Unable to fetch slave volume details. Please check the slave cluster and slave volume. geo-replication command failed<br>
<br>
The error is not that important in this case, the problem is the slave IP address<br>
<br>
2015-03-16T11:41:08.101229+00:00 xxx kernel: TCP LOGDROP: IN= OUT=eth0 SRC=1.1.1.1 DST=192.168.0.1 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=24243 DF PROTO=TCP SPT=1015 DPT=24007 WINDOW=14600 RES=0x00 SYN URGP=0<br>
As you can see in the firewall drop log above, the port 24007 of the slave gluster daemon is advertised on private IP of the interface eth0 on slave server and should be the IP of the eth1 private IP. So master cannot connect and will time out<br>
<br>
Is there a way to force gluster server to advertise interface eth1 or bind to it only?<br>
<br>
Thank you<br>
--<br>
*******************************************************************************************************************************************************<br>
********************<br>
This email and any files transmitted with it are confidential and intended<br>
solely for the use of the individual or entity to whom they are addressed.<br>
If you have received this email in error please reply to the sender<br>
indicating that fact and delete the copy you received.<br>
In addition, if you are not the intended recipient, you should not print,<br>
copy, retransmit, disseminate, or otherwise use the information<br>
contained in this communication. Thank you.<br>
<br>
Newsweaver is a Trade Mark of E-Search Ltd. Registered in Ireland No.<br>
254994.<br>
Registered Office: 2200 Airport Business Park, Kinsale Road, Cork, Ireland.<br>
International Telephone Number: +353 21 2427277.<br>
*******************************************************************************************************************************************************<br>
********************<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150316/a0f25019/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150316/a0f25019/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 3<br>
Date: Mon, 16 Mar 2015 16:17:04 +0100<br>
From: Pierre L?onard <<a href="mailto:pleonard@jouy.inra.fr">pleonard@jouy.inra.fr</a>><br>
To: "<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>" <<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>><br>
Subject: [Gluster-users] adding a node<br>
Message-ID: <<a href="mailto:5506F3F0.40803@jouy.inra.fr">5506F3F0.40803@jouy.inra.fr</a>><br>
Content-Type: text/plain; charset="utf-8"; Format="flowed"<br>
<br>
Hi All,<br>
<br>
I am note the alone guy having problem with the addition of a node to an<br>
existing distributed stripped or replicated volume.<br>
And I never read a real solution.<br>
<br>
My configuration is the following :<br>
<br>
* four node with a gvHome stripped 2 volume ;<br>
* first I probe the new node ans its OK ;<br>
* then I start to add the node:/brick to the four other and get the<br>
answer that I have to add a multiple of the striped so 4 other node.<br>
But I have only one ;<br>
* So I try to make the addition from the new node. If I list the<br>
volume on the new node it recognize the stripped volume. So I stop<br>
the volume on the four and add : gluster volume add-brick gvHome<br>
stripe 2 boogy1:/mnt/gvHome/brick1 boogy2:/mnt/gvHome/brick1<br>
boogy3:/mnt/gvHome/brick1 boogy4:/mnt/gvHome/brick1 the answer is<br>
the following :<br>
<br>
[root@moody ~]# gluster volume add-brick gvHome stripe 2<br>
boogy1:/mnt/gvHome/brick1 boogy2:/mnt/gvHome/brick1<br>
boogy3:/mnt/gvHome/brick1 boogy4:/mnt/gvHome/brick1<br>
Changing the 'stripe count' of the volume is not a supported feature. In<br>
some cases it may result in data loss on the volume. Also there may be<br>
issues with regular filesystem operations on the volume after the<br>
change. Do you really want to continue with 'stripe' count option ? (y/n) y<br>
volume add-brick: failed: Staging failed on boogy2. Error: Brick:<br>
boogy2:/mnt/gvHome/brick1 not available. Brick may be containing or be<br>
contained by an existing brick<br>
Staging failed on boogy3. Error: Brick: boogy3:/mnt/gvHome/brick1 not<br>
available. Brick may be containing or be contained by an existing brick<br>
Staging failed on boogy1. Error: Brick: boogy1:/mnt/gvHome/brick1 not<br>
available. Brick may be containing or be contained by an existing brick<br>
Staging failed on 138.102.172.94. Error: Brick:<br>
boogy4:/mnt/gvHome/brick1 not available. Brick may be containing or be<br>
contained by an existing brick<br>
<br>
<br>
So how to add node one by one ?<br>
<br>
Many Thank's<br>
<br>
sincerely<br>
<br>
--<br>
Signature electronique<br>
INRA <<a href="http://www.inra.fr" target="_blank">http://www.inra.fr</a>><br>
*Pierre L?onard*<br>
*Senior IT Manager*<br>
*MetaGenoPolis*<br>
<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a> <mailto:<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a>><br>
T?l. : <a href="tel:%2B33%20%280%291%2034%2065%2029%2078" value="+33134652978">+33 (0)1 34 65 29 78</a><br>
<br>
Centre de recherche INRA<br>
Domaine de Vilvert ? B?t. 325 R+1<br>
78 352 Jouy-en-Josas CEDEX<br>
France<br>
<a href="http://www.mgps.eu" target="_blank">www.mgps.eu</a> <<a href="http://www.mgps.eu" target="_blank">http://www.mgps.eu</a>><br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150316/11707f48/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150316/11707f48/attachment-0001.html</a>><br>
-------------- next part --------------<br>
A non-text attachment was scrubbed...<br>
Name: logo-INRA2013.gif<br>
Type: image/gif<br>
Size: 2560 bytes<br>
Desc: <a href="http://www.inra.fr/" target="_blank">http://www.inra.fr/</a><br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150316/11707f48/attachment-0001.gif" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150316/11707f48/attachment-0001.gif</a>><br>
<br>
------------------------------<br>
<br>
Message: 4<br>
Date: Mon, 16 Mar 2015 16:28:44 +0100<br>
From: JF Le Fill?tre <<a href="mailto:jean-francois.lefillatre@uni.lu">jean-francois.lefillatre@uni.lu</a>><br>
To: <<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>><br>
Subject: Re: [Gluster-users] adding a node<br>
Message-ID: <<a href="mailto:5506F6AC.3060907@uni.lu">5506F6AC.3060907@uni.lu</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
<br>
Hello,<br>
<br>
You cannot. As the message said, you cannot change the stripe count. So<br>
you can add 4 bricks and you have a distributed + striped volume, but<br>
you cannot change the stripe count by re-adding existing bricks into the<br>
volume with a different stripe count, or by adding an additional brick<br>
to the volume.<br>
<br>
Your email isn't very clear; could you please send:<br>
- the list of commands in the order that you typed them;<br>
- the output of "gluster volume info" and "gluster volume status"<br>
<br>
Thanks!<br>
JF<br>
<br>
<br>
<br>
On 16/03/15 16:17, Pierre L?onard wrote:<br>
> Hi All,<br>
><br>
> I am note the alone guy having problem with the addition of a node to an<br>
> existing distributed stripped or replicated volume.<br>
> And I never read a real solution.<br>
><br>
> My configuration is the following :<br>
><br>
> * four node with a gvHome stripped 2 volume ;<br>
> * first I probe the new node ans its OK ;<br>
> * then I start to add the node:/brick to the four other and get the<br>
> answer that I have to add a multiple of the striped so 4 other node.<br>
> But I have only one ;<br>
> * So I try to make the addition from the new node. If I list the<br>
> volume on the new node it recognize the stripped volume. So I stop<br>
> the volume on the four and add : gluster volume add-brick gvHome<br>
> stripe 2 boogy1:/mnt/gvHome/brick1 boogy2:/mnt/gvHome/brick1<br>
> boogy3:/mnt/gvHome/brick1 boogy4:/mnt/gvHome/brick1 the answer is<br>
> the following :<br>
><br>
> [root@moody ~]# gluster volume add-brick gvHome stripe 2<br>
> boogy1:/mnt/gvHome/brick1 boogy2:/mnt/gvHome/brick1<br>
> boogy3:/mnt/gvHome/brick1 boogy4:/mnt/gvHome/brick1<br>
> Changing the 'stripe count' of the volume is not a supported feature. In<br>
> some cases it may result in data loss on the volume. Also there may be<br>
> issues with regular filesystem operations on the volume after the<br>
> change. Do you really want to continue with 'stripe' count option ? (y/n) y<br>
> volume add-brick: failed: Staging failed on boogy2. Error: Brick:<br>
> boogy2:/mnt/gvHome/brick1 not available. Brick may be containing or be<br>
> contained by an existing brick<br>
> Staging failed on boogy3. Error: Brick: boogy3:/mnt/gvHome/brick1 not<br>
> available. Brick may be containing or be contained by an existing brick<br>
> Staging failed on boogy1. Error: Brick: boogy1:/mnt/gvHome/brick1 not<br>
> available. Brick may be containing or be contained by an existing brick<br>
> Staging failed on 138.102.172.94. Error: Brick:<br>
> boogy4:/mnt/gvHome/brick1 not available. Brick may be containing or be<br>
> contained by an existing brick<br>
><br>
><br>
> So how to add node one by one ?<br>
><br>
> Many Thank's<br>
><br>
> sincerely<br>
><br>
> --<br>
> Signature electronique<br>
> INRA <<a href="http://www.inra.fr" target="_blank">http://www.inra.fr</a>><br>
><br>
> *Pierre L?onard*<br>
> *Senior IT Manager*<br>
> *MetaGenoPolis*<br>
> <a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a> <mailto:<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a>><br>
> T?l. : <a href="tel:%2B33%20%280%291%2034%2065%2029%2078" value="+33134652978">+33 (0)1 34 65 29 78</a><br>
><br>
> Centre de recherche INRA<br>
> Domaine de Vilvert ? B?t. 325 R+1<br>
> 78 352 Jouy-en-Josas CEDEX<br>
> France<br>
> <a href="http://www.mgps.eu" target="_blank">www.mgps.eu</a> <<a href="http://www.mgps.eu" target="_blank">http://www.mgps.eu</a>><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Gluster-users mailing list<br>
> <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> <a href="http://www.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
><br>
<br>
--<br>
<br>
Jean-Fran?ois Le Fill?tre<br>
-------------------------------<br>
HPC Systems Administrator<br>
LCSB - University of Luxembourg<br>
-------------------------------<br>
PGP KeyID 0x134657C6<br>
<br>
<br>
------------------------------<br>
<br>
Message: 5<br>
Date: Mon, 16 Mar 2015 16:56:45 +0100<br>
From: Pierre L?onard <<a href="mailto:pleonard@jouy.inra.fr">pleonard@jouy.inra.fr</a>><br>
To: JF Le Fill?tre <<a href="mailto:jean-francois.lefillatre@uni.lu">jean-francois.lefillatre@uni.lu</a>>,<br>
"<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>" <<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>><br>
Subject: Re: [Gluster-users] adding a node<br>
Message-ID: <<a href="mailto:5506FD3D.1080200@jouy.inra.fr">5506FD3D.1080200@jouy.inra.fr</a>><br>
Content-Type: text/plain; charset="utf-8"; Format="flowed"<br>
<br>
Hi JF Le Fill?tre,<br>
<br>
Hello,<br>
<br>
You cannot. As the message said, you cannot change the stripe count.<br>
<br>
I don't change the strip count, the number is the same.<br>
<br>
So<br>
you can add 4 bricks and you have a distributed + striped volume, but<br>
you cannot change the stripe count by re-adding existing bricks into the<br>
volume with a different stripe count, or by adding an additional brick<br>
to the volume.<br>
<br>
That is the real question : "is it possible to add a node to a stripe volume one by one ?"<br>
I not when I receive a new node I can't insert it in the global cluster service !<br>
<br>
<br>
Your email isn't very clear; could you please send:<br>
- the list of commands in the order that you typed them;<br>
- the output of "gluster volume info" and "gluster volume status"<br>
<br>
On one of the four node :<br>
[root@boogy2 ~]# gluster volume stop gvHome<br>
<br>
on the new node :<br>
[root@moody gvHome]# gluster volume add-brick gvHome moody:/mnt/gvHome/brick1<br>
volume add-brick: failed: Incorrect number of bricks supplied 1 with count 2<br>
<br>
[root@boogy2 ~]# gluster volume info gvHome<br>
<br>
Volume Name: gvHome<br>
Type: Distributed-Stripe<br>
Volume ID: aa7bc5be-e6f6-43b0-bba2-b9d51ed2e4ef<br>
Status: Started<br>
Number of Bricks: 2 x 2 = 4<br>
Transport-type: tcp<br>
Bricks:<br>
Brick1: boogy1:/mnt/gvHome/brick1<br>
Brick2: boogy2:/mnt/gvHome/brick1<br>
Brick3: boogy3:/mnt/gvHome/brick1<br>
Brick4: boogy4:/mnt/gvHome/brick1<br>
<br>
<br>
[root@boogy2 ~]# gluster volume status gvHome<br>
Status of volume: gvHome<br>
Gluster process Port Online Pid<br>
------------------------------------------------------------------------------<br>
Brick boogy1:/mnt/gvHome/brick1 49153 Y 21735<br>
Brick boogy2:/mnt/gvHome/brick1 49153 Y 121607<br>
Brick boogy3:/mnt/gvHome/brick1 49153 Y 26828<br>
Brick boogy4:/mnt/gvHome/brick1 49153 Y 116839<br>
NFS Server on localhost N/A N N/A<br>
NFS Server on moody N/A N N/A<br>
NFS Server on 138.102.172.94 2049 Y 116853<br>
NFS Server on boogy1 2049 Y 21749<br>
NFS Server on boogy3 2049 Y 26843<br>
<br>
Task Status of Volume gvHome<br>
------------------------------------------------------------------------------<br>
There are no active volume tasks<br>
<br>
<br>
<br>
[root@boogy2 ~]# glusterd -V<br>
glusterfs 3.6.1 built on Nov 7 2014 15:15:48<br>
<br>
<br>
that's all.<br>
<br>
<br>
Many thank's<br>
<br>
sincereley<br>
<br>
--<br>
Signature electronique<br>
INRA <<a href="http://www.inra.fr" target="_blank">http://www.inra.fr</a>><br>
*Pierre L?onard*<br>
*Senior IT Manager*<br>
*MetaGenoPolis*<br>
<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a> <mailto:<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a>><br>
T?l. : <a href="tel:%2B33%20%280%291%2034%2065%2029%2078" value="+33134652978">+33 (0)1 34 65 29 78</a><br>
<br>
Centre de recherche INRA<br>
Domaine de Vilvert ? B?t. 325 R+1<br>
78 352 Jouy-en-Josas CEDEX<br>
France<br>
<a href="http://www.mgps.eu" target="_blank">www.mgps.eu</a> <<a href="http://www.mgps.eu" target="_blank">http://www.mgps.eu</a>><br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150316/526d34d5/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150316/526d34d5/attachment-0001.html</a>><br>
-------------- next part --------------<br>
A non-text attachment was scrubbed...<br>
Name: logo-INRA2013.gif<br>
Type: image/gif<br>
Size: 2560 bytes<br>
Desc: <a href="http://www.inra.fr/" target="_blank">http://www.inra.fr/</a><br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150316/526d34d5/attachment-0001.gif" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150316/526d34d5/attachment-0001.gif</a>><br>
<br>
------------------------------<br>
<br>
Message: 6<br>
Date: Mon, 16 Mar 2015 17:16:47 +0100<br>
From: JF Le Fill?tre <<a href="mailto:jean-francois.lefillatre@uni.lu">jean-francois.lefillatre@uni.lu</a>><br>
To: Pierre L?onard <<a href="mailto:pleonard@jouy.inra.fr">pleonard@jouy.inra.fr</a>>,<br>
"<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>" <<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>><br>
Subject: Re: [Gluster-users] adding a node<br>
Message-ID: <<a href="mailto:550701EF.4040101@uni.lu">550701EF.4040101@uni.lu</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
<br>
Hi,<br>
<br>
OK, I was confused because in your original email you mentioned "a<br>
multiple of the striped so 4 other node".<br>
<br>
So yes your stripe count is 2, so you have to add bricks 2 by 2, as the<br>
error message says. If you didn't, then it would mean that you would<br>
distribute your data over two 2-brick stripes, and one 1-brick single<br>
volume, which I am reasonably sure wouldn't work.<br>
<br>
If you want to be able to add bricks one by one, you would need a<br>
regular distributed volume, without:<br>
- striping (requires adding as many drives as stripe count)<br>
- replication (requires adding as many drives as stripe replication count)<br>
- dispersion (requires adding as many drives as whatever your dispersion<br>
disk count is)<br>
<br>
The only two things that you can extend is the number of brick groups in<br>
a distributed volume, with a similar brick group, or the number of<br>
replicas (but I'm sure that there are corner cases where you can't do<br>
it, I haven't tried them yet).<br>
<br>
Is that clearer now?<br>
<br>
Thanks!<br>
JF<br>
<br>
<br>
<br>
On 16/03/15 16:56, Pierre L?onard wrote:<br>
> Hi JF Le Fill?tre,<br>
><br>
> Hello,<br>
><br>
> You cannot. As the message said, you cannot change the stripe count.<br>
><br>
> I don't change the strip count, the number is the same.<br>
><br>
> So<br>
> you can add 4 bricks and you have a distributed + striped volume, but<br>
> you cannot change the stripe count by re-adding existing bricks into the<br>
> volume with a different stripe count, or by adding an additional brick<br>
> to the volume.<br>
><br>
> That is the real question : "is it possible to add a node to a stripe volume one by one ?"<br>
> I not when I receive a new node I can't insert it in the global cluster service !<br>
><br>
><br>
> Your email isn't very clear; could you please send:<br>
> - the list of commands in the order that you typed them;<br>
> - the output of "gluster volume info" and "gluster volume status"<br>
><br>
> On one of the four node :<br>
> [root@boogy2 ~]# gluster volume stop gvHome<br>
><br>
> on the new node :<br>
> [root@moody gvHome]# gluster volume add-brick gvHome moody:/mnt/gvHome/brick1<br>
> volume add-brick: failed: Incorrect number of bricks supplied 1 with count 2<br>
><br>
> [root@boogy2 ~]# gluster volume info gvHome<br>
><br>
> Volume Name: gvHome<br>
> Type: Distributed-Stripe<br>
> Volume ID: aa7bc5be-e6f6-43b0-bba2-b9d51ed2e4ef<br>
> Status: Started<br>
> Number of Bricks: 2 x 2 = 4<br>
> Transport-type: tcp<br>
> Bricks:<br>
> Brick1: boogy1:/mnt/gvHome/brick1<br>
> Brick2: boogy2:/mnt/gvHome/brick1<br>
> Brick3: boogy3:/mnt/gvHome/brick1<br>
> Brick4: boogy4:/mnt/gvHome/brick1<br>
><br>
><br>
> [root@boogy2 ~]# gluster volume status gvHome<br>
> Status of volume: gvHome<br>
> Gluster process Port Online Pid<br>
> ------------------------------------------------------------------------------<br>
> Brick boogy1:/mnt/gvHome/brick1 49153 Y 21735<br>
> Brick boogy2:/mnt/gvHome/brick1 49153 Y 121607<br>
> Brick boogy3:/mnt/gvHome/brick1 49153 Y 26828<br>
> Brick boogy4:/mnt/gvHome/brick1 49153 Y 116839<br>
> NFS Server on localhost N/A N N/A<br>
> NFS Server on moody N/A N N/A<br>
> NFS Server on 138.102.172.94 2049 Y 116853<br>
> NFS Server on boogy1 2049 Y 21749<br>
> NFS Server on boogy3 2049 Y 26843<br>
><br>
> Task Status of Volume gvHome<br>
> ------------------------------------------------------------------------------<br>
> There are no active volume tasks<br>
><br>
><br>
><br>
> [root@boogy2 ~]# glusterd -V<br>
> glusterfs 3.6.1 built on Nov 7 2014 15:15:48<br>
><br>
><br>
> that's all.<br>
><br>
><br>
> Many thank's<br>
><br>
> sincereley<br>
><br>
> --<br>
> Signature electronique<br>
> INRA <<a href="http://www.inra.fr" target="_blank">http://www.inra.fr</a>><br>
><br>
> *Pierre L?onard*<br>
> *Senior IT Manager*<br>
> *MetaGenoPolis*<br>
> <a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a> <mailto:<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a>><br>
> T?l. : <a href="tel:%2B33%20%280%291%2034%2065%2029%2078" value="+33134652978">+33 (0)1 34 65 29 78</a><br>
><br>
> Centre de recherche INRA<br>
> Domaine de Vilvert ? B?t. 325 R+1<br>
> 78 352 Jouy-en-Josas CEDEX<br>
> France<br>
> <a href="http://www.mgps.eu" target="_blank">www.mgps.eu</a> <<a href="http://www.mgps.eu" target="_blank">http://www.mgps.eu</a>><br>
><br>
<br>
--<br>
<br>
Jean-Fran?ois Le Fill?tre<br>
-------------------------------<br>
HPC Systems Administrator<br>
LCSB - University of Luxembourg<br>
-------------------------------<br>
PGP KeyID 0x134657C6<br>
<br>
<br>
------------------------------<br>
<br>
Message: 7<br>
Date: Mon, 16 Mar 2015 16:22:03 +0000<br>
From: "Osborne, Paul (<a href="mailto:paul.osborne@canterbury.ac.uk">paul.osborne@canterbury.ac.uk</a>)"<br>
<<a href="mailto:paul.osborne@canterbury.ac.uk">paul.osborne@canterbury.ac.uk</a>><br>
To: "<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>" <<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>><br>
Subject: [Gluster-users] tune2fs exited with non-zero exit status<br>
Message-ID:<br>
<<a href="mailto:AM3PR06MB1160480D807B23B1660DEE9BB020@AM3PR06MB116.eurprd06.prod.outlook.com">AM3PR06MB1160480D807B23B1660DEE9BB020@AM3PR06MB116.eurprd06.prod.outlook.com</a>><br>
<br>
Content-Type: text/plain; charset="us-ascii"<br>
<br>
Hi,<br>
<br>
I am just looking through my logs and am seeing a lot of entries of the form:<br>
<br>
[2015-03-16 16:02:55.553140] I [glusterd-handler.c:3530:__glusterd_handle_status_volume] 0-management: Received status volume req for volume wiki<br>
[2015-03-16 16:02:55.561173] E [glusterd-utils.c:5140:glusterd_add_inode_size_to_dict] 0-management: tune2fs exited with non-zero exit status<br>
[2015-03-16 16:02:55.561204] E [glusterd-utils.c:5166:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size<br>
<br>
Having had a rummage I *suspect* it is because gluster is trying to get the volume status by querying the superblock on the filesystem for a brick volume. However this is an issue as when the volume was created it was done so in the form:<br>
<br>
root@gfsi-rh-01:/mnt# gluster volume create gfs1 replica 2 transport tcp \<br>
gfsi-rh-01:/srv/hod/wiki \<br>
gfsi-isr-01:/srv/hod/wiki force<br>
<br>
Where the those paths to the bricks are not the raw paths but instead are paths to the mount points on the local server.<br>
<br>
Volume status returns:<br>
gluster volume status wiki<br>
Status of volume: wiki<br>
Gluster process Port Online Pid<br>
------------------------------------------------------------------------------<br>
Brick gfsi-rh-01.core.canterbury.ac.uk:/srv/hod/wiki 49157 Y 3077<br>
Brick gfsi-isr-01.core.canterbury.ac.uk:/srv/hod/wiki 49156 Y 3092<br>
Brick gfsi-cant-01.core.canterbury.ac.uk:/srv/hod/wiki 49152 Y 2908<br>
NFS Server on localhost 2049 Y 35065<br>
Self-heal Daemon on localhost N/A Y 35073<br>
NFS Server on <a href="http://gfsi-cant-01.core.canterbury.ac.uk" target="_blank">gfsi-cant-01.core.canterbury.ac.uk</a> 2049 Y 2920<br>
Self-heal Daemon on <a href="http://gfsi-cant-01.core.canterbury.ac.uk" target="_blank">gfsi-cant-01.core.canterbury.ac.uk</a> N/A Y 2927<br>
NFS Server on <a href="http://gfsi-isr-01.core.canterbury.ac.uk" target="_blank">gfsi-isr-01.core.canterbury.ac.uk</a> 2049 Y 32680<br>
Self-heal Daemon on <a href="http://gfsi-isr-01.core.canterbury.ac.uk" target="_blank">gfsi-isr-01.core.canterbury.ac.uk</a> N/A Y 32687<br>
Task Status of Volume wiki<br>
------------------------------------------------------------------------------<br>
There are no active volume tasks<br>
<br>
Which is what I would expect.<br>
<br>
Interestingly to check my thoughts:<br>
<br>
# tune2fs -l /srv/hod/wiki/<br>
tune2fs 1.42.5 (29-Jul-2012)<br>
tune2fs: Attempt to read block from filesystem resulted in short read while trying to open /srv/hod/wiki/<br>
Couldn't find valid filesystem superblock.<br>
<br>
Does what I expect as it is checking a mount point and is what it looks like gluster is trying to do.<br>
<br>
But:<br>
<br>
# tune2fs -l /dev/mapper/bricks-wiki<br>
tune2fs 1.42.5 (29-Jul-2012)<br>
Filesystem volume name: wiki<br>
Last mounted on: /srv/hod/wiki<br>
Filesystem UUID: a75306ac-31fa-447d-9da7-23ef66d9756b<br>
Filesystem magic number: 0xEF53<br>
Filesystem revision #: 1 (dynamic)<br>
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize<br>
Filesystem flags: signed_directory_hash<br>
Default mount options: user_xattr acl<br>
Filesystem state: clean<br>
<snipped><br>
<br>
This leaves me with a couple of questions:<br>
<br>
Is there any way that I can get this sorted in the gluster configuration so that it actually checks the raw volume rather than the local mount point for that volume?<br>
<br>
Should the volume have been created using the raw path /dev/mapper/..... rather than the mount point?<br>
<br>
Or should I have created the volume (as I *now* see in the RedHat Storage Admin Guide) - under a sub directrory directory below the mounted filestore (ie: /srv/hod/wiki/brick ?<br>
<br>
If I need to move the data and recreate the bricks it is not an issue for me as this is still proof of concept for what we are doing, what I need to know whether doing so will stop the continual log churn.<br>
<br>
Many thanks<br>
<br>
Paul<br>
<br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150316/a080617b/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150316/a080617b/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 8<br>
Date: Mon, 16 Mar 2015 17:30:04 +0100<br>
From: Pierre L?onard <<a href="mailto:pleonard@jouy.inra.fr">pleonard@jouy.inra.fr</a>><br>
To: <a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
Subject: Re: [Gluster-users] adding a node<br>
Message-ID: <<a href="mailto:5507050C.2060508@jouy.inra.fr">5507050C.2060508@jouy.inra.fr</a>><br>
Content-Type: text/plain; charset="utf-8"; Format="flowed"<br>
<br>
Hi,<br>
> Hi,<br>
><br>
> OK, I was confused because in your original email you mentioned "a<br>
> multiple of the striped so 4 other node".<br>
><br>
> So yes your stripe count is 2, so you have to add bricks 2 by 2, as the<br>
> error message says. If you didn't, then it would mean that you would<br>
> distribute your data over two 2-brick stripes, and one 1-brick single<br>
> volume, which I am reasonably sure wouldn't work.<br>
><br>
> If you want to be able to add bricks one by one, you would need a<br>
> regular distributed volume, without:<br>
> - striping (requires adding as many drives as stripe count)<br>
> - replication (requires adding as many drives as stripe replication count)<br>
> - dispersion (requires adding as many drives as whatever your dispersion<br>
> disk count is)<br>
><br>
> The only two things that you can extend is the number of brick groups in<br>
> a distributed volume, with a similar brick group, or the number of<br>
> replicas (but I'm sure that there are corner cases where you can't do<br>
> it, I haven't tried them yet).<br>
><br>
> Is that clearer now?<br>
Yes.<br>
So I have to find another node, or maybe is it possible to create a<br>
second brick on the new node ?<br>
<br>
Many Thank's.<br>
<br>
--<br>
Signature electronique<br>
INRA <<a href="http://www.inra.fr" target="_blank">http://www.inra.fr</a>><br>
*Pierre L?onard*<br>
*Senior IT Manager*<br>
*MetaGenoPolis*<br>
<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a> <mailto:<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a>><br>
T?l. : +33 (0)1 34 65 29 78<br>
<br>
Centre de recherche INRA<br>
Domaine de Vilvert ? B?t. 325 R+1<br>
78 352 Jouy-en-Josas CEDEX<br>
France<br>
<a href="http://www.mgps.eu" target="_blank">www.mgps.eu</a> <<a href="http://www.mgps.eu" target="_blank">http://www.mgps.eu</a>><br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150316/33b6a618/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150316/33b6a618/attachment-0001.html</a>><br>
-------------- next part --------------<br>
A non-text attachment was scrubbed...<br>
Name: logo-INRA2013.gif<br>
Type: image/gif<br>
Size: 2560 bytes<br>
Desc: <a href="http://www.inra.fr/" target="_blank">http://www.inra.fr/</a><br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150316/33b6a618/attachment-0001.gif" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150316/33b6a618/attachment-0001.gif</a>><br>
<br>
------------------------------<br>
<br>
Message: 9<br>
Date: Mon, 16 Mar 2015 17:39:47 +0100<br>
From: JF Le Fill?tre <<a href="mailto:jean-francois.lefillatre@uni.lu">jean-francois.lefillatre@uni.lu</a>><br>
To: <<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>><br>
Subject: Re: [Gluster-users] adding a node<br>
Message-ID: <<a href="mailto:55070753.20100@uni.lu">55070753.20100@uni.lu</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
<br>
Hello again,<br>
<br>
Yes both are possible. What matters in that case is that you have two<br>
bricks to keep the stripe count of two.<br>
<br>
Note that if you split your node into two bricks and stripe over them,<br>
you will share the bandwidth between those bricks, and they will likely<br>
be of different size than the other brick stripes. From time to time<br>
you'll probably have to rebalance.<br>
<br>
Thanks,<br>
JF<br>
<br>
<br>
<br>
On 16/03/15 17:30, Pierre L?onard wrote:<br>
> Hi,<br>
>> Hi,<br>
>><br>
>> OK, I was confused because in your original email you mentioned "a<br>
>> multiple of the striped so 4 other node".<br>
>><br>
>> So yes your stripe count is 2, so you have to add bricks 2 by 2, as the<br>
>> error message says. If you didn't, then it would mean that you would<br>
>> distribute your data over two 2-brick stripes, and one 1-brick single<br>
>> volume, which I am reasonably sure wouldn't work.<br>
>><br>
>> If you want to be able to add bricks one by one, you would need a<br>
>> regular distributed volume, without:<br>
>> - striping (requires adding as many drives as stripe count)<br>
>> - replication (requires adding as many drives as stripe replication count)<br>
>> - dispersion (requires adding as many drives as whatever your dispersion<br>
>> disk count is)<br>
>><br>
>> The only two things that you can extend is the number of brick groups in<br>
>> a distributed volume, with a similar brick group, or the number of<br>
>> replicas (but I'm sure that there are corner cases where you can't do<br>
>> it, I haven't tried them yet).<br>
>><br>
>> Is that clearer now?<br>
> Yes.<br>
> So I have to find another node, or maybe is it possible to create a<br>
> second brick on the new node ?<br>
><br>
> Many Thank's.<br>
><br>
> --<br>
> Signature electronique<br>
> INRA <<a href="http://www.inra.fr" target="_blank">http://www.inra.fr</a>><br>
><br>
> *Pierre L?onard*<br>
> *Senior IT Manager*<br>
> *MetaGenoPolis*<br>
> <a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a> <mailto:<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a>><br>
> T?l. : +33 (0)1 34 65 29 78<br>
><br>
> Centre de recherche INRA<br>
> Domaine de Vilvert ? B?t. 325 R+1<br>
> 78 352 Jouy-en-Josas CEDEX<br>
> France<br>
> <a href="http://www.mgps.eu" target="_blank">www.mgps.eu</a> <<a href="http://www.mgps.eu" target="_blank">http://www.mgps.eu</a>><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Gluster-users mailing list<br>
> <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> <a href="http://www.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
><br>
<br>
--<br>
<br>
Jean-Fran?ois Le Fill?tre<br>
-------------------------------<br>
HPC Systems Administrator<br>
LCSB - University of Luxembourg<br>
-------------------------------<br>
PGP KeyID 0x134657C6<br>
<br>
<br>
------------------------------<br>
<br>
Message: 10<br>
Date: Mon, 16 Mar 2015 17:46:11 +0100<br>
From: Pierre L?onard <<a href="mailto:pleonard@jouy.inra.fr">pleonard@jouy.inra.fr</a>><br>
To: <a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
Subject: Re: [Gluster-users] adding a node<br>
Message-ID: <<a href="mailto:550708D3.7050606@jouy.inra.fr">550708D3.7050606@jouy.inra.fr</a>><br>
Content-Type: text/plain; charset="utf-8"; Format="flowed"<br>
<br>
Hi,<br>
> Hello again,<br>
><br>
> Yes both are possible. What matters in that case is that you have two<br>
> bricks to keep the stripe count of two.<br>
><br>
> Note that if you split your node into two bricks and stripe over them,<br>
> you will share the bandwidth between those bricks, and they will likely<br>
> be of different size than the other brick stripes. From time to time<br>
> you'll probably have to rebalance.<br>
><br>
> Thanks,<br>
> JF<br>
I think, that is the solution for today.<br>
I have enough disc for creating another brick. and the network is 10Gbs.<br>
so I am not afraid by the throuput needed.<br>
<br>
Many thank'k to you for that exchange and ideas.<br>
<br>
sincerely.<br>
--<br>
Signature electronique<br>
INRA <<a href="http://www.inra.fr" target="_blank">http://www.inra.fr</a>><br>
*Pierre L?onard*<br>
*Senior IT Manager*<br>
*MetaGenoPolis*<br>
<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a> <mailto:<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a>><br>
T?l. : +33 (0)1 34 65 29 78<br>
<br>
Centre de recherche INRA<br>
Domaine de Vilvert ? B?t. 325 R+1<br>
78 352 Jouy-en-Josas CEDEX<br>
France<br>
<a href="http://www.mgps.eu" target="_blank">www.mgps.eu</a> <<a href="http://www.mgps.eu" target="_blank">http://www.mgps.eu</a>><br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150316/ebc9a3e0/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150316/ebc9a3e0/attachment-0001.html</a>><br>
-------------- next part --------------<br>
A non-text attachment was scrubbed...<br>
Name: logo-INRA2013.gif<br>
Type: image/gif<br>
Size: 2560 bytes<br>
Desc: <a href="http://www.inra.fr/" target="_blank">http://www.inra.fr/</a><br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150316/ebc9a3e0/attachment-0001.gif" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150316/ebc9a3e0/attachment-0001.gif</a>><br>
<br>
------------------------------<br>
<br>
Message: 11<br>
Date: Mon, 16 Mar 2015 18:30:21 +0100<br>
From: F?lix de Lelelis <<a href="mailto:felix.delelisdd@gmail.com">felix.delelisdd@gmail.com</a>><br>
To: <a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
Subject: [Gluster-users] Gluster 3.6 issue<br>
Message-ID:<br>
<CAL4JRLmUPpRa=<a href="mailto:CHCxZKPHmQTg41fYd6B91DWgDxSJPErEgHpUg@mail.gmail.com">CHCxZKPHmQTg41fYd6B91DWgDxSJPErEgHpUg@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Hi,<br>
<br>
I have a cluster with 2 nodes, and sometimes when I lunch gluster volume<br>
status, appear a error on the log:<br>
<br>
[2015-03-16 17:24:25.215352] E<br>
[glusterd-utils.c:7364:glusterd_add_inode_size_to_dict] 0-management:<br>
xfs_info exited with non-zero exit status<br>
[2015-03-16 17:24:25.215379] E<br>
[glusterd-utils.c:7390:glusterd_add_inode_size_to_dict] 0-management:<br>
failed to get inode size<br>
<br>
That you can be due?<br>
<br>
Thanks<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150316/a9874d12/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150316/a9874d12/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 12<br>
Date: Mon, 16 Mar 2015 20:44:29 +0000<br>
From: Jonathan Heese <<a href="mailto:jheese@inetu.net">jheese@inetu.net</a>><br>
To: "<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>" <<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>><br>
Subject: [Gluster-users] I/O error on replicated volume<br>
Message-ID: <<a href="mailto:79aaa9ff350749d6a313601caf3c2d6b@int-exch6.int.inetu.net">79aaa9ff350749d6a313601caf3c2d6b@int-exch6.int.inetu.net</a>><br>
Content-Type: text/plain; charset="us-ascii"<br>
<br>
Hello,<br>
<br>
So I resolved my previous issue with split-brains and the lack of self-healing by dropping my installed glusterfs* packages from 3.6.2 to 3.5.3, but now I've picked up a new issue, which actually makes normal use of the volume practically impossible.<br>
<br>
A little background for those not already paying close attention:<br>
I have a 2 node 2 brick replicating volume whose purpose in life is to hold iSCSI target files, primarily for use to provide datastores to a VMware ESXi cluster. The plan is to put a handful of image files on the Gluster volume, mount them locally on both Gluster nodes, and run tgtd on both, pointed to the image files on the mounted gluster volume. Then the ESXi boxes will use multipath (active/passive) iSCSI to connect to the nodes, with automatic failover in case of planned or unplanned downtime of the Gluster nodes.<br>
<br>
In my most recent round of testing with 3.5.3, I'm seeing a massive failure to write data to the volume after about 5-10 minutes, so I've simplified the scenario a bit (to minimize the variables) to: both Gluster nodes up, only one node (duke) mounted and running tgtd, and just regular (single path) iSCSI from a single ESXi server.<br>
<br>
About 5-10 minutes into migration a VM onto the test datastore, /var/log/messages on duke gets blasted with a ton of messages exactly like this:<br>
Mar 15 22:24:06 duke tgtd: bs_rdwr_request(180) io error 0x1781e00 2a -1 512 22971904, Input/output error<br>
<br>
And /var/log/glusterfs/mnt-gluster_disk.log gets blased with a ton of messages exactly like this:<br>
[2015-03-16 02:24:07.572279] W [fuse-bridge.c:2242:fuse_writev_cbk] 0-glusterfs-fuse: 635299: WRITE => -1 (Input/output error)<br>
<br>
And the write operation from VMware's side fails as soon as these messages start.<br>
<br>
I don't see any other errors (in the log files I know of) indicating the root cause of these i/o errors. I'm sure that this is not enough information to tell what's going on, but can anyone help me figure out what to look at next to figure this out?<br>
<br>
I've also considered using Dan Lambright's libgfapi gluster module for tgtd (or something similar) to avoid going through FUSE, but I'm not sure whether that would be irrelevant to this problem, since I'm not 100% sure if it lies in FUSE or elsewhere.<br>
<br>
Thanks!<br>
<br>
Jon Heese<br>
Systems Engineer<br>
INetU Managed Hosting<br>
P: 610.266.7441 x 261<br>
F: 610.266.7434<br>
<a href="http://www.inetu.net" target="_blank">www.inetu.net</a><<a href="https://www.inetu.net/" target="_blank">https://www.inetu.net/</a>><br>
** This message contains confidential information, which also may be privileged, and is intended only for the person(s) addressed above. Any unauthorized use, distribution, copying or disclosure of confidential and/or privileged information is strictly prohibited. If you have received this communication in error, please erase all copies of the message and its attachments and notify the sender immediately via reply e-mail. **<br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150316/68d109cd/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150316/68d109cd/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 13<br>
Date: Tue, 17 Mar 2015 09:07:12 +0800<br>
From: Punit Dambiwal <<a href="mailto:hypunit@gmail.com">hypunit@gmail.com</a>><br>
To: Vered Volansky <<a href="mailto:vered@redhat.com">vered@redhat.com</a>><br>
Cc: Martin Pavlik <<a href="mailto:mpavlik@redhat.com">mpavlik@redhat.com</a>>, "<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>"<br>
<<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>>, Martin Perina <<a href="mailto:mperina@redhat.com">mperina@redhat.com</a>>,<br>
"<a href="mailto:users@ovirt.org">users@ovirt.org</a>" <<a href="mailto:users@ovirt.org">users@ovirt.org</a>><br>
Subject: Re: [Gluster-users] [ovirt-users] Zombie disks failed to<br>
remove..<br>
Message-ID:<br>
<CAGZcrBnRMQ2eCEEhtwrwmLK30qeJd1h+TrrgaV=<a href="mailto:97CZ5nTrC4A@mail.gmail.com">97CZ5nTrC4A@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Hi,<br>
<br>
Can any body help on this....or it's bug in ovirt or gluster..??<br>
<br>
On Thu, Feb 26, 2015 at 12:07 PM, Punit Dambiwal <<a href="mailto:hypunit@gmail.com">hypunit@gmail.com</a>> wrote:<br>
<br>
> Hi Vered,<br>
><br>
> Please find the attached logs...<br>
><br>
> Thanks,<br>
> Punit<br>
><br>
> On Wed, Feb 25, 2015 at 2:16 PM, Vered Volansky <<a href="mailto:vered@redhat.com">vered@redhat.com</a>> wrote:<br>
><br>
>> Please send vdsm and engine logs so we can see why this happened.<br>
>><br>
>> Regards,<br>
>> Vered<br>
>><br>
>> ----- Original Message -----<br>
>> > From: "Punit Dambiwal" <<a href="mailto:hypunit@gmail.com">hypunit@gmail.com</a>><br>
>> > To: <a href="mailto:users@ovirt.org">users@ovirt.org</a>, <a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>, "Martin Pavlik" <<br>
>> <a href="mailto:mpavlik@redhat.com">mpavlik@redhat.com</a>>, "Martin Perina"<br>
>> > <<a href="mailto:mperina@redhat.com">mperina@redhat.com</a>><br>
>> > Sent: Tuesday, February 24, 2015 8:50:04 PM<br>
>> > Subject: [ovirt-users] Zombie disks failed to remove..<br>
>> ><br>
>> > Hi,<br>
>> ><br>
>> > In my ovirt infra...ovirt (3.5.1) with glusterfs (3.6.1)....now when i<br>
>> try to<br>
>> > remove the vm...the vm remove successfully but the vm disk (vdisk)<br>
>> remain<br>
>> > there.....if i try to remove this unattached disk..it's failed to<br>
>> remove....<br>
>> ><br>
>> ><br>
>> ><br>
>> > Thanks,<br>
>> > Punit<br>
>> ><br>
>> > _______________________________________________<br>
>> > Users mailing list<br>
>> > <a href="mailto:Users@ovirt.org">Users@ovirt.org</a><br>
>> > <a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
>> ><br>
>><br>
><br>
><br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150317/29d2cc85/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150317/29d2cc85/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 14<br>
Date: Mon, 16 Mar 2015 21:39:09 -0500<br>
From: Jay Strauss <<a href="mailto:me@heyjay.com">me@heyjay.com</a>><br>
To: <a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
Subject: [Gluster-users] Running a SQL database on Gluster on AWS<br>
Message-ID:<br>
<CANpdWGXynaqof+Y=<a href="mailto:g8BQaA-qS4k7Fm_V6%2BLmBpCf1tAZOqrNPA@mail.gmail.com">g8BQaA-qS4k7Fm_V6+LmBpCf1tAZOqrNPA@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Hi,<br>
<br>
Apologies, if this is worn earth, first time poster. I did the search the<br>
archives and didn't see any threads like mine.<br>
<br>
I want to setup postgresql upon gluster. I intend to build my cluster upon<br>
AWS. I'm running a variant of postgresql, which is a parallel version,<br>
that lets one start postgres on multiple nodes, each which may read the<br>
database files.<br>
<br>
Questions:<br>
<br>
1) I want to run upon Ubuntu v14.04 LTS, which packages would you<br>
recommend?<br>
/pub/gluster/glusterfs/LATEST/Debian<br>
Wheezy or Jessie or other??<br>
<br>
2) I am going to use AWS and EBS volumes. I watched a video by Louis<br>
Zuckerman, in which he indicated to use "many bricks per server".<br>
a) does that mean many EBS volumes per server? Or multiple bricks per EBS<br>
volume?<br>
b) How many is "many"?<br>
- My database will have 100s (maybe 1000s) of files, each will be between<br>
10-500MB.<br>
<br>
3) Can I use the EC2 units to do double duty, running the gluster process<br>
AND my postgresql processes (or is that a bad idea)?<br>
a) if I can do double duty, can my postgresql processes gain performance<br>
when they read local files? or do all process read via the network,<br>
regardless of whether the process wants files that may actually be local.<br>
<br>
Any other suggestions regarding running, setting up, on AWS?<br>
<br>
Thanks<br>
Jay<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150316/a5c748e3/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150316/a5c748e3/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 15<br>
Date: Tue, 17 Mar 2015 10:05:05 +0530<br>
From: Ravishankar N <<a href="mailto:ravishankar@redhat.com">ravishankar@redhat.com</a>><br>
To: Jonathan Heese <<a href="mailto:jheese@inetu.net">jheese@inetu.net</a>>, "<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>"<br>
<<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>><br>
Subject: Re: [Gluster-users] I/O error on replicated volume<br>
Message-ID: <<a href="mailto:5507AEF9.1090703@redhat.com">5507AEF9.1090703@redhat.com</a>><br>
Content-Type: text/plain; charset="windows-1252"; Format="flowed"<br>
<br>
<br>
On 03/17/2015 02:14 AM, Jonathan Heese wrote:<br>
> Hello,<br>
><br>
> So I resolved my previous issue with split-brains and the lack of<br>
> self-healing by dropping my installed glusterfs* packages from 3.6.2<br>
> to 3.5.3, but now I've picked up a new issue, which actually makes<br>
> normal use of the volume practically impossible.<br>
><br>
> A little background for those not already paying close attention:<br>
> I have a 2 node 2 brick replicating volume whose purpose in life is to<br>
> hold iSCSI target files, primarily for use to provide datastores to a<br>
> VMware ESXi cluster. The plan is to put a handful of image files on<br>
> the Gluster volume, mount them locally on both Gluster nodes, and run<br>
> tgtd on both, pointed to the image files on the mounted gluster<br>
> volume. Then the ESXi boxes will use multipath (active/passive) iSCSI<br>
> to connect to the nodes, with automatic failover in case of planned or<br>
> unplanned downtime of the Gluster nodes.<br>
><br>
> In my most recent round of testing with 3.5.3, I'm seeing a massive<br>
> failure to write data to the volume after about 5-10 minutes, so I've<br>
> simplified the scenario a bit (to minimize the variables) to: both<br>
> Gluster nodes up, only one node (duke) mounted and running tgtd, and<br>
> just regular (single path) iSCSI from a single ESXi server.<br>
><br>
> About 5-10 minutes into migration a VM onto the test datastore,<br>
> /var/log/messages on duke gets blasted with a ton of messages exactly<br>
> like this:<br>
><br>
> Mar 15 22:24:06 duke tgtd: bs_rdwr_request(180) io error 0x1781e00 2a<br>
> -1 512 22971904, Input/output error<br>
><br>
><br>
> And /var/log/glusterfs/mnt-gluster_disk.log gets blased with a ton of<br>
> messages exactly like this:<br>
><br>
> [2015-03-16 02:24:07.572279] W [fuse-bridge.c:2242:fuse_writev_cbk]<br>
> 0-glusterfs-fuse: 635299: WRITE => -1 (Input/output error)<br>
><br>
><br>
<br>
Are there any messages in the mount log from AFR about split-brain just<br>
before the above line appears?<br>
Does `gluster v heal <VOLNAME> info` show any files? Performing I/O on<br>
files that are in split-brain fail with EIO.<br>
<br>
-Ravi<br>
<br>
> And the write operation from VMware's side fails as soon as these<br>
> messages start.<br>
><br>
><br>
> I don't see any other errors (in the log files I know of) indicating<br>
> the root cause of these i/o errors. I'm sure that this is not enough<br>
> information to tell what's going on, but can anyone help me figure out<br>
> what to look at next to figure this out?<br>
><br>
><br>
> I've also considered using Dan Lambright's libgfapi gluster module for<br>
> tgtd (or something similar) to avoid going through FUSE, but I'm not<br>
> sure whether that would be irrelevant to this problem, since I'm not<br>
> 100% sure if it lies in FUSE or elsewhere.<br>
><br>
><br>
> Thanks!<br>
><br>
><br>
> /Jon Heese/<br>
> /Systems Engineer/<br>
> *INetU Managed Hosting*<br>
> P: 610.266.7441 x 261<br>
> F: 610.266.7434<br>
> <a href="http://www.inetu.net" target="_blank">www.inetu.net</a><<a href="https://www.inetu.net/" target="_blank">https://www.inetu.net/</a>><br>
><br>
> /** This message contains confidential information, which also may be<br>
> privileged, and is intended only for the person(s) addressed above.<br>
> Any unauthorized use, distribution, copying or disclosure of<br>
> confidential and/or privileged information is strictly prohibited. If<br>
> you have received this communication in error, please erase all copies<br>
> of the message and its attachments and notify the sender immediately<br>
> via reply e-mail. **/<br>
><br>
><br>
><br>
> _______________________________________________<br>
> Gluster-users mailing list<br>
> <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> <a href="http://www.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150317/d7c640a2/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150317/d7c640a2/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 16<br>
Date: Tue, 17 Mar 2015 12:52:31 +0800<br>
From: Punit Dambiwal <<a href="mailto:hypunit@gmail.com">hypunit@gmail.com</a>><br>
To: "<a href="mailto:users@ovirt.org">users@ovirt.org</a>" <<a href="mailto:users@ovirt.org">users@ovirt.org</a>>, "<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>"<br>
<<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>>, Vered Volansky <<a href="mailto:vered@redhat.com">vered@redhat.com</a>>,<br>
Michal Skrivanek <<a href="mailto:michal.skrivanek@redhat.com">michal.skrivanek@redhat.com</a>>, Humble Devassy<br>
Chirammal <<a href="mailto:humble.devassy@gmail.com">humble.devassy@gmail.com</a>>, Kanagaraj <<a href="mailto:kmayilsa@redhat.com">kmayilsa@redhat.com</a>>,<br>
Kaushal M <<a href="mailto:kshlmster@gmail.com">kshlmster@gmail.com</a>><br>
Subject: [Gluster-users] VM failed to start | Bad volume specification<br>
Message-ID:<br>
<<a href="mailto:CAGZcrBnbEGCSU4S92mKC0AGVK6B9aV0_SkCroHRL0dhVK_%2B3AA@mail.gmail.com">CAGZcrBnbEGCSU4S92mKC0AGVK6B9aV0_SkCroHRL0dhVK_+3AA@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Hi,<br>
<br>
I am facing one strange issue with ovirt/glusterfs....still didn't find<br>
this issue is related with glusterfs or Ovirt....<br>
<br>
Ovirt :- 3.5.1<br>
Glusterfs :- 3.6.1<br>
Host :- 4 Hosts (Compute+ Storage)...each server has 24 bricks<br>
Guest VM :- more then 100<br>
<br>
Issue :- When i deploy this cluster first time..it work well for me(all the<br>
guest VM created and running successfully)....but suddenly one day my one<br>
of the host node rebooted and none of the VM can boot up now...and failed<br>
with the following error "Bad Volume Specification"<br>
<br>
VMId :- d877313c18d9783ca09b62acf5588048<br>
<br>
VDSM Logs :- <a href="http://ur1.ca/jxabi" target="_blank">http://ur1.ca/jxabi</a><br>
Engine Logs :- <a href="http://ur1.ca/jxabv" target="_blank">http://ur1.ca/jxabv</a><br>
<br>
------------------------<br>
[root@cpu01 ~]# vdsClient -s 0 getVolumeInfo<br>
e732a82f-bae9-4368-8b98-dedc1c3814de 00000002-0002-0002-0002-000000000145<br>
6d123509-6867-45cf-83a2-6d679b77d3c5 9030bb43-6bc9-462f-a1b9-f6d5a02fb180<br>
status = OK<br>
domain = e732a82f-bae9-4368-8b98-dedc1c3814de<br>
capacity = 21474836480<br>
voltype = LEAF<br>
description =<br>
parent = 00000000-0000-0000-0000-000000000000<br>
format = RAW<br>
image = 6d123509-6867-45cf-83a2-6d679b77d3c5<br>
uuid = 9030bb43-6bc9-462f-a1b9-f6d5a02fb180<br>
disktype = 2<br>
legality = LEGAL<br>
mtime = 0<br>
apparentsize = 21474836480<br>
truesize = 4562972672<br>
type = SPARSE<br>
children = []<br>
pool =<br>
ctime = 1422676305<br>
---------------------<br>
<br>
I opened same thread earlier but didn't get any perfect answers to solve<br>
this issue..so i reopen it...<br>
<br>
<a href="https://www.mail-archive.com/users@ovirt.org/msg25011.html" target="_blank">https://www.mail-archive.com/users@ovirt.org/msg25011.html</a><br>
<br>
Thanks,<br>
Punit<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150317/2ceff835/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150317/2ceff835/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 17<br>
Date: Tue, 17 Mar 2015 06:24:15 +0100<br>
From: Melkor Lord <<a href="mailto:melkor.lord@gmail.com">melkor.lord@gmail.com</a>><br>
To: Joe Julian <<a href="mailto:joe@julianfamily.org">joe@julianfamily.org</a>><br>
Cc: <a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
Subject: Re: [Gluster-users] Working but some issues<br>
Message-ID:<br>
<CAOXaFyy==<a href="mailto:KXAPnLNzNf5yLoaJa4A9tsd9YZ1OZCgaEZGd0YSCA@mail.gmail.com">KXAPnLNzNf5yLoaJa4A9tsd9YZ1OZCgaEZGd0YSCA@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
On Mon, Mar 16, 2015 at 5:14 AM, Joe Julian <<a href="mailto:joe@julianfamily.org">joe@julianfamily.org</a>> wrote:<br>
<br>
><br>
> I'll just address this one point in this email because it's such an<br>
> important one. This is not just an open-source project because some<br>
> company's developing a product and lets you have it for free, this is an<br>
> open-source *project*. We, the members of the community, are as<br>
> responsible for that problem as the folks that know how to write in C;<br>
> perhaps even more so.<br>
><br>
> I implore you to add your skills to improving the documentation. You have<br>
> the ability to see the documentation from a completely different<br>
> perspective from the folks that wrote the code. They may not have<br>
> documented --remote-host because they perhaps added it for an internal<br>
> reason and didn't expect users to use it. By looking at it from a different<br>
> perspective, you may see a need for documentation and have the ability to<br>
> implement it.<br>
><br>
<br>
I globally agree with your statement but there's a catch here, more of a<br>
chicken-and-egg problem actually! In order to contribute to the<br>
documentation or help other users, I must first be able to understand the<br>
project myself! The mere documentation is missing at almost every stage in<br>
GlusterFS and this is problematic. If I'm not able to put GlusterFS at use<br>
understanding how it works and how it interacts between all of its<br>
components, how am I supposed to explain it to other people?<br>
<br>
I'm a sysadmin for over 2 decades and this is the first time I see such a<br>
major project (GlusterFS is not a small HelloWorld app, it's featureful,<br>
complex and envolves learning some concepts) with so little documentation.<br>
As I said before, I currently have *no* *clue* of all the options and<br>
directives I can use in the main configuration file<br>
/etc/glusterfs/glusterd.vol for example! There's only a "sample"<br>
configuration file with no further detail than the classic "paste this into<br>
the file". Well no thank you ;) I won't paste anything to any configuration<br>
file without understanding it first and have a complete set of directives I<br>
can use. I have the reponsability of having a running service and can't<br>
just "paste things" as told :-)<br>
<br>
The only way I got GlusterFS to work is by searching BLOG posts and this<br>
bad IMHO. The way I see how a project ecosystem should be managed is like<br>
this : The devs should not only code but provide the full information,<br>
documenting every single option and directive because no other than them<br>
know the project better than they do! After that, the ecosystem will grow<br>
by itself thanks to technical people that create blog posts and articles to<br>
explain various creative ways of using the software. The documentation from<br>
the devs does not have to be ultra exhaustive explaining all possible use<br>
cases of course but at least, document everything that needs to be<br>
documented to let other people understand what they are dealing with.<br>
<br>
Let me take 2 real world examples to get the general idea : Postfix and<br>
NGinX! They are flexible enough to provide a quite large set of use cases.<br>
Their documentation is impeccable from my point of view. They provide an<br>
exhaustive documentation of their inner options like this -<br>
<a href="http://www.postfix.org/postconf.5.html" target="_blank">http://www.postfix.org/postconf.5.html</a> - and this -<br>
<a href="http://nginx.org/en/docs/dirindex.html" target="_blank">http://nginx.org/en/docs/dirindex.html</a><br>
<br>
See, even if you forget all the HowTo's, articles and stuff, which are<br>
great additions and bonuses, you can manage to get out of the woods with<br>
these docs. That's exactly what I miss most in GlusterFS. There are here<br>
and there options explained but often with no context.<br>
<br>
To the very specific case of "--remote-host" option, there's a design<br>
problem in "gluster" command. Launching it without arguments and you get a<br>
prompt and the command completion helps a bit. Now, try "gluster -h" (or<br>
--help or -? or whatever) and you end up with "unrecognized option --XXX".<br>
This is counter intuitive again. You can't experiment by trial and error to<br>
figure out things when you're in the dark, that's why I had to take a peek<br>
to the source code and find out the existence of other options.<br>
<br>
If you spend so much time trying to find information instead of<br>
experimenting with a project, you may grow bored and leave. This would be<br>
bad because the lack of documentation may lead people to avoid a project<br>
which could be really useful and good! GlusterFS features exactly what I<br>
want for my usage, that's why I picked it up but I didn't think it would be<br>
so hard to get proper documentation.<br>
<br>
For example, I can't get SSL to work with my 3.6.2 setup and there's not a<br>
single bit of doc about it. There's only<br>
<a href="http://blog.gluster.org/author/zbyszek/" target="_blank">http://blog.gluster.org/author/zbyszek/</a> but even after following the<br>
necessary steps, I end up with a cryptic log entry "Cannot authenticate<br>
client from fs-00-22666-2015/03/16-11:42:54:167984-Data-client-0-0-0 3.6.2"<br>
and repeated for all the replicas :-( I don't know what GlusterFS expects<br>
in this case so I can't solve the problem for now.<br>
<br>
I'll stop boring you now, you get the point ;) You can only explain what<br>
you clearly understand and for now, this is still way too foogy for me :)<br>
<br>
--<br>
Unix _IS_ user friendly, it's just selective about who its friends are.<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150317/b357a41e/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150317/b357a41e/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 18<br>
Date: Mon, 16 Mar 2015 23:43:12 -0700<br>
From: Joe Julian <<a href="mailto:joe@julianfamily.org">joe@julianfamily.org</a>><br>
To: Melkor Lord <<a href="mailto:melkor.lord@gmail.com">melkor.lord@gmail.com</a>><br>
Cc: <a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
Subject: Re: [Gluster-users] Working but some issues<br>
Message-ID: <<a href="mailto:5507CD00.1000503@julianfamily.org">5507CD00.1000503@julianfamily.org</a>><br>
Content-Type: text/plain; charset="utf-8"; Format="flowed"<br>
<br>
On 03/16/2015 10:24 PM, Melkor Lord wrote:<br>
><br>
> On Mon, Mar 16, 2015 at 5:14 AM, Joe Julian <<a href="mailto:joe@julianfamily.org">joe@julianfamily.org</a><br>
> <mailto:<a href="mailto:joe@julianfamily.org">joe@julianfamily.org</a>>> wrote:<br>
><br>
><br>
> I'll just address this one point in this email because it's such<br>
> an important one. This is not just an open-source project because<br>
> some company's developing a product and lets you have it for free,<br>
> this is an open-source *project*. We, the members of the<br>
> community, are as responsible for that problem as the folks that<br>
> know how to write in C; perhaps even more so.<br>
><br>
> I implore you to add your skills to improving the documentation.<br>
> You have the ability to see the documentation from a completely<br>
> different perspective from the folks that wrote the code. They may<br>
> not have documented --remote-host because they perhaps added it<br>
> for an internal reason and didn't expect users to use it. By<br>
> looking at it from a different perspective, you may see a need for<br>
> documentation and have the ability to implement it.<br>
><br>
><br>
> I globally agree with your statement but there's a catch here, more of<br>
> a chicken-and-egg problem actually! In order to contribute to the<br>
> documentation or help other users, I must first be able to understand<br>
> the project myself! The mere documentation is missing at almost every<br>
> stage in GlusterFS and this is problematic. If I'm not able to put<br>
> GlusterFS at use understanding how it works and how it interacts<br>
> between all of its components, how am I supposed to explain it to<br>
> other people?<br>
<br>
Good question. It took me months to figure all that out (with far less<br>
documentation than there is now) and even with pictures and arrows and<br>
8x10 glossies with a paragraph on the back of each one, people still<br>
have a hard time getting it. That's why so much effort has gone in to<br>
making a cli that does it all for you and doesn't require you to be a<br>
storage engineer to use it.<br>
<br>
><br>
> I'm a sysadmin for over 2 decades and this is the first time I see<br>
> such a major project (GlusterFS is not a small HelloWorld app, it's<br>
> featureful, complex and envolves learning some concepts) with so<br>
> little documentation.<br>
<br>
I'll split this out as I think you're unaware of the admin guide that's<br>
pretty detailed and is, at least, published with the source code (it may<br>
be on the <a href="http://gluster.org" target="_blank">gluster.org</a> site somewhere, but I'm too tired right now to<br>
look). The source can readily be found on github<br>
<<a href="https://github.com/GlusterFS/glusterfs/tree/master/doc/admin-guide/en-US/markdown" target="_blank">https://github.com/GlusterFS/glusterfs/tree/master/doc/admin-guide/en-US/markdown</a>>.<br>
<br>
> As I said before, I currently have *no* *clue* of all the options and<br>
> directives I can use in the main configuration file<br>
> /etc/glusterfs/glusterd.vol for example!<br>
<br>
Right. There's nearly no need to mess with that except under the lesser<br>
circumstance that an unprivileged user needs access to the management port.<br>
<br>
> There's only a "sample" configuration file with no further detail than<br>
> the classic "paste this into the file". Well no thank you ;) I won't<br>
> paste anything to any configuration file without understanding it<br>
> first and have a complete set of directives I can use. I have the<br>
> reponsability of having a running service and can't just "paste<br>
> things" as told :-)<br>
><br>
> The only way I got GlusterFS to work is by searching BLOG posts and<br>
> this bad IMHO. The way I see how a project ecosystem should be managed<br>
> is like this : The devs should not only code but provide the full<br>
> information, documenting every single option and directive because no<br>
> other than them know the project better than they do! After that, the<br>
> ecosystem will grow by itself thanks to technical people that create<br>
> blog posts and articles to explain various creative ways of using the<br>
> software. The documentation from the devs does not have to be ultra<br>
> exhaustive explaining all possible use cases of course but at least,<br>
> document everything that needs to be documented to let other people<br>
> understand what they are dealing with.<br>
<br>
That is the way it's done, btw. The developers are required to document<br>
their features before a release.<br>
<br>
> Let me take 2 real world examples to get the general idea : Postfix<br>
> and NGinX! They are flexible enough to provide a quite large set of<br>
> use cases. Their documentation is impeccable from my point of view.<br>
> They provide an exhaustive documentation of their inner options like<br>
> this - <a href="http://www.postfix.org/postconf.5.html" target="_blank">http://www.postfix.org/postconf.5.html</a> - and this -<br>
> <a href="http://nginx.org/en/docs/dirindex.html" target="_blank">http://nginx.org/en/docs/dirindex.html</a><br>
<br>
Postfix, 16 years old, and hasn't always had very detailed<br>
documentation. Do you also remember when it was worse than sendmail's?<br>
<br>
I could counter with any of the myriad of horrible documentation for<br>
some of the most popular software systems out there only to point out<br>
that Gluster's isn't all that bad by comparison to a great many of its<br>
peers.<br>
<br>
><br>
> See, even if you forget all the HowTo's, articles and stuff, which are<br>
> great additions and bonuses, you can manage to get out of the woods<br>
> with these docs. That's exactly what I miss most in GlusterFS. There<br>
> are here and there options explained but often with no context.<br>
><br>
> To the very specific case of "--remote-host" option, there's a design<br>
> problem in "gluster" command. Launching it without arguments and you<br>
> get a prompt and the command completion helps a bit. Now, try "gluster<br>
> -h" (or --help or -? or whatever) and you end up with "unrecognized<br>
> option --XXX". This is counter intuitive again. You can't experiment<br>
> by trial and error to figure out things when you're in the dark,<br>
> that's why I had to take a peek to the source code and find out the<br>
> existence of other options.<br>
<br>
"gluster help" is pretty intuitive, imho, as is<br>
<br>
# gluster<br>
gluster> help<br>
<br>
and the more detailed than any other software I can think of, "gluster<br>
volume set help" which has all the settings you can tweak in your volume<br>
along with their description equivalent to that postfix document.<br>
<br>
> If you spend so much time trying to find information instead of<br>
> experimenting with a project, you may grow bored and leave.<br>
<br>
Agreed, and that's something that the <a href="http://gluster.org" target="_blank">gluster.org</a> web site's been<br>
failing at since the last 1 or 2 web site revamps.<br>
<br>
> This would be bad because the lack of documentation may lead people to<br>
> avoid a project which could be really useful and good! GlusterFS<br>
> features exactly what I want for my usage, that's why I picked it up<br>
> but I didn't think it would be so hard to get proper documentation.<br>
><br>
> For example, I can't get SSL to work with my 3.6.2 setup and there's<br>
> not a single bit of doc about it. There's only<br>
> <a href="http://blog.gluster.org/author/zbyszek/" target="_blank">http://blog.gluster.org/author/zbyszek/</a> but even after following the<br>
> necessary steps, I end up with a cryptic log entry "Cannot<br>
> authenticate client from<br>
> fs-00-22666-2015/03/16-11:42:54:167984-Data-client-0-0-0 3.6.2" and<br>
> repeated for all the replicas :-( I don't know what GlusterFS expects<br>
> in this case so I can't solve the problem for now.<br>
<br>
<a href="https://github.com/GlusterFS/glusterfs/blob/master/doc/admin-guide/en-US/markdown/admin_ssl.md" target="_blank">https://github.com/GlusterFS/glusterfs/blob/master/doc/admin-guide/en-US/markdown/admin_ssl.md</a><br>
<-- not a blog<br>
<br>
><br>
> I'll stop boring you now, you get the point ;) You can only explain<br>
> what you clearly understand and for now, this is still way too foogy<br>
> for me :)<br>
><br>
<br>
Useful and eloquent perspectives and bits that infra is looking at<br>
rectifying. The web site is covered in too many words. The "Getting<br>
Started" entry has 13 sub-entries. That's not "getting started" that's<br>
"tl;dr". A new vision is being put together that will try to not just<br>
build a fancy web thingy, but will define goals such as usability,<br>
engagement, community interfacing, that kind of stuff - and measure the<br>
effectiveness of the changes that are made. It'll be change for the sake<br>
of improvement rather than just change for the sake of change.<br>
<br>
> --<br>
> Unix _IS_ user friendly, it's just selective about who its friends are.<br>
<br>
But I still say you should still document things you find in the source<br>
if they aren't documented - since you're in there anyway. :-D<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150316/eb2aa98f/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150316/eb2aa98f/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 19<br>
Date: Tue, 17 Mar 2015 16:09:47 +0800<br>
From: Punit Dambiwal <<a href="mailto:hypunit@gmail.com">hypunit@gmail.com</a>><br>
To: "<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>" <<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>>, Kaushal M<br>
<<a href="mailto:kshlmster@gmail.com">kshlmster@gmail.com</a>>, Kanagaraj <<a href="mailto:kmayilsa@redhat.com">kmayilsa@redhat.com</a>>, Vijay Bellur<br>
<<a href="mailto:vbellur@redhat.com">vbellur@redhat.com</a>><br>
Subject: [Gluster-users] Bad volume specification | Connection timeout<br>
Message-ID:<br>
<CAGZcrB=n=D0VUuPXLUONOKbJptMjv7=M8QuOJQ=<a href="mailto:QNVSTuG248g@mail.gmail.com">QNVSTuG248g@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Hi,<br>
<br>
I am facing one strange issue with ovirt/glusterfs....still didn't find<br>
this issue is related with glusterfs or Ovirt....<br>
<br>
Ovirt :- 3.5.1<br>
Glusterfs :- 3.6.1<br>
Host :- 4 Hosts (Compute+ Storage)...each server has 24 bricks<br>
Guest VM :- more then 100<br>
<br>
Issue :- When i deploy this cluster first time..it work well for me(all the<br>
guest VM created and running successfully)....but suddenly one day my one<br>
of the host node rebooted and none of the VM can boot up now...and failed<br>
with the following error "Bad Volume Specification"<br>
<br>
VMId :- d877313c18d9783ca09b62acf5588048<br>
<br>
VDSM Logs :- <a href="http://ur1.ca/jxabi" target="_blank">http://ur1.ca/jxabi</a><br>
Engine Logs :- <a href="http://ur1.ca/jxabv" target="_blank">http://ur1.ca/jxabv</a><br>
<br>
------------------------<br>
[root@cpu01 ~]# vdsClient -s 0 getVolumeInfo<br>
e732a82f-bae9-4368-8b98-dedc1c3814de 00000002-0002-0002-0002-000000000145<br>
6d123509-6867-45cf-83a2-6d679b77d3c5 9030bb43-6bc9-462f-a1b9-f6d5a02fb180<br>
status = OK<br>
domain = e732a82f-bae9-4368-8b98-dedc1c3814de<br>
capacity = 21474836480<br>
voltype = LEAF<br>
description =<br>
parent = 00000000-0000-0000-0000-000000000000<br>
format = RAW<br>
image = 6d123509-6867-45cf-83a2-6d679b77d3c5<br>
uuid = 9030bb43-6bc9-462f-a1b9-f6d5a02fb180<br>
disktype = 2<br>
legality = LEGAL<br>
mtime = 0<br>
apparentsize = 21474836480<br>
truesize = 4562972672<br>
type = SPARSE<br>
children = []<br>
pool =<br>
ctime = 1422676305<br>
---------------------<br>
<br>
I opened same thread earlier but didn't get any perfect answers to solve<br>
this issue..so i reopen it...<br>
<br>
<a href="https://www.mail-archive.com/users@ovirt.org/msg25011.html" target="_blank">https://www.mail-archive.com/users@ovirt.org/msg25011.html</a><br>
<br>
Thanks,<br>
Punit<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150317/eb027eaf/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150317/eb027eaf/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 20<br>
Date: Tue, 17 Mar 2015 10:20:12 +0100<br>
From: Pierre L?onard <<a href="mailto:pleonard@jouy.inra.fr">pleonard@jouy.inra.fr</a>><br>
To: aytac zeren <<a href="mailto:aytaczeren@gmail.com">aytaczeren@gmail.com</a>>, "<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>"<br>
<<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>><br>
Subject: Re: [Gluster-users] adding a node<br>
Message-ID: <<a href="mailto:5507F1CC.5080209@jouy.inra.fr">5507F1CC.5080209@jouy.inra.fr</a>><br>
Content-Type: text/plain; charset="utf-8"; Format="flowed"<br>
<br>
Hi aytac zeren,<br>
> In order to have a healthy volume, you need to have bricks as factors<br>
> of 4. Which means if you have 4 bricks and want to extend your setup<br>
> with additional brick, then you will need 4 other bricks in your<br>
> cluster, for your setup.<br>
><br>
> I don't recommend to host more than one brick on a host as it would<br>
> cause data loss on failure of the node, if your master and redundant<br>
> copy is stored on the same host.<br>
> Regards<br>
> Aytac<br>
Yes I understand. But You can understand and I hope that the gluster<br>
team can understand, that if I want to expand my computational power and<br>
storage I can't buy four node In one time, it's to heavy.<br>
Another example, I have a big cluster with 14 node stripe 7. I can't by<br>
14 other nodes to expand it in one time. It's a big limitation in the<br>
flexible usage of glusterfs.<br>
<br>
Does that mean that glusterfs is only dedicated to small cluster ?<br>
<br>
Sincerely.<br>
--<br>
Signature electronique<br>
INRA <<a href="http://www.inra.fr" target="_blank">http://www.inra.fr</a>><br>
*Pierre L?onard*<br>
*Senior IT Manager*<br>
*MetaGenoPolis*<br>
<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a> <mailto:<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a>><br>
T?l. : +33 (0)1 34 65 29 78<br>
<br>
Centre de recherche INRA<br>
Domaine de Vilvert ? B?t. 325 R+1<br>
78 352 Jouy-en-Josas CEDEX<br>
France<br>
<a href="http://www.mgps.eu" target="_blank">www.mgps.eu</a> <<a href="http://www.mgps.eu" target="_blank">http://www.mgps.eu</a>><br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150317/9bb6d373/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150317/9bb6d373/attachment-0001.html</a>><br>
-------------- next part --------------<br>
A non-text attachment was scrubbed...<br>
Name: logo-INRA2013.gif<br>
Type: image/gif<br>
Size: 2560 bytes<br>
Desc: <a href="http://www.inra.fr/" target="_blank">http://www.inra.fr/</a><br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150317/9bb6d373/attachment-0001.gif" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150317/9bb6d373/attachment-0001.gif</a>><br>
<br>
------------------------------<br>
<br>
Message: 21<br>
Date: Tue, 17 Mar 2015 10:38:04 +0100<br>
From: Niels de Vos <<a href="mailto:ndevos@redhat.com">ndevos@redhat.com</a>><br>
To: <a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>, <a href="mailto:gluster-devel@gluster.org">gluster-devel@gluster.org</a><br>
Subject: [Gluster-users] REMINDER: Gluster Community Bug Triage<br>
meeting today at 12:00 UTC<br>
Message-ID: <<a href="mailto:20150317093804.GJ29220@ndevos-x240.usersys.redhat.com">20150317093804.GJ29220@ndevos-x240.usersys.redhat.com</a>><br>
Content-Type: text/plain; charset="us-ascii"<br>
<br>
Hi all,<br>
<br>
This meeting is scheduled for anyone that is interested in learning more<br>
about, or assisting with the Bug Triage.<br>
<br>
Meeting details:<br>
- location: #gluster-meeting on Freenode IRC<br>
( <a href="https://webchat.freenode.net/?channels=gluster-meeting" target="_blank">https://webchat.freenode.net/?channels=gluster-meeting</a> )<br>
- date: every Tuesday<br>
- time: 12:00 UTC, 13:00 CET<br>
(in your terminal, run: date -d "12:00 UTC")<br>
- agenda: <a href="https://public.pad.fsfe.org/p/gluster-bug-triage" target="_blank">https://public.pad.fsfe.org/p/gluster-bug-triage</a><br>
<br>
Currently the following items are listed:<br>
* Roll Call<br>
* Status of last weeks action items<br>
* Group Triage<br>
* Open Floor<br>
<br>
The last two topics have space for additions. If you have a suitable bug<br>
or topic to discuss, please add it to the agenda.<br>
<br>
Thanks,<br>
Niels<br>
-------------- next part --------------<br>
A non-text attachment was scrubbed...<br>
Name: not available<br>
Type: application/pgp-signature<br>
Size: 181 bytes<br>
Desc: not available<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150317/df268b1d/attachment-0001.sig" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150317/df268b1d/attachment-0001.sig</a>><br>
<br>
------------------------------<br>
<br>
Message: 22<br>
Date: Tue, 17 Mar 2015 11:13:12 +0100<br>
From: JF Le Fill?tre <<a href="mailto:jean-francois.lefillatre@uni.lu">jean-francois.lefillatre@uni.lu</a>><br>
To: <<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>><br>
Subject: Re: [Gluster-users] adding a node<br>
Message-ID: <<a href="mailto:5507FE38.30507@uni.lu">5507FE38.30507@uni.lu</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
<br>
Hello Pierre,<br>
<br>
I see your point and I understand your arguments. But as it happens, the<br>
way you create your volumes has an impact on what you can do afterward,<br>
and how you can expand them. *It has nothing to do with GlusterFS<br>
itself, nor the developers or the community, it's all in your<br>
architectural choices in the beginning.*<br>
<br>
It would be the same problem with many other distributed FS, or even<br>
some RAID setups. That is the kind of thing that you have to think about<br>
in the very beginning, and that will have consequences later.<br>
<br>
Aytac's argument doesn't apply to you as you don't have a replicated<br>
volume, but a distributed+striped one. So if you lose one of your<br>
original 4 nodes, you lose half of one of your stripes, and that will<br>
make the files on that stripe inaccessible anyway. If you lose a node<br>
with both bricks in the stripe, well the result will be the same. So I<br>
don't think that the colocation of striped bricks on the same host<br>
matters much in your case, outside of performance concerns.<br>
<br>
As for your 14-node cluster, if it's again a distributed+striped volume,<br>
then it's a 2 ? 7 volume (distributed over 2 brick groups, each being a<br>
7-disk stripe). To extend it, you will have to add 7 more bricks to<br>
create a new striped brick group, and your volume will be transformed<br>
into a 3 ? 7 one.<br>
<br>
I would advise you to read carefully the master documentation pertaining<br>
to volume creation and architecture. Maybe it will help you understand<br>
better the way things work and the impacts of your choices:<br>
<br>
<a href="https://github.com/gluster/glusterfs/blob/master/doc/admin-guide/en-US/markdown/admin_setting_volumes.md" target="_blank">https://github.com/gluster/glusterfs/blob/master/doc/admin-guide/en-US/markdown/admin_setting_volumes.md</a><br>
<br>
Thanks,<br>
JF<br>
<br>
<br>
On 17/03/15 10:20, Pierre L?onard wrote:<br>
> Hi aytac zeren,<br>
>> In order to have a healthy volume, you need to have bricks as factors<br>
>> of 4. Which means if you have 4 bricks and want to extend your setup<br>
>> with additional brick, then you will need 4 other bricks in your<br>
>> cluster, for your setup.<br>
>><br>
>> I don't recommend to host more than one brick on a host as it would<br>
>> cause data loss on failure of the node, if your master and redundant<br>
>> copy is stored on the same host.<br>
>> Regards<br>
>> Aytac<br>
> Yes I understand. But You can understand and I hope that the gluster<br>
> team can understand, that if I want to expand my computational power and<br>
> storage I can't buy four node In one time, it's to heavy.<br>
> Another example, I have a big cluster with 14 node stripe 7. I can't by<br>
> 14 other nodes to expand it in one time. It's a big limitation in the<br>
> flexible usage of glusterfs.<br>
><br>
> Does that mean that glusterfs is only dedicated to small cluster ?<br>
><br>
> Sincerely.<br>
> --<br>
> Signature electronique<br>
> INRA <<a href="http://www.inra.fr" target="_blank">http://www.inra.fr</a>><br>
><br>
> *Pierre L?onard*<br>
> *Senior IT Manager*<br>
> *MetaGenoPolis*<br>
> <a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a> <mailto:<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a>><br>
> T?l. : +33 (0)1 34 65 29 78<br>
><br>
> Centre de recherche INRA<br>
> Domaine de Vilvert ? B?t. 325 R+1<br>
> 78 352 Jouy-en-Josas CEDEX<br>
> France<br>
> <a href="http://www.mgps.eu" target="_blank">www.mgps.eu</a> <<a href="http://www.mgps.eu" target="_blank">http://www.mgps.eu</a>><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Gluster-users mailing list<br>
> <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> <a href="http://www.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
><br>
<br>
--<br>
<br>
Jean-Fran?ois Le Fill?tre<br>
-------------------------------<br>
HPC Systems Administrator<br>
LCSB - University of Luxembourg<br>
-------------------------------<br>
PGP KeyID 0x134657C6<br>
<br>
<br>
------------------------------<br>
<br>
Message: 23<br>
Date: Tue, 17 Mar 2015 11:48:35 +0100<br>
From: Pierre L?onard <<a href="mailto:pleonard@jouy.inra.fr">pleonard@jouy.inra.fr</a>><br>
To: <a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
Subject: Re: [Gluster-users] adding a node<br>
Message-ID: <<a href="mailto:55080683.50300@jouy.inra.fr">55080683.50300@jouy.inra.fr</a>><br>
Content-Type: text/plain; charset="utf-8"; Format="flowed"<br>
<br>
Hi JF,<br>
> Hello Pierre,<br>
><br>
> I see your point and I understand your arguments. But as it happens, the<br>
> way you create your volumes has an impact on what you can do afterward,<br>
> and how you can expand them. *It has nothing to do with GlusterFS<br>
> itself, nor the developers or the community, it's all in your<br>
> architectural choices in the beginning.*<br>
Yes Theoretical you are right. Here at INRA, we are in a specific unit<br>
with a constant growing. But the budget is dedicated annually.<br>
So I have build three clusters dedicated to different type of<br>
computations and when the new plateform will need storage I have to<br>
either build a new cluster or by a complete set of Four node or seven<br>
node depending of my budget.<br>
Our unit is not in a data-center profile for the computation, the<br>
evolution is a continue process.<br>
<br>
<br>
> It would be the same problem with many other distributed FS, or even<br>
> some RAID setups. That is the kind of thing that you have to think about<br>
> in the very beginning, and that will have consequences later.<br>
Yes, that's the reason we have a scratch zone dedicated for the<br>
computation and intermediate files, that is not secured all the user<br>
know that it's like your DD in your PC. And other volume secured with<br>
raid 5 and replica.<br>
I think that the choice ar good for three years, the time for us to get<br>
the first DD problems.<br>
<br>
> As for your 14-node cluster, if it's again a distributed+striped volume,<br>
> then it's a 2 ? 7 volume (distributed over 2 brick groups, each being a<br>
> 7-disk stripe). To extend it, you will have to add 7 more bricks to<br>
> create a new striped brick group, and your volume will be transformed<br>
> into a 3 ? 7 one.<br>
OK that's better.<br>
> I would advise you to read carefully the master documentation pertaining<br>
> to volume creation and architecture. Maybe it will help you understand<br>
> better the way things work and the impacts of your choices:<br>
><br>
> <a href="https://github.com/gluster/glusterfs/blob/master/doc/admin-guide/en-US/markdown/admin_setting_volumes.md" target="_blank">https://github.com/gluster/glusterfs/blob/master/doc/admin-guide/en-US/markdown/admin_setting_volumes.md</a><br>
I will.<br>
<br>
Many thanks for you and All.<br>
<br>
Sincerely.<br>
<br>
--<br>
Signature electronique<br>
INRA <<a href="http://www.inra.fr" target="_blank">http://www.inra.fr</a>><br>
*Pierre L?onard*<br>
*Senior IT Manager*<br>
*MetaGenoPolis*<br>
<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a> <mailto:<a href="mailto:Pierre.Leonard@jouy.inra.fr">Pierre.Leonard@jouy.inra.fr</a>><br>
T?l. : +33 (0)1 34 65 29 78<br>
<br>
Centre de recherche INRA<br>
Domaine de Vilvert ? B?t. 325 R+1<br>
78 352 Jouy-en-Josas CEDEX<br>
France<br>
<a href="http://www.mgps.eu" target="_blank">www.mgps.eu</a> <<a href="http://www.mgps.eu" target="_blank">http://www.mgps.eu</a>><br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150317/c29f4366/attachment-0001.html" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150317/c29f4366/attachment-0001.html</a>><br>
-------------- next part --------------<br>
A non-text attachment was scrubbed...<br>
Name: logo-INRA2013.gif<br>
Type: image/gif<br>
Size: 2560 bytes<br>
Desc: <a href="http://www.inra.fr/" target="_blank">http://www.inra.fr/</a><br>
URL: <<a href="http://www.gluster.org/pipermail/gluster-users/attachments/20150317/c29f4366/attachment-0001.gif" target="_blank">http://www.gluster.org/pipermail/gluster-users/attachments/20150317/c29f4366/attachment-0001.gif</a>><br>
<br>
------------------------------<br>
<br>
Message: 24<br>
Date: Tue, 17 Mar 2015 11:03:59 +0100<br>
From: Nico Schottelius <<a href="mailto:nico-gluster-users@schottelius.org">nico-gluster-users@schottelius.org</a>><br>
To: Joe Julian <<a href="mailto:joe@julianfamily.org">joe@julianfamily.org</a>><br>
Cc: <a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
Subject: Re: [Gluster-users] Working but some issues<br>
Message-ID: <<a href="mailto:20150317100359.GA1028@schottelius.org">20150317100359.GA1028@schottelius.org</a>><br>
Content-Type: text/plain; charset=us-ascii<br>
<br>
Joe Julian [Mon, Mar 16, 2015 at 11:43:12PM -0700]:<br>
> Good question. It took me months to figure all that out (with far less<br>
> documentation than there is now) [...]<br>
<br>
... just wondering: Why don't we run a kickstarter/indiegogo campaign to<br>
finance people to write (from scratch?) documentation?<br>
<br>
There are various good examples of GREAT documentation for various<br>
projects and given that glusterfs is an evolving software that becomes<br>
more popular, it could be a way to improve documentation and thus<br>
experience with glusterfs.<br>
<br>
--<br>
New PGP key: 659B 0D91 E86E 7E24 FD15 69D0 C729 21A1 293F 2D24<br>
<br>
<br>
------------------------------<br>
<br>
Message: 25<br>
Date: Tue, 17 Mar 2015 07:38:44 -0400<br>
From: Kaleb KEITHLEY <<a href="mailto:kkeithle@redhat.com">kkeithle@redhat.com</a>><br>
To: "<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>" <<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>>, Gluster<br>
Devel <<a href="mailto:gluster-devel@gluster.org">gluster-devel@gluster.org</a>><br>
Subject: [Gluster-users] GlusterFS 3.4.7beta2 is now available for<br>
testing<br>
Message-ID: <<a href="mailto:55081244.4080709@redhat.com">55081244.4080709@redhat.com</a>><br>
Content-Type: text/plain; charset=windows-1252; format=flowed<br>
<br>
<br>
Many thanks to all our users that have reported bugs against the 3.4<br>
version of GlusterFS! glusterfs-3.4.7beta2 has been made available for<br>
testing.<br>
<br>
N.B. glusterfs-3.4.7beta1 was released but did not build with gcc-5.<br>
<br>
If you filed a bug against 3.4.x and it is listed as fixed in the<br>
Release Notes, please test it to confirm that it is fixed. Please update<br>
the bug report as soon as possible if you find that it has not been<br>
fixed. If any assistance is needed, do not hesitate to send a request to<br>
the Gluster Users mailinglist (<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>) or start a<br>
discussion in the #gluster channel on Freenode IRC.<br>
<br>
The release notes can be found at<br>
<a href="http://blog.gluster.org/2015/03/glusterfs-3-4-7beta2-is-now-available-for-testing/" target="_blank">http://blog.gluster.org/2015/03/glusterfs-3-4-7beta2-is-now-available-for-testing/</a><br>
<br>
Packages for selected distributions can be found on the main download<br>
server at<br>
<a href="http://download.gluster.org/pub/gluster/glusterfs/qa-releases/3.4.7beta2/" target="_blank">http://download.gluster.org/pub/gluster/glusterfs/qa-releases/3.4.7beta2/</a><br>
<br>
Thank you in advance for testing,<br>
<br>
--<br>
<br>
your friendly GlusterFS-3.4 release wrangler<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
<br>
End of Gluster-users Digest, Vol 83, Issue 18<br>
*********************************************<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div>ramos ye </div><div><a href="mailto:yt.tienon@gmail.com" target="_blank">yt.tienon@gmail.com</a></div></div></div>
</div>