<HTML><HEAD>
<STYLE id=eMClientCss>blockquote.cite { margin-left: 5px; margin-right: 0px; padding-left: 10px; padding-right:0px; border-left: 1px solid #cccccc }
blockquote.cite2 {margin-left: 5px; margin-right: 0px; padding-left: 10px; padding-right:0px; border-left: 1px solid #cccccc; margin-top: 3px; padding-top: 0px; }
.plain pre, .plain tt { font-family: monospace; font-size: 100%; font-weight: normal; font-style: normal; white-space: pre-wrap; }
a img { border: 0px; }body {font-family: Times New Roman;font-size: 12pt;}
.plain pre, .plain tt {font-family: Times New Roman;font-size: 12pt;}
</STYLE>
<STYLE></STYLE>
</HEAD>
<BODY scroll=auto class>
<DIV>Thanks for the update Michael. </DIV>
<DIV> </DIV>
<DIV> </DIV>
<DIV id=x184e3b0d789d4c60b229d90b621b08ba>
<BLOCKQUOTE class=cite2 cite=CA+czFiDwVA84ShXhW6FDPiDCZbJVXvxCHYftj-6eSJjcVAwQBg@mail.gmail.com type="cite">
<DIV dir=ltr>That's more a limitation in the gluster protocol as it currently sits, not so much with whether you're using FUSE or libgfapi. There are probably other things you can do, though. </DIV></BLOCKQUOTE>
<DIV>I thought that less "trips" were required when using libgfapi, and hence the likely better performance. </DIV>
<DIV><BR> </DIV>
<BLOCKQUOTE class=cite2 cite=CA+czFiDwVA84ShXhW6FDPiDCZbJVXvxCHYftj-6eSJjcVAwQBg@mail.gmail.com type="cite">
<DIV dir=ltr>
<DIV>One might be to use dispersion instead of replication. Another might be to disable atime on the server. Per this thread ( <A href="http://lists.gnu.org/archive/html/gluster-devel/2008-03/msg00149.html">http://lists.gnu.org/archive/html/gluster-devel/2008-03/msg00149.html</A> ), if the server fs does not have atime enabled, gluster won't update it. That may well save you a few network round trips.</DIV></DIV></BLOCKQUOTE>
<DIV>Already do this. We use noatime on all of the brick mounts.</DIV>
<DIV> <BR></DIV>
<BLOCKQUOTE class=cite2 cite=CA+czFiDwVA84ShXhW6FDPiDCZbJVXvxCHYftj-6eSJjcVAwQBg@mail.gmail.com type="cite">
<DIV dir=ltr>
<DIV>Another thing you might try is taking steps to ensure your ARP caches on your clients and bricks remain hot. (Random guess, here).</DIV></DIV></BLOCKQUOTE>
<DIV>No idea how to do this... </DIV>
<DIV> </DIV>
<BLOCKQUOTE class=cite2 cite=CA+czFiDwVA84ShXhW6FDPiDCZbJVXvxCHYftj-6eSJjcVAwQBg@mail.gmail.com type="cite">
<DIV dir=ltr>
<DIV></DIV>
<DIV><BR></DIV>
<DIV>I've seen feature request tickets about bundling IOPs on the wire to opportunistically eliminate round trips, but I don't know what the status is of those.</DIV></DIV></BLOCKQUOTE>
<DIV> </DIV>
<DIV>Copy that... Thanks again... David</DIV>
<DIV> </DIV>
<DIV> </DIV>
<BLOCKQUOTE class=cite2 cite=CA+czFiDwVA84ShXhW6FDPiDCZbJVXvxCHYftj-6eSJjcVAwQBg@mail.gmail.com type="cite">
<DIV dir=ltr>
<DIV><BR>
<DIV><BR>
<DIV class=gmail_quote>
<DIV dir=ltr>On Thu, Jul 30, 2015 at 12:07 PM David Robinson <<A href="mailto:david.robinson@corvidtec.com">david.robinson@corvidtec.com</A>> wrote:<BR></DIV>
<BLOCKQUOTE class=gmail_quote style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">Copy that. I am very aware of the small file performance issues and I<BR>have been tracking 3.7. I am waiting for 3.7 to stabilize before I roll<BR>it out to my production system.<BR><BR>My thought was that it looks like Bareos now has built in use of<BR>libgfapi, which from my understanding should improve performance for<BR>gluster. My issues with rsync aren't with the actual transfer speed of<BR>the files. My issue is with the time it takes to search through the<BR>filesystem to figure out which files to transfer. This takes an<BR>extremely long time on 400TB of data, especially when it is going<BR>through the directories with large numbers of small files.<BR><BR>I was curious if Bareos with built-in support for libgfapi would be<BR>faster than rsync backups between gluster machines.<BR>Anyone on the email list care to comment? Thanks in advance for any<BR>info that can be provided.<BR><BR>David<BR><BR><BR>------ Original Message ------<BR>From: "André Bauer" <<A href="mailto:abauer@magix.net">abauer@magix.net</A>><BR>To: "David F. Robinson" <<A href="mailto:david.robinson@corvidtec.com">david.robinson@corvidtec.com</A>><BR>Sent: 7/30/2015 11:35:21 AM<BR>Subject: Re: [Gluster-users] Bareos backup from Gluster mount<BR><BR>>Hi David,<BR>><BR>>i never used Bareos until now. We like to switch from Bacula in the<BR>>future but i think this will not happen before next Ubuntu LTS release<BR>>(16.04).<BR>><BR>>I also never directly compared with rsync but i think rsync is faster<BR>>in<BR>>transfering because it does not have to do any compression and so on...<BR>><BR>>What i can say about Bacula on Glusterfs volumes is, that copying big<BR>>files works at reasonable speed while small files (especialy if there<BR>>are a lot) are a bit slow, whats in Glusterfs nature until versions<BR>>prior 3.6(?).<BR>><BR>>With Glusterfs 3.6 / 3.7 this should be a bit faster in the meantime<BR>>but<BR>>i have no experience with the performance gains because i'm still on<BR>>Glusterfs 3.5.5.<BR>><BR>>In conclusion i still prefer Bacula over Rsync even if its slower.<BR>><BR>>Some more info about Glusterfs small file performance can be found<BR>>here:<BR>><BR>><A href="https://gluster.readthedocs.org/en/latest/Feature%20Planning/GlusterFS%203.7/Small%20File%20Performance/" rel=noreferrer>https://gluster.readthedocs.org/en/latest/Feature%20Planning/GlusterFS%203.7/Small%20File%20Performance/</A><BR>><BR>>Regards<BR>>André<BR>><BR>><BR>><BR>>Am 30.07.2015 um 15:23 schrieb David F. Robinson:<BR>>> Andre,<BR>>><BR>>> I am looking at a backup alternative to rsnc for gluster. My storage<BR>>>system is growing and rsync takes too long on my system (300TB). Do<BR>>>you have any idea of the relative performance of bareos as compared to<BR>>>that of rsync? Can it be run in a multi-threaded mode? Rsync takes an<BR>>>extremely long time just searching the directory tree to figure out<BR>>>what to copy. Before digging into bareos, I was wondering if you had<BR>>>any thoughts on performance for gluster.<BR>>><BR>>> David (Sent from mobile)<BR>>><BR>>> ===============================<BR>>> David F. Robinson, Ph.D.<BR>>> President - Corvid Technologies<BR>>> 704.799.6944 x101 [office]<BR>>> 704.252.1310 [cell]<BR>>> 704.799.7974 [fax]<BR>>> <A href="mailto:David.Robinson@corvidtec.com">David.Robinson@corvidtec.com</A><BR>>> <A href="http://www.corvidtechnologies.com/" rel=noreferrer>http://www.corvidtechnologies.com</A><BR>>><BR>>>> On Jul 29, 2015, at 1:36 PM, André Bauer <<A href="mailto:abauer@magix.net">abauer@magix.net</A>> wrote:<BR>>>><BR>>>> We're using Bacula (Bareos is a fork of it) for backups.<BR>>>> Never had any problems doing backups of Gluster volumes.<BR>>>><BR>>>>> Am 27.07.2015 um 23:02 schrieb Ryan Clough:<BR>>>>> Hello,<BR>>>>><BR>>>>> I have cross-posted this question in the bareos-users mailing list.<BR>>>>><BR>>>>> Wondering if anyone has tried this because I am unable to backup<BR>>>>>data<BR>>>>> that is mounted via Gluster Fuse or Gluster NFS. Basically, I have<BR>>>>>the<BR>>>>> Gluster volume mounted on the Bareos Director which also has the<BR>>>>>tape<BR>>>>> changer attached.<BR>>>>><BR>>>>> Here is some information about versions:<BR>>>>> Bareos version 14.2.2<BR>>>>> Gluster version 3.7.2<BR>>>>> Scientific Linux version 6.6<BR>>>>><BR>>>>> Our Gluster volume consists of two nodes in distribute only. Here<BR>>>>>is the<BR>>>>> configuration of our volume:<BR>>>>> [root@hgluster02 ~]# gluster volume info<BR>>>>><BR>>>>> Volume Name: export_volume<BR>>>>> Type: Distribute<BR>>>>> Volume ID: c74cc970-31e2-4924-a244-4c70d958dadb<BR>>>>> Status: Started<BR>>>>> Number of Bricks: 2<BR>>>>> Transport-type: tcp<BR>>>>> Bricks:<BR>>>>> Brick1: hgluster01:/gluster_data<BR>>>>> Brick2: hgluster02:/gluster_data<BR>>>>> Options Reconfigured:<BR>>>>> performance.io-thread-count: 24<BR>>>>> server.event-threads: 20<BR>>>>> client.event-threads: 4<BR>>>>> performance.readdir-ahead: on<BR>>>>> features.inode-quota: on<BR>>>>> features.quota: on<BR>>>>> nfs.disable: off<BR>>>>> auth.allow: 192.168.10.*,10.0.10.*,10.8.0.*,10.2.0.*,10.0.60.*<BR>>>>> server.allow-insecure: on<BR>>>>> server.root-squash: on<BR>>>>> performance.read-ahead: on<BR>>>>> features.quota-deem-statfs: on<BR>>>>> diagnostics.brick-log-level: WARNING<BR>>>>><BR>>>>> When I try to backup a directory from Gluster Fuse or Gluster NFS<BR>>>>>mount<BR>>>>> and I monitor the network communication I only see data being<BR>>>>>pulled<BR>>>>> from the hgluster01 brick. When the job finishes Bareos thinks that<BR>>>>>it<BR>>>>> completed without error but included in the messages for the job<BR>>>>>are<BR>>>>> lots and lots of permission denied errors like this:<BR>>>>> 15-Jul 02:03 ripper.red.dsic.com-fd JobId 613: Cannot open<BR>>>>> "/export/rclough/psdv-2014-archives-2/scan_111.tar.bak":<BR>>>>>ERR=Permission<BR>>>>> denied.<BR>>>>> 15-Jul 02:03 ripper.red.dsic.com-fd JobId 613: Cannot open<BR>>>>> "/export/rclough/psdv-2014-archives-2/run_219.tar.bak":<BR>>>>>ERR=Permission<BR>>>>> denied.<BR>>>>> 15-Jul 02:03 ripper.red.dsic.com-fd JobId 613: Cannot open<BR>>>>> "/export/rclough/psdv-2014-archives-2/scan_112.tar.bak":<BR>>>>>ERR=Permission<BR>>>>> denied.<BR>>>>> 15-Jul 02:03 ripper.red.dsic.com-fd JobId 613: Cannot open<BR>>>>> "/export/rclough/psdv-2014-archives-2/run_220.tar.bak":<BR>>>>>ERR=Permission<BR>>>>> denied.<BR>>>>> 15-Jul 02:03 ripper.red.dsic.com-fd JobId 613: Cannot open<BR>>>>> "/export/rclough/psdv-2014-archives-2/scan_114.tar.bak":<BR>>>>>ERR=Permission<BR>>>>> denied.<BR>>>>><BR>>>>> At first I thought this might be a root-squash problem but, if I<BR>>>>>try to<BR>>>>> read/copy a file using the root user from the Bareos server that is<BR>>>>> trying to do the backup, I can read files just fine.<BR>>>>><BR>>>>> When the job finishes is reports that it finished "OK -- with<BR>>>>>warnings"<BR>>>>> but, again the log for the job is filled with "ERR=Permission<BR>>>>>denied"<BR>>>>> messages. In my opinion, this job did not finish OK and should be<BR>>>>> Failed. Some of the files from the HGluster02 brick are backed up<BR>>>>>but<BR>>>>> all of the ones with permission errors do not. When I restore the<BR>>>>>job,<BR>>>>> all of the files with permission errors are empty.<BR>>>>><BR>>>>> Has anyone successfully used Bareos to backup data from Gluster<BR>>>>>mounts?<BR>>>>> This is an important use case for us because this is the largest<BR>>>>>single<BR>>>>> volume that we have to prepare large amounts of data to be<BR>>>>>archived.<BR>>>>><BR>>>>> Thank you for your time,<BR>>>>> ___________________________________________<BR>>>>> ¯\_(ツ)_/¯<BR>>>>> Ryan Clough<BR>>>>> Information Systems<BR>>>>> Decision Sciences International Corporation<BR>>>>><BR>>>>><<A href="http://www.decisionsciencescorp.com/" rel=noreferrer>http://www.decisionsciencescorp.com/</A>><<A href="http://www.decisionsciencescorp.com/" rel=noreferrer>http://www.decisionsciencescorp.com/</A>><BR>>>>><BR>>>>> This email and its contents are confidential. If you are not the<BR>>>>> intended recipient, please do not disclose or use the information<BR>>>>>within<BR>>>>> this email or its attachments. If you have received this email in<BR>>>>>error,<BR>>>>> please report the error to the sender by return email and delete<BR>>>>>this<BR>>>>> communication from your records.<BR>>>>><BR>>>>><BR>>>>> _______________________________________________<BR>>>>> Gluster-users mailing list<BR>>>>> <A href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</A><BR>>>>> <A href="http://www.gluster.org/mailman/listinfo/gluster-users" rel=noreferrer>http://www.gluster.org/mailman/listinfo/gluster-users</A><BR>>>>><BR>>>><BR>>>><BR>>>> --<BR>>>> Mit freundlichen Grüßen<BR>>>> André Bauer<BR>>>><BR>>>> MAGIX Software GmbH<BR>>>> André Bauer<BR>>>> Administrator<BR>>>> August-Bebel-Straße 48<BR>>>> 01219 Dresden<BR>>>> GERMANY<BR>>>><BR>>>> tel.: 0351 41884875<BR>>>> e-mail: <A href="mailto:abauer@magix.net">abauer@magix.net</A><BR>>>> <A href="mailto:abauer@magix.net">abauer@magix.net</A> <mailto:<A href="mailto:Email">Email</A>><BR>>>> <A href="http://www.magix.com/" rel=noreferrer>www.magix.com</A> <<A href="http://www.magix.com/" rel=noreferrer>http://www.magix.com/</A>><BR>>>><BR>>>><BR>>>> Geschäftsführer | Managing Directors: Dr. Arnd Schröder, Michael<BR>>>>Keith<BR>>>> Amtsgericht | Commercial Register: Berlin Charlottenburg, HRB 127205<BR>>>><BR>>>> Find us on:<BR>>>><BR>>>> <<A href="http://www.facebook.com/MAGIX" rel=noreferrer>http://www.facebook.com/MAGIX</A>> <<A href="http://www.twitter.com/magix_de" rel=noreferrer>http://www.twitter.com/magix_de</A>><BR>>>> <<A href="http://www.youtube.com/wwwmagixcom" rel=noreferrer>http://www.youtube.com/wwwmagixcom</A>> <<A href="http://www.magixmagazin.de/" rel=noreferrer>http://www.magixmagazin.de</A>><BR>>>><BR>>>>----------------------------------------------------------------------<BR>>>> The information in this email is intended only for the addressee<BR>>>>named<BR>>>> above. Access to this email by anyone else is unauthorized. If you<BR>>>>are<BR>>>> not the intended recipient of this message any disclosure, copying,<BR>>>> distribution or any action taken in reliance on it is prohibited and<BR>>>> may be unlawful. MAGIX does not warrant that any attachments are<BR>>>>free<BR>>>> from viruses or other defects and accepts no liability for any<BR>>>>losses<BR>>>> resulting from infected email transmissions. Please note that any<BR>>>> views expressed in this email may be those of the originator and do<BR>>>> not necessarily represent the agenda of the company.<BR>>>><BR>>>>----------------------------------------------------------------------<BR>>>> _______________________________________________<BR>>>> Gluster-users mailing list<BR>>>> <A href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</A><BR>>>> <A href="http://www.gluster.org/mailman/listinfo/gluster-users" rel=noreferrer>http://www.gluster.org/mailman/listinfo/gluster-users</A><BR>>><BR>><BR>><BR>>--<BR>>Mit freundlichen Grüßen<BR>>André Bauer<BR>><BR>>MAGIX Software GmbH<BR>>André Bauer<BR>>Administrator<BR>>August-Bebel-Straße 48<BR>>01219 Dresden<BR>>GERMANY<BR>><BR>>tel.: 0351 41884875<BR>>e-mail: <A href="mailto:abauer@magix.net">abauer@magix.net</A><BR>><A href="mailto:abauer@magix.net">abauer@magix.net</A> <mailto:<A href="mailto:Email">Email</A>><BR>><A href="http://www.magix.com/" rel=noreferrer>www.magix.com</A> <<A href="http://www.magix.com/" rel=noreferrer>http://www.magix.com/</A>><BR>><BR>><BR>>Geschäftsführer | Managing Directors: Dr. Arnd Schröder, Michael Keith<BR>>Amtsgericht | Commercial Register: Berlin Charlottenburg, HRB 127205<BR>><BR>>Find us on:<BR>><BR>><<A href="http://www.facebook.com/MAGIX" rel=noreferrer>http://www.facebook.com/MAGIX</A>> <<A href="http://www.twitter.com/magix_de" rel=noreferrer>http://www.twitter.com/magix_de</A>><BR>><<A href="http://www.youtube.com/wwwmagixcom" rel=noreferrer>http://www.youtube.com/wwwmagixcom</A>> <<A href="http://www.magixmagazin.de/" rel=noreferrer>http://www.magixmagazin.de</A>><BR>>----------------------------------------------------------------------<BR>>The information in this email is intended only for the addressee named<BR>>above. Access to this email by anyone else is unauthorized. If you are<BR>>not the intended recipient of this message any disclosure, copying,<BR>>distribution or any action taken in reliance on it is prohibited and<BR>>may be unlawful. MAGIX does not warrant that any attachments are free<BR>>from viruses or other defects and accepts no liability for any losses<BR>>resulting from infected email transmissions. Please note that any<BR>>views expressed in this email may be those of the originator and do<BR>>not necessarily represent the agenda of the company.<BR>>----------------------------------------------------------------------<BR><BR>_______________________________________________<BR>Gluster-users mailing list<BR><A href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</A><BR><A href="http://www.gluster.org/mailman/listinfo/gluster-users" rel=noreferrer>http://www.gluster.org/mailman/listinfo/gluster-users</A></BLOCKQUOTE></DIV></DIV></DIV></DIV></BLOCKQUOTE></DIV></BODY></HTML>