[Gluster-users] Regarding 2.0.4 booster

Sudipto Mukhopadhyay Sudipto.Mukhopadhyay at openwave.com
Tue Jul 21 07:22:53 UTC 2009


This could be tricky as you don't want too lookup too many
alternatives!!
But, as you are doing LD_PRELOAD, can you not ask the application to
specify the paths (I know it's going to be little error prone based on
what application supplies) 

For example: 

/mnt/glusterfs 
If the application run dir is /mnt/application then, 
../glusterfs is another way to get to the glusterfs. 

So, for both the paths the booster intercepts the calls...

Thanks and regards,
Sudipto

-----Original Message-----
From: Shehjar Tikoo [mailto:shehjart at gluster.com] 
Sent: Monday, July 20, 2009 11:28 PM
To: Sudipto Mukhopadhyay
Cc: service at gluster.com; avati at gluster.com; gluster-users at gluster.org
Subject: Re: [Gluster-users] Regarding 2.0.4 booster

Sudipto Mukhopadhyay wrote:
> Makes sense. 
> Yes I am running the same program. 
> I will be running couple of more tests to verify this. 
> BTW. two more questions on the related topics: 
> 
> 1. How much of performance boost does booster provide?

That it provides a performance boost is evident from some of our tests
and from reports from users. The exact improvement really
depends on various factors such as the volume configuration,
the type of translators used, the network setup, etc.

> 2. does the following
>
http://www.gluster.org/docs/index.php/BoosterConfiguration#Virtual_Mount
> _Points
> mean that from the application we always need to use absolute path to
> the files being operated? 
> 

That is a good question. I've also considered the need to remove
the dependency on absolute paths but the use cases have been limited
or none, till now to help me evolve the exact behaviour. Could you
please describe what exactly you have in mind? I could take it from
there.

One approach is to redirect file system operations into GlusterFS
when booster sees a string identifier or a token prepended to a path.
This token could be specified through the booster.conf file.
Since there will be no / prepended to this string, it does not
remain an absolute path anymore. Also, since this is a global
string identifier for booster, it is very different from a relative
path, so can be used from anywhere in the local file system tree,
as long as booster knows where to redirect these operations.

Thanks
Shehjar



> Thanks and regards,
> Sudipto
>  
> 
> 
> -----Original Message-----
> From: Shehjar Tikoo [mailto:shehjart at gluster.com] 
> Sent: Monday, July 20, 2009 9:46 PM
> To: Sudipto Mukhopadhyay
> Cc: service at gluster.com; avati at gluster.com; gluster-users at gluster.org
> Subject: Re: [Gluster-users] Regarding 2.0.4 booster
> 
> Sudipto Mukhopadhyay wrote:
>> Hi,
>>
>>  
>>
>> After modifying the booster.conf, I started getting the logs under 
>> /tmp/booster.log but, I see the following error:
>>
>>  
>>
>>  
>>
>> [2009-07-20 12:24:58] D [booster.c:1025:booster_init] booster:
booster
> 
>> is inited
>>
>> [2009-07-20 12:24:58] D 
>> [libglusterfsclient.c:1340:libgf_vmp_search_entry]
libglusterfsclient:
> 
>> VMP Entry found: /mnt/glusterfs/abc.txt: /mnt/glusterfs/
>>
>> [2009-07-20 12:24:58] N [client-protocol.c:5559:client_setvolume_cbk]

>> remote1: Connected to 10.16.80.53:6996, attached to remote volume
> 'afr'.
>> [2009-07-20 12:24:58] N [client-protocol.c:5559:client_setvolume_cbk]

>> remote1: Connected to 10.16.80.53:6996, attached to remote volume
> 'afr'.
>> [2009-07-20 12:24:58] N [client-protocol.c:5559:client_setvolume_cbk]

>> remote2: Connected to 10.16.80.55:6996, attached to remote volume
> 'afr'.
>> [2009-07-20 12:24:58] D 
>> [libglusterfsclient-dentry.c:224:__do_path_resolve] 
>> libglusterfsclient-dentry: loc->parent(1) already present. sending 
>> lookup for 1//abc.txt
>>
>> [2009-07-20 12:24:58] D 
>> [libglusterfsclient-dentry.c:245:__do_path_resolve] 
>> libglusterfsclient-dentry: resolved path(/abc.txt) till 1(/). sending

>> lookup for remaining path
>>
>> [2009-07-20 12:24:58] D 
>> [libglusterfsclient.c:1608:libgf_client_lookup_cbk]
> libglusterfsclient: 
>> 1: (op_num=0) /abc.txt => -1 (No such file or directory)
>>
>> [2009-07-20 12:24:58] E
[libglusterfsclient.c:2447:glusterfs_glh_open]
> 
>> libglusterfsclient: path lookup failed for (/abc.txt)
>>
> 
> Assuming you're the using the same program as you'd pasted earlier,
> this last line is an error that says the file was not found in
> the file system when trying to open it. This is the reason why
> your program took the file creation branch in the "if" block.
> 
> We can be sure about this error if you could confirm, before running
> your test program, whether the file really did not exist on the
backend.
> 
> Regards
> -Shehjar
> 
> 
> 
>>  
>>
>>  
>>
>> Can you please explain the above errors and what is that causing
these
> 
>> errors?
>>
>>  
>>
>> Thanks and regards,
>>
>> Sudipto
>>
>>  
>>
>>  
>>
>>  
>>
>> -----Original Message-----
>> From: Sudipto Mukhopadhyay
>> Sent: Monday, July 20, 2009 3:59 PM
>> To: Shehjar Tikoo
>> Cc: service at gluster.com; avati at gluster.com; gluster-users at gluster.org
>> Subject: RE: [Gluster-users] Regarding 2.0.4 booster
>>
>>  
>>
>> Hi Shehjar,
>>
>>  
>>
>> The contents in booster.conf are in two separate lines.
>>
>> I will try out the test again by merging the contents of booster.conf

>> into one single line after I get my nodes back.
>>
>>  
>>
>> Thanks and regards,
>>
>> Sudipto
>>
>>  
>>
>> -----Original Message-----
>>
>> From: Shehjar Tikoo [mailto:shehjart at gluster.com]
>>
>> Sent: Thursday, July 16, 2009 10:08 PM
>>
>> To: Sudipto Mukhopadhyay
>>
>> Cc: service at gluster.com; avati at gluster.com; gluster-users at gluster.org
>>
>> Subject: Re: [Gluster-users] Regarding 2.0.4 booster
>>
>>  
>>
>> Sudipto Mukhopadhyay wrote:
>>
>>>  My mistake I meant booster log is not getting generated.
>>>  Thanks and regards,
>>>  Sudipto
>>  
>>
>>  
>>
>> Hi
>>
>>  
>>
>> A few questions. Please see inlined text.
>>
>>  
>>
>>>  -----Original Message-----
>>>  From: gluster-users-bounces at gluster.org
>>>  [mailto:gluster-users-bounces at gluster.org] On Behalf Of Sudipto
>>>  Mukhopadhyay
>>>  Sent: Tuesday, July 14, 2009 2:07 PM
>>>  To: Shehjar Tikoo
>>>  Cc: avati at gluster.com; gluster-users at gluster.org
>>>  Subject: Re: [Gluster-users] Regarding 2.0.4 booster
>>>  Hi Shehjar,
>>>  Thanks for looking into this issue.
>>>  The file abc.txt is getting created; if you look at the C program;
> the
>>>  following line is basically printing out the file handle on stdout:
>>>  printf ("File handle %d\n",fh);
>>>  But, the booster log is getting generated.
>>>  Thanks and regards,
>>>  Sudipto
>>>  -----Original Message-----
>>>  From: Shehjar Tikoo [mailto:shehjart at gluster.com]
>>>  Sent: Tuesday, July 14, 2009 2:34 AM
>>>  To: Sudipto Mukhopadhyay
>>>  Cc: gluster-users at gluster.org; avati at gluster.com
>>>  Subject: Re: [Gluster-users] Regarding 2.0.4 booster
>>>  Sudipto Mukhopadhyay wrote:
>>>> Hi,
>>>> I trying to run some tests w/booster client library and I had to
>>>> upgrade the glusterfs to version 2.0.4 from 2.0.2 (due to
> LD_PRELOAD
>>>> failures of the booster client library,
>
http://git.savannah.gnu.org/cgit/gluster.git/commit/?id=a3ece0caa52ad2ea
>>>  cf8a8691aaca53295cde972f).
>>>> Now, w/version 2.0.4 the LD_PRELOAD works OK, but the booster logs
>>>> are not generating and I am not sure whether booster is working or,
>>>> not.
>>>> I have the following booster.conf and client vol specification:
>>>> $cat /etc/glusterfs/booster.conf
>>>> /etc/glusterfs/glusterfs-client.vol /mnt/glusterfs glusterfs
>>>> subvolume=brick,logfile=/tmp/booster.log,loglevel=DEBUG
>>  
>>
>> Please confirm whether the above content in booster.conf is on
>>
>> a single line or two separate lines. It should all be on one line for
>>
>> the logfile setting to be associated with /mnt/glusterfs.
>>
>>  
>>
>>>> $cat /etc/glusterfs/glusterfs-client.vol
>>>> volume brick
>>>> type protocol/client
>>>> option transport-type tcp/client # for TCP/IP transport
>>>> option remote-host 10.16.80.53   # IP address of the server
>>>> option remote-subvolume afr      # name of the remote volume
>>>> end-volume
>>>> volume writebehind
>>>> type performance/write-behind
>>>> option window-size 1MB
>>>> subvolumes brick
>>>> end-volume
>>>> I have written a simple program to test few system calls and the
>>>> booster functionality:
>>>> int main() {
>>>> int fh;
>>>> char buffer[65];
>>>> fh = open("/mnt/glusterfs/abc.txt",O_WRONLY);
>>>> if (fh == -1) {
>>>> fh = creat("/mnt/glusterfs/abc.txt",O_WRONLY);
>>>> }
>>>> printf ("File handle %d\n",fh);
>>>> close(fh);
>>>> return 0;
>>>> }
>>>> When I run my program with LD_PRELOAD option, I get the following
>>>> message:
>>>> LD_PRELOAD="/usr/local/lib/glusterfs/glusterfs-booster.so"
>>>> /root/fstest/a.out
>>>> [2009-07-13 16:19:02] E
>>>> [libglusterfsclient.c:2447:glusterfs_glh_open] libglusterfsclient:
>>>> path lookup failed for (/abc.txt)
>>  
>>
>> Is the above log message the only message that is being output? Were
>>
>> there more lines that resembled this log line that you probably
>>
>> removed for the purpose of this email?
>>
>>  
>>
>> Thanks
>>
>> Shehjar
>>
>>  
>>
>>  
>>
>>>> File handle 4
>>>> And there is no /tmp/booster.log created. The above log is only
>>>> observed while creating the file (in case abc.txt is not present).
>>>> Could you please advice on the booster config file and let me know
>>>> why I am not seeing any booster log?
>>>  I'll check why the booster log is not getting generated.
>>>  In the mean time, can you check if the file abc.txt really does not
>>>  get created on the backend?
>>>  If open is failing, it looks like the fd 4 is being returned by
>>>  creat call.
>>>  -Shehjar
>>>> Appreciate your help.
>>>> Thanks and regards,
>>>> Sudipto
>
------------------------------------------------------------------------
>>>> _______________________________________________ Gluster-users
> mailing
>>>>  list Gluster-users at gluster.org
>>>> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
>>>  _______________________________________________
>>>  Gluster-users mailing list
>>>  Gluster-users at gluster.org
>>>  http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
>>  
>>
> 
> 





More information about the Gluster-users mailing list