by on February 1, 2013

Plugin Initialization

Like many programs, GlusterFS has a plugin interface. We call them translators, but the idea is the same – use dlopen to load modules according to configuration, use dlsym to find functions with certain “well known” names, then call those. The problem is that in our case init does the initialization for a single translator object, of which there might be multiple associated with a single shared library, and sometimes you want to do some initialization exactly once for the library as a whole. It’s a common problem, and there are multiple solutions. Let’s go through a few of them.

If all you’re worried about is multiple sequential initialization calls, you have a very easy, robust, and portable solution: a simple static variable.

    static int is_inited = 0;
 
    if (!is_inited) {
        do_one_time_stuff();
        is_inited = 1;
    }

In fact I think this would even work for the GlusterFS translator case, because of the way that translators’ init functions get called. The problem is that you’ll often need to deal with multiple concurrent initialization calls as well. For that, you need to do something slightly more complicated.

    static int is_inited = 0;
    static pthread_mutex_t lock_mutex = PTHREAD_MUTEX_INITIALIZER;
 
    pthread_lock(&lock_mutex);
    if (!is_inited) {
        do_one_time_stuff();
        is_inited = 1;
    }
    pthread_unlock(&mutex);

So far, so good. As it turns out, this is such a common need that pthreads even includes a simpler method.

    static pthread_once_t init_once = PTHREAD_ONCE_INIT;
 
    pthread_once(&init_once,do_one_time_stuff());

This is simple, and it should work even for multiple concurrent calls on any platform that has pthreads. A less obvious benefit is that you can decide first whether you want to do this particular initialization at all. For example, the thing that got me looking at this was the SSL code in GlusterFS. There, we only want to do our OpenSSL initialization if and when we encounter the first connection that uses SSL, and usually there are none. All of the solutions we’ve looked at so far can handle this, so why are we even talking about it? Let’s consider a method that doesn’t handle this as well.

    void __attribute__((constructor))
    do_one_time_stuff (void)
    {
        ...
    }

Yes, the double parens are necessary. Also, it might look like a C++ thing but it works for C as well. This will cause do_one_time_stuff to be called automatically when the library is loaded. You can even wrap up the __attribute__ weirdness in a macro, so it’s even simpler than the pthread_once method. What’s not to like? Portability is one concern. I’d be a bit surprised if LLVM doesn’t support this feature, but it wouldn’t be a total shocker. With other compilers it would even seem quite likely. On the other hand, if you’re using some kind of non-pthreads threading that doesn’t have a pthread_once equivalent, but your compiler does support an __attribute__ equivalent (perhaps with a different syntax) then this might be just the thing for you. Remember, though, that you’ll be giving up the ability to do certain types of initialization conditionally based on run-time state. In those cases you’d be better off with our fifth and last method, which has to be implemented in your main program rather than the plugin itself.

    if (ssl_is_needed()) {
        pi_init = (ssl_init_func_t *)dlsym(dl_handle,"plugin_ssl_init");
        if (pi_init) {
            pi_init(our_ssl_args);
        }
    }

That’s simple (for the plugin author), robust, and portable to any platform that can run your main program. The snippet above doesn’t handle threading issues, which might also be a concern for you. The other disadvantage is that it’s very special-purpose. Instead of plugin authors adding initialization functions as they need to, the core author has to add a special hook each time. That’s not exactly a technical issue, but the whole nature of a plugin interface is that it allows third-party development so the logistical issue is likely to be quite significant.

Finally, let’s look at a solution that doesn’t work, because if I don’t mention it I’m sure someone will present it as the “obvious” answer in the comments.

    void
    _init (void)
    {
        internal_init_1();
        internal_init_2();
    }

This has all the drawbacks of the previous approach, plus one huge show-stopper. The first time you try it, your compiler will probably complain about a conflict with the _init that’s already defined in the standard library. You can work around that in gcc with –nostdlib but then you run into another problem: automake and friends don’t always add or maintain that flag properly for shared libraries. You might be OK or you might not, and anything that might leave people fighting with autobreak has to go on the not-recommended list.

So there you have it – four solutions that work with varying tradeoffs, one that’s redundant (pthread_mutex_init vs. pthread_once), and one that’s clearly inferior (_init vs. __attribute__). Choose carefully.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>