Advanced Python - How to Dynamically Load Modules or Classes

Every now and then you'll find yourself needing to load modules or classes dynamically. In other words, you'll want to be able to import a module without knowing ahead of time which one you're going to import. In this article, we'll look at two ways to accomplish this feat in Python.

Using the __import__ Magic Method

The easiest way to do this sort of thing is to use the "magic" method, __import__. In fact, if you did a Google search on this topic, that's probably the first method you'll find. Here's the basic methodology:

module = __import__(module_name)
my_class = getattr(module, class_name)
instance = my_class()

Both module_name and class_name have to be strings in the above code. If the class you're importing requires some parameters passed to it, then you'll have to add that logic too. Here's a more fleshed out example to help you understand how this works:

########################################################################
class DynamicImporter:
    """"""

    #----------------------------------------------------------------------
    def __init__(self, module_name, class_name):
        """Constructor"""
        module = __import__(module_name)
        my_class = getattr(module, class_name)
        instance = my_class()
        print instance
                
if __name__ == "__main__":
    DynamicImporter("decimal", "Context")

If you run this code, you should see something like the following output to stdout:

Context(prec=28, rounding=ROUND_HALF_EVEN, Emin=-999999999, Emax=999999999, capitals=1, flags=[], traps=[DivisionByZero, Overflow, InvalidOperation])

This shows that the code works as expected in that it imports the decimal and returns an instance of the Context class. That was pretty straight-forward. Let's look at the other method of doing this!

Using Python's imp Module

Using the imp module is a little bit more complicated. You end up needing to do a little recursive call and you'll want to wrap everything in exception handlers too. Let's take a look at the code and then I'll explain why:

import imp
import sys

#----------------------------------------------------------------------
def dynamic_importer(name, class_name):
    """
    Dynamically imports modules / classes
    """
    try:
        fp, pathname, description = imp.find_module(name)
    except ImportError:
        print "unable to locate module: " + name
        return (None, None)
    
    try:
        example_package = imp.load_module(name, fp, pathname, description)
    except Exception, e:
        print e
        
    try:
        myclass = imp.load_module("%s.%s" % (name, class_name), fp, pathname, description)
        print myclass
    except Exception, e:
        print e
        
    return example_package, myclass
    
if __name__ == "__main__":
    module, modClass = dynamic_importer("decimal", "Context")

The imp module comes with a find_module method that will look for the module for you. I wasn't able to get it to always work reliably though, so I wrapped it in a try/except. For example, it couldn't find SQLAlchemy at all and when I tried to find wx.Frame, I got it, but not the right one. I don't know if that latter issue is with wxPython or imp though. Anyway, after imp finds the module, it returns an open file handler, the path to the module that imp found and a description of sorts (see the docs or the PyMOTW article below). Next you'll need to load the module to get it "imported". If you want to pull out a class inside the module, then you use load_module again. At this point, you should have the same objects that you had in the first method.

Update: I've had one commenter utterly trash this article. So in response, I wanted to clear up a few things. First of all, bare try/excepts are usually bad. I fixed the code, but I see it all the time on blogs, in production code and even in books. Is it a good idea? No. If you know what errors to expect, you should handle them or re-raise the error. Secondly, I've been told that "__import__" isn't a "method". True enough. It's actually a function in the internals of Python. However, every double-underscore function in Python that I've seen are called "magic methods". See Foord's book or this blog that lists a bunch of other resources on the topic. Finally, my IDE (Wingware if anyone cares) adds some silly stuff to classes and functions/methods all by itself, such as empty docstrings or in the __init__ docstring, it will add "Constructor".

Additional Reading

• StackOverflow: Python dynamic instantiation from string name of a class in dynamically imported module
• StackOverflow: How can I import a package using __import__() when the package name is only known at runtime?
• PyMOTW: imp – Interface to module import mechanism
• Python official documentation for the imp module