— aleatory

How to Dynamically Discover & Create Classes in Python

ye olde Python logo

Or reasons why Python is cool #273.

There are many snippets of code on the internet detailing how to go about dynamic class creation when the compiler (and thus, you the programmer) knows the name and module location of the class some time in advance, e.g. initial deploy time or via some parameter being entered in by the user when they want to instantiate such a class.

However if I want to create objects without knowing their module/class in advance a little bit more thinking is required. Here is a simple tutorial that will leave you with a package that when updated with new modules will introspect & instantiate their classes on demand. If the classes are defined to the same known interface, they can then be used in an app immediately without any extra code or even restarting.

Step 1: The initial loader

1
2
3
4
5
6
7
8
9
10
11
12
import inspect,sys,pkgutil
 
pkg = 'app.model.dynamic'
__import__(pkg)
package = sys.modules[pkg]
prefix = pkg + "."
 
for importer,modname,ispkg in pkgutil.iter_modules(package.__path__,prefix):
 module = __import__(modname,locals(),[],-1)
 for name,cls in inspect.getmembers(module):
  if inspect.isclass(cls):
   self.obj[cls.__name__] = cls(cls)

The above utility code is run on the initial app load. The package is imported and contained modules are iterated through, each one inspected for members and if classes are instantiated and persisted in the app. Here I add them to a dictionary with the class name as the key.

Step 2: The Class to load

1
2
3
4
5
6
7
8
9
from app.model import interface
 
class DynamicExample(interface.IExample):
 def __init__(self):
  pass
 
 // implement methods from interface.IExample
 def exampleMethod(self):
  print "Hello World!"

The above class would be placed in a module and it’s important that both be named according to a defined convention in relation to the key they are stored as in the self.obj dict in step 1 above, in order to call their methods when we want to use them at a later date. So for my app I will simply have the module named as a lower case version of the class it contains, and a 1:1 relationship between each module and class under the app.model.dynamic package. Thus the above is placed inside a python source file named ‘dynamicexample.py’

Step 3: Calling and using the classes

1
2
3
4
5
for name,cls in self.obj.iteritems():
 module = name.lower()
 pkg = 'app.model.dynamic.'+module
 mod = sys.modules[pkg]
 getattr(mod,name).exampleMethod()

Storing the objects in a dict allows amongst other things, easy batch-style running of maintenance-related jobs common to all. Alternatively individual objects can be specified by their class name if needs be.

Step 4: Dynamic Loading

1
2
3
4
5
6
7
8
pkg = 'app.model.dynamic'
package = sys.modules[pkg]
reload(package)
for importer,modname,ispkg in pkgutil.iter_modules(package.__path__,prefix):
 module = __import__(modname,locals(),[],-1)
 for name,cls in inspect.getmembers(module):
  if inspect.isclass(cls) and cls.__name__ not in self.obj:
   self.obj[cls.__name__] = cls(cls)

reload() is the magic here, allowing the app to import any new modules that may have since been added to the source directory and again store them in self.obj if not already present. A trigger to call the above segment would maybe be a reload button or a file system monitor that notifies when the contents of the source directory gets updated. Once created, I can use an object DB to persist objects in my app. Overall it’s a nice example of convention over configuration and makes updating my app that bit easier as more & more services are added.

0 comments
Submit comment