Dabeaz

Recently Raymond Hettinger posted an amazing article Python's super() considered super!". Even if you think you know what super() does, you should go read it.

A commonly cited applications of super() is using it to implement a kind of cooperative inheritance as is sometimes found with mixin classes. Consider this code which is a slight variation of Raymond's example:

class LoggedSetItemMixin:
    def __setitem__(self,index,value):
        logging.info('Setting %r to %r', index,value)
        super().__setitem__(index,value)

Using this class, you could add logging to any class that implements __setitem__() by combining classes via multiple inheritance. For example:

class LoggingDict(LoggedSetItemMixin,dict):
    pass

class LoggingList(LoggedSetItemMixin,list):
    pass

Here's some sample output:

>>> d = LoggingDict()
>>> d['a'] = 1
INFO:root:Setting 'a' to 1
>>> e = LoggingList([0,1,2])
>>> e[0] = 99
INFO:root:Setting 0 to 99
>>>

The whole reason that this works is that super() delegates to the next class on the MRO. Thus, the __setitem__() call in LoggedSetItemMixin actually steps over to the next class in MRO of whatever kind of instance is being used. If you find this amazing, consider the fact that LoggedSetItemMixin is using super() even though it doesn't even specify a base class! It's pretty cool--maybe even a slight bit diabolical.

As amazing as this is, I've recently been thinking about a completely different approach to these kinds of problems based on class decorators. Consider this function:

def LoggedSetItem(cls):
    orig_setitem = cls.__setitem__
    def __setitem__(self, index, value):
        logging.info('Setting %r to %r' ,index, value)
        return orig_setitem(self,index,value)
    cls.__setitem__ = __setitem__
    return cls

This function is meant to be used as a decorator to class definitions. For example:

@LoggedSetItem
class LoggingDict(dict):
    pass

@LoggedSetItem
class LoggingList(list):
    pass

Carefully study the implementation of LoggedSetItem. As input, it receives a class object. It then looks up the unbound __setitem__ method and stores it in a variable. This lookup, as it turns out, is doing exactly the same work as super(). That is, it simply finds the implementation of the method being used by the class regardless of where it is actually located. After that, the function simply defines a replacement for __setitem__ with added logging and attaches it back to the class object. References to the original implementation of __setitem__ are held inside a closure so it all works out.

The class decorator approach has several notable features. First, it doesn't even involve the use of super() (or multiple inheritance for that matter). Second, as with super(), you don't have to hard-code any classnames--the class is simply passed in as an argument. Third,it has very good runtime performance. This is because the work normally performed by super() is only performed once, at the time of class decoration. Finally, there is a kind of built-in error checking. For example, if you try to apply the decorator to a class that doesn't support the required method, you will immediately get an error:

>>> @LoggedSetItem
class loggedint(int): pass

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "logsetitem.py", line 9, in LoggedSetItem
    orig_setitem = cls.__setitem__
AttributeError: type object 'loggedint' has no attribute '__setitem__'
>>> 

As interesting as this is, I have no idea if using class decorators in this manner would be considered to be good practice or not. One potential problem is that by putting the code in a decorator, a lot of the work is performed just once at the time of class definition. If a program was playing sneaky tricks like dynamically changing method definitions at runtime, it clearly wouldn't work. There's also a certain risk that this approach is just too clever for it's own good.

Do you see any other downsides? I'd love to get your feedback.

posted by Dave Beazley  # 2:44 PM