Thursday, August 6, 2015

Basics of Metaclasses

This is a quick tutorial over the basics of what metaclasses do.

The Metaclass

Metaclasses, while seemingly a complex topic, really just do something very simple. They control what happens when you have code that turns into a class object. The normal place they are executed is right after the class statement. Let's see that in action by using print as our metaclass.

Note: this post uses Python 3 metaclass notation. Python 2 uses assignment to a special __metaclass__ attribute to set the metaclass. Also, Python 2 requires explicit, 2 argument super() calls.

In [13]:
class WillNotBeAClass(object, metaclass=print):
    x=1
WillNotBeAClass (<class 'object'>,) {'x': 1, '__module__': '__main__', '__qualname__': 'WillNotBeAClass'}

Here, we have replaced the metaclass (type) with print, just to investigate how it works. This is quite useless, of course, but does show that the metaclass gets called with three arguments when a class is created. The first is the name of the class to be created, the second is a tuple of base classes, and the third is a dictionary that has the namespace of the body of a class, with a few extra special values added.

Given this, we know see show to make this into a class using type: (I will not bother to add __module__ and __qualname__ for now, they are not needed)

In [16]:
class WillBeAClass(object):
    x=1
WillBeAClass
Out[16]:
__main__.WillBeAClass
In [17]:
WillAlsoBeAClass = type('WillAlsoBeAClass', (object,), {'x':1})
WillAlsoBeAClass
Out[17]:
__main__.WillAlsoBeAClass

These two objects, WillBeAClass and WillAlsoBeAClass, are basically the same thing. The second method is exactly what the class statement does (with __module__ and __qualname__ added).

The type

So, we are done with metaclasses, that's all there is to know. However, to actually make useful classes, you probably want to make normal classes, just with some sort of modification. For that, you need to understand type, and how it works, and how to subclass it.

First, let's pretend we can just patch type and ignore subclassing. You probably already see how to do that:

In [24]:
def newtype(*args):
    print("I'm sort of a new type, but I have a problem!")
    return type(*args)
In [27]:
class NewClass(object, metaclass=newtype):
    x = 1
I'm sort of a new type, but I have a problem!
In [28]:
class NewNewClass(NewClass):
    y = 2

All was fine and well, until we subclassed NewClass. The metaclass did not come along for the ride! That's because type adds a reference to itself when it creates a class:

In [30]:
NewClass.__class__
Out[30]:
type

Note: the standard way to check the class of an object is to call type(NewClass), however, since that is an unrelated use of type that is there for historical reasons, I've avoided using it here)

How type works

We must subclass type to get a metaclass that actually works on subclasses, too: (Here I'm overriding all the used parameters, so that you can see where each gets called)

In [41]:
class NewType(type):
    def __new__(cls, *args, **kargs):
        print("I'm a new type! __new__")
        return super().__new__(cls, *args, **kargs)
    def __init__(self, *args, **kargs):
        print("I'm a new type! __init__")
        super().__init__(*args, **kargs)
    @classmethod
    def __prepare__(cls, *args, **kargs):
        print("I'm new in Python 3! __prepare__")
        return super().__prepare__(cls, *args, **kargs)
    def __call__(self, *args, **kargs):
        print("I'm a new type! __call__")
        return super().__call__(*args, **kargs)
In [42]:
class NewClass(object, metaclass=NewType):
    def __init__(self):
        print("I'm init in the class")
    def __new__(cls):
        print("I'm new")
        return super().__new__(cls)
I'm new in Python 3! __prepare__
I'm a new type! __new__
I'm a new type! __init__
In [17]:
class NewNewClass(NewClass):
    y = 2
I'm new in Python 3! __prepare__
I'm a new type! __new__
I'm a new type! __init__
In [18]:
instance = NewClass()
I'm a new type! __call__
I'm new
I'm init in the class

Notice how __init__ was used, too? This gives us a peek at one more feature of metaclasses: the __class__ parameter of a class is used to create instances. The super part of __call__ actually puts together the class, it's where __new__ and __init__ are called, etc.

Python 3 only

As you already have seen, the __prepare__ method is only in Python 3, and allows you to customize the __dict__ before __new__, however, as a reminder, in CPython the dict for a class is written in C and is not customizable (ie, can't be ordered, etc). So you'll have to manage that yourself, but __prepare__ helps. It returns a dictionary-like object that then collects the namespace, then gets passed to __new__.

In [40]:
from collections import OrderedDict
class PrepareMeta(type):
    def __new__(cls, name, bases, ns):
        print(ns)
        return super().__new__(cls, name, bases, ns)

    @classmethod
    def __prepare__(cls, *args, **kargs):
        return OrderedDict()
In [29]:
class PrepareClass(metaclass=PrepareMeta):
    y = 2
OrderedDict([('__module__', '__main__'), ('__qualname__', 'PrepareClass'), ('y', 2)])

We only get that one look at the dict, since it becomes the special C mappingproxy once the class is created.

In [30]:
PrepareClass.__dict__
Out[30]:
mappingproxy({'__doc__': None, '__weakref__': <attribute '__weakref__' of 'PrepareClass' objects>, '__module__': '__main__', 'y': 2, '__dict__': <attribute '__dict__' of 'PrepareClass' objects>})

Another Python 3 only feature is class level arguments. You can do things like this:

In [39]:
class ArgMeta(type):
    def __new__(cls, *args, **kargs):
        print(kargs)
        return super().__new__(cls, *args)
    def __init__(self, *args, **kargs):
        return super().__init__(*args)

class ArgClass(metaclass=ArgMeta, kwarg = 2):
    y = 2
{'kwarg': 2}

Example

A dictionary can be make using the following ugly hack:

In [44]:
class a_dictionary(metaclass=lambda name, bases, ns: {n:ns[n] for n in ns if '__' not in n}):
    one = 1
    two = 2
    three = 3
In [46]:
print(a_dictionary)
{'three': 3, 'one': 1, 'two': 2}

An ordered class can be make using the __prepare__ method (Python 3 only):

In [49]:
class OrderedMeta(type):
    def __new__(cls, name, bases, ns):
        ns['orderednames']= list(ns)
        return super().__new__(cls, name, bases, ns)
    @classmethod
    def __prepare__(cls, *args, **kargs):
        return OrderedDict()
In [55]:
class Ordered(metaclass=OrderedMeta):
    one = 1
    two = 2
    three = 3
    four = 4
    five = 5
In [57]:
print(Ordered.__dict__)
{'one': 1, 'three': 3, 'orderednames': ['__module__', '__qualname__', 'one', 'two', 'three', 'four', 'five'], '__weakref__': <attribute '__weakref__' of 'Ordered' objects>, '__dict__': <attribute '__dict__' of 'Ordered' objects>, '__doc__': None, 'five': 5, 'two': 2, 'four': 4, '__module__': '__main__'}
['__module__', '__qualname__', 'one', 'two', 'three', 'four', 'five']

Here we can see that the list orderednames is ordered correctly.

No comments:

Post a Comment