<commit_before><HTML>

<HEAD>
<TITLE>Metaclasses in Python 1.5</TITLE>
</HEAD>

<BODY BGCOLOR="FFFFFF">

<H1>Metaclasses in Python 1.5</H1>
<H2>(A.k.a. The Killer Joke :-)</H2>

<HR>

(<i>Postscript:</i> reading this essay is probably not the best way to
understand the metaclass hook described here.  See a <A
HREF="meta-vladimir.txt">message posted by <NAME></A>
which may give a gentler introduction to the matter.  You may also
want to search Deja News for messages with "metaclass" in the subject
posted to comp.lang.python in July and August 1998.)

<HR>

<P>In previous Python releases (and still in 1.5), there is something
called the ``Don Beaudry hook'', after its inventor and champion.
This allows C extensions to provide alternate class behavior, thereby
allowing the Python class syntax to be used to define other class-like
entities.  <NAME> has used this in his infamous <A
HREF="http://maigret.cog.brown.edu/pyutil/">MESS</A> package; Jim
Fulton has used it in his <A
HREF="http://www.digicool.com/papers/ExtensionClass.html">Extension
Classes</A> package.  (It has also been referred to as the ``Don
Beaudry <i>hack</i>,'' but that's a misnomer.  There's nothing hackish
about it -- in fact, it is rather elegant and deep, even though
there's something dark to it.)

<P>(On first reading, you may want to skip directly to the examples in
the section "Writing Metaclasses in Python" below, unless you want
your head to explode.)

<P>

<HR>

<P>Documentation of the Don Beaudry hook has purposefully been kept
minimal, since it is a feature of incredible power, and is easily
abused.  Basically, it checks whether the <b>type of the base
class</b> is callable, and if so, it is called to create the new
class.

<P>Note the two indirection levels.  Take a simple example:

<PRE>
class B:
    pass

class C(B):
    pass
</PRE>

Take a look at the second class definition, and try to fathom ``the
type of the base class is callable.''

<P>(Types are not classes, by the way.  See questions 4.2, 4.19 and in
particular 6.22 in the <A
HREF="http://grail.cnri.reston.va.us/cgi-bin/faqw.py" >Python FAQ</A>
for more on this topic.)

<P>

<UL>

<LI>The <b>base class</b> is B; this one's easy.<P>

<LI>Since B is a class, its type is ``class''; so the <b>type of the
base class</b> is the type ``class''.  This is also known as
types.ClassType, assuming the standard module <code>types</code> has
been imported.<P>

<LI>Now is the type ``class'' <b>callable</b>?  No, because types (in
core Python) are never callable.  Classes are callable (calling a
class creates a new instance) but types aren't.<P>

</UL>

<P>So our conclusion is that in our example, the type of the base
class (of C) is not callable.  So the Don Beaudry hook does not apply,
and the default class creation mechanism is used (which is also used
when there is no base class).  In fact, the Don Beaudry hook never
applies when using only core Python, since the type of a core object
is never callable.

<P>So what do Don and Jim do in order to use Don's hook?  Write an
extension that defines at least two new Python object types.  The
first would be the type for ``class-like'' objects usable as a base
class, to trigger Don's hook.  This type must be made callable.
That's why we need a second type.  Whether an object is callable
depends on its type.  So whether a type object is callable depends on
<i>its</i> type, which is a <i>meta-type</i>.  (In core Python there
is only one meta-type, the type ``type'' (types.TypeType), which is
the type of all type objects, even itself.)  A new meta-type must
be defined that makes the type of the class-like objects callable.
(Normally, a third type would also be needed, the new ``instance''
type, but this is not an absolute requirement -- the new class type
could return an object of some existing type when invoked to create an
instance.)

<P>Still confused?  Here's a simple device due to Don himself to
explain metaclasses.  Take a simple class definition; assume B is a
special class that triggers Don's hook:

<PRE>
class C(B):
    a = 1
    b = 2
</PRE>

This can be though of as equivalent to:

<PRE>
C = type(B)('C', (B,), {'a': 1, 'b': 2})
</PRE>

If that's too dense for you, here's the same thing written out using
temporary variables:

<PRE>
creator = type(B)               # The type of the base class
name = 'C'                      # The name of the new class
bases = (B,)                    # A tuple containing the base class(es)
namespace = {'a': 1, 'b': 2}    # The namespace of the class statement
C = creator(name, bases, namespace)
</PRE>

This is analogous to what happens without the Don Beaudry hook, except
that in that case the creator function is set to the default class
creator.

<P>In either case, the creator is called with three arguments.  The
first one, <i>name</i>, is the name of the new class (as given at the
top of the class statement).  The <i>bases</i> argument is a tuple of
base classes (a singleton tuple if there's only one base class, like
the example).  Finally, <i>namespace</i> is a dictionary containing
the local variables collected during execution of the class statement.

<P>Note that the contents of the namespace dictionary is simply
whatever names were defined in the class statement.  A little-known
fact is that when Python executes a class statement, it enters a new
local namespace, and all assignments and function definitions take
place in this namespace.  Thus, after executing the following class
statement:

<PRE>
class C:
    a = 1
    def f(s): pass
</PRE>

the class namespace's contents would be {'a': 1, 'f': &lt;function f
...&gt;}.

<P>But enough already about writing Python metaclasses in C; read the
documentation of <A
HREF="http://maigret.cog.brown.edu/pyutil/">MESS</A> or <A
HREF="http://www.digicool.com/papers/ExtensionClass.html" >Extension
Classes</A> for more information.

<P>

<HR>

<H2>Writing Metaclasses in Python</H2>

<P>In Python 1.5, the requirement to write a C extension in order to
write metaclasses has been dropped (though you can still do
it, of course).  In addition to the check ``is the type of the base
class callable,'' there's a check ``does the base class have a
__class__ attribute.''  If so, it is assumed that the __class__
attribute refers to a class.

<P>Let's repeat our simple example from above:

<PRE>
class C(B):
    a = 1
    b = 2
</PRE>

Assuming B has a __class__ attribute, this translates into:

<PRE>
C = B.__class__('C', (B,), {'a': 1, 'b': 2})
</PRE>

This is exactly the same as before except that instead of type(B),
B.__class__ is invoked.  If you have read <A HREF=
"http://grail.cnri.reston.va.us/cgi-bin/faqw.py?req=show&file=faq06.022.htp"
>FAQ question 6.22</A> you will understand that while there is a big
technical difference between type(B) and B.__class__, they play the
same role at different abstraction levels.  And perhaps at some point
in the future they will really be the same thing (at which point you
would be able to derive subclasses from built-in types).

<P>At this point it may be worth mentioning that C.__class__ is the
same object as B.__class__, i.e., C's metaclass is the same as B's
metaclass.  In other words, subclassing an existing class creates a
new (meta)inststance of the base class's metaclass.

<P>Going back to the example, the class B.__class__ is instantiated,
passing its constructor the same three arguments that are passed to
the default class constructor or to an extension's metaclass:
<i>name</i>, <i>bases</i>, and <i>namespace</i>.

<P>It is easy to be confused by what exactly happens when using a<commit_msg>Fix some broken links.<commit_after><HTML>

<HEAD>
<TITLE>Metaclasses in Python 1.5</TITLE>
</HEAD>

<BODY BGCOLOR="FFFFFF">

<H1>Metaclasses in Python 1.5</H1>
<H2>(A.k.a. The Killer Joke :-)</H2>

<HR>

(<i>Postscript:</i> reading this essay is probably not the best way to
understand the metaclass hook described here.  See a <A
HREF="meta-vladimir.txt">message posted by <NAME></A>
which may give a gentler introduction to the matter.  You may also
want to search Deja News for messages with "metaclass" in the subject
posted to comp.lang.python in July and August 1998.)

<HR>

<P>In previous Python releases (and still in 1.5), there is something
called the ``Don Beaudry hook'', after its inventor and champion.
This allows C extensions to provide alternate class behavior, thereby
allowing the Python class syntax to be used to define other class-like
entities.  <NAME> has used this in his infamous <A
HREF="http://maigret.cog.brown.edu/pyutil/">MESS</A> package; Jim
Fulton has used it in his <A
HREF="http://www.digicool.com/releases/ExtensionClass/">Extension
Classes</A> package.  (It has also been referred to as the ``Don
Beaudry <i>hack</i>,'' but that's a misnomer.  There's nothing hackish
about it -- in fact, it is rather elegant and deep, even though
there's something dark to it.)

<P>(On first reading, you may want to skip directly to the examples in
the section "Writing Metaclasses in Python" below, unless you want
your head to explode.)

<P>

<HR>

<P>Documentation of the Don Beaudry hook has purposefully been kept
minimal, since it is a feature of incredible power, and is easily
abused.  Basically, it checks whether the <b>type of the base
class</b> is callable, and if so, it is called to create the new
class.

<P>Note the two indirection levels.  Take a simple example:

<PRE>
class B:
    pass

class C(B):
    pass
</PRE>

Take a look at the second class definition, and try to fathom ``the
type of the base class is callable.''

<P>(Types are not classes, by the way.  See questions 4.2, 4.19 and in
particular 6.22 in the <A
HREF="http://www.python.org/cgi-bin/faqw.py" >Python FAQ</A>
for more on this topic.)

<P>

<UL>

<LI>The <b>base class</b> is B; this one's easy.<P>

<LI>Since B is a class, its type is ``class''; so the <b>type of the
base class</b> is the type ``class''.  This is also known as
types.ClassType, assuming the standard module <code>types</code> has
been imported.<P>

<LI>Now is the type ``class'' <b>callable</b>?  No, because types (in
core Python) are never callable.  Classes are callable (calling a
class creates a new instance) but types aren't.<P>

</UL>

<P>So our conclusion is that in our example, the type of the base
class (of C) is not callable.  So the Don Beaudry hook does not apply,
and the default class creation mechanism is used (which is also used
when there is no base class).  In fact, the Don Beaudry hook never
applies when using only core Python, since the type of a core object
is never callable.

<P>So what do Don and Jim do in order to use Don's hook?  Write an
extension that defines at least two new Python object types.  The
first would be the type for ``class-like'' objects usable as a base
class, to trigger Don's hook.  This type must be made callable.
That's why we need a second type.  Whether an object is callable
depends on its type.  So whether a type object is callable depends on
<i>its</i> type, which is a <i>meta-type</i>.  (In core Python there
is only one meta-type, the type ``type'' (types.TypeType), which is
the type of all type objects, even itself.)  A new meta-type must
be defined that makes the type of the class-like objects callable.
(Normally, a third type would also be needed, the new ``instance''
type, but this is not an absolute requirement -- the new class type
could return an object of some existing type when invoked to create an
instance.)

<P>Still confused?  Here's a simple device due to Don himself to
explain metaclasses.  Take a simple class definition; assume B is a
special class that triggers Don's hook:

<PRE>
class C(B):
    a = 1
    b = 2
</PRE>

This can be though of as equivalent to:

<PRE>
C = type(B)('C', (B,), {'a': 1, 'b': 2})
</PRE>

If that's too dense for you, here's the same thing written out using
temporary variables:

<PRE>
creator = type(B)               # The type of the base class
name = 'C'                      # The name of the new class
bases = (B,)                    # A tuple containing the base class(es)
namespace = {'a': 1, 'b': 2}    # The namespace of the class statement
C = creator(name, bases, namespace)
</PRE>

This is analogous to what happens without the Don Beaudry hook, except
that in that case the creator function is set to the default class
creator.

<P>In either case, the creator is called with three arguments.  The
first one, <i>name</i>, is the name of the new class (as given at the
top of the class statement).  The <i>bases</i> argument is a tuple of
base classes (a singleton tuple if there's only one base class, like
the example).  Finally, <i>namespace</i> is a dictionary containing
the local variables collected during execution of the class statement.

<P>Note that the contents of the namespace dictionary is simply
whatever names were defined in the class statement.  A little-known
fact is that when Python executes a class statement, it enters a new
local namespace, and all assignments and function definitions take
place in this namespace.  Thus, after executing the following class
statement:

<PRE>
class C:
    a = 1
    def f(s): pass
</PRE>

the class namespace's contents would be {'a': 1, 'f': &lt;function f
...&gt;}.

<P>But enough already about writing Python metaclasses in C; read the
documentation of <A
HREF="http://maigret.cog.brown.edu/pyutil/">MESS</A> or <A
HREF="http://www.digicool.com/papers/ExtensionClass.html" >Extension
Classes</A> for more information.

<P>

<HR>

<H2>Writing Metaclasses in Python</H2>

<P>In Python 1.5, the requirement to write a C extension in order to
write metaclasses has been dropped (though you can still do
it, of course).  In addition to the check ``is the type of the base
class callable,'' there's a check ``does the base class have a
__class__ attribute.''  If so, it is assumed that the __class__
attribute refers to a class.

<P>Let's repeat our simple example from above:

<PRE>
class C(B):
    a = 1
    b = 2
</PRE>

Assuming B has a __class__ attribute, this translates into:

<PRE>
C = B.__class__('C', (B,), {'a': 1, 'b': 2})
</PRE>

This is exactly the same as before except that instead of type(B),
B.__class__ is invoked.  If you have read <A HREF=
"http://www.python.org/cgi-bin/faqw.py?req=show&file=faq06.022.htp"
>FAQ question 6.22</A> you will understand that while there is a big
technical difference between type(B) and B.__class__, they play the
same role at different abstraction levels.  And perhaps at some point
in the future they will really be the same thing (at which point you
would be able to derive subclasses from built-in types).

<P>At this point it may be worth mentioning that C.__class__ is the
same object as B.__class__, i.e., C's metaclass is the same as B's
metaclass.  In other words, subclassing an existing class creates a
new (meta)inststance of the base class's metaclass.

<P>Going back to the example, the class B.__class__ is instantiated,
passing its constructor the same three arguments that are passed to
the default class constructor or to an extension's metaclass:
<i>name</i>, <i>bases</i>, and <i>namespace</i>.

<P>It is easy to be confused by what exactly happens when using a
{'fraction_non_alphanumeric': 0.0922824302134647, 'fraction_numerical': 0.0060426929392446635, 'mean_word_length': 3.948326291842704, 'pattern_counts': {'":': 0, '<': 209, '<?xml version=': 0, '>': 209, 'https://': 0, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4009443', 'n_tokens_mistral': 4814, 'n_tokens_neox': 4589, 'n_words': 2352}