<commit_before>.. highlightlang:: c


.. _defining-new-types:

******************
Defining New Types
******************

.. sectionauthor:: <NAME> <<EMAIL>>
.. sectionauthor:: <NAME> <<EMAIL>>
.. sectionauthor:: <NAME> <<EMAIL>>


As mentioned in the last chapter, Python allows the writer of an extension
module to define new types that can be manipulated from Python code, much like
strings and lists in core Python.

This is not hard; the code for all extension types follows a pattern, but there
are some details that you need to understand before you can get started.

.. note::

   The way new types are defined changed dramatically (and for the better) in
   Python 2.2.  This document documents how to define new types for Python 2.2 and
   later.  If you need to support older versions of Python, you will need to refer
   to `older versions of this documentation
   <http://www.python.org/doc/versions/>`_.


.. _dnt-basics:

The Basics
==========

The Python runtime sees all Python objects as variables of type
:ctype:`PyObject\*`.  A :ctype:`PyObject` is not a very magnificent object - it
just contains the refcount and a pointer to the object's "type object".  This is
where the action is; the type object determines which (C) functions get called
when, for instance, an attribute gets looked up on an object or it is multiplied
by another object.  These C functions are called "type methods" to distinguish
them from things like ``[].append`` (which we call "object methods").

So, if you want to define a new object type, you need to create a new type
object.

This sort of thing can only be explained by example, so here's a minimal, but
complete, module that defines a new type:

.. literalinclude:: ../includes/noddy.c


Now that's quite a bit to take in at once, but hopefully bits will seem familiar
from the last chapter.

The first bit that will be new is::

   typedef struct {
       PyObject_HEAD
   } noddy_NoddyObject;

This is what a Noddy object will contain---in this case, nothing more than every
Python object contains, namely a refcount and a pointer to a type object.  These
are the fields the ``PyObject_HEAD`` macro brings in.  The reason for the macro
is to standardize the layout and to enable special debugging fields in debug
builds.  Note that there is no semicolon after the ``PyObject_HEAD`` macro; one
is included in the macro definition.  Be wary of adding one by accident; it's
easy to do from habit, and your compiler might not complain, but someone else's
probably will!  (On Windows, MSVC is known to call this an error and refuse to
compile the code.)

For contrast, let's take a look at the corresponding definition for standard
Python integers::

   typedef struct {
       PyObject_HEAD
       long ob_ival;
   } PyIntObject;

Moving on, we come to the crunch --- the type object. ::

   static PyTypeObject noddy_NoddyType = {
       PyObject_HEAD_INIT(NULL)
       0,                         /*ob_size*/
       "noddy.Noddy",             /*tp_name*/
       sizeof(noddy_NoddyObject), /*tp_basicsize*/
       0,                         /*tp_itemsize*/
       0,                         /*tp_dealloc*/
       0,                         /*tp_print*/
       0,                         /*tp_getattr*/
       0,                         /*tp_setattr*/
       0,                         /*tp_compare*/
       0,                         /*tp_repr*/
       0,                         /*tp_as_number*/
       0,                         /*tp_as_sequence*/
       0,                         /*tp_as_mapping*/
       0,                         /*tp_hash */
       0,                         /*tp_call*/
       0,                         /*tp_str*/
       0,                         /*tp_getattro*/
       0,                         /*tp_setattro*/
       0,                         /*tp_as_buffer*/
       Py_TPFLAGS_DEFAULT,        /*tp_flags*/
       "Noddy objects",           /* tp_doc */
   };

Now if you go and look up the definition of :ctype:`PyTypeObject` in
:file:`object.h` you'll see that it has many more fields that the definition<commit_msg>Fix one more PyInt occurrence.<commit_after>.. highlightlang:: c


.. _defining-new-types:

******************
Defining New Types
******************

.. sectionauthor:: <NAME> <<EMAIL>>
.. sectionauthor:: <NAME> <<EMAIL>>
.. sectionauthor:: <NAME> <<EMAIL>>


As mentioned in the last chapter, Python allows the writer of an extension
module to define new types that can be manipulated from Python code, much like
strings and lists in core Python.

This is not hard; the code for all extension types follows a pattern, but there
are some details that you need to understand before you can get started.

.. note::

   The way new types are defined changed dramatically (and for the better) in
   Python 2.2.  This document documents how to define new types for Python 2.2 and
   later.  If you need to support older versions of Python, you will need to refer
   to `older versions of this documentation
   <http://www.python.org/doc/versions/>`_.


.. _dnt-basics:

The Basics
==========

The Python runtime sees all Python objects as variables of type
:ctype:`PyObject\*`.  A :ctype:`PyObject` is not a very magnificent object - it
just contains the refcount and a pointer to the object's "type object".  This is
where the action is; the type object determines which (C) functions get called
when, for instance, an attribute gets looked up on an object or it is multiplied
by another object.  These C functions are called "type methods" to distinguish
them from things like ``[].append`` (which we call "object methods").

So, if you want to define a new object type, you need to create a new type
object.

This sort of thing can only be explained by example, so here's a minimal, but
complete, module that defines a new type:

.. literalinclude:: ../includes/noddy.c


Now that's quite a bit to take in at once, but hopefully bits will seem familiar
from the last chapter.

The first bit that will be new is::

   typedef struct {
       PyObject_HEAD
   } noddy_NoddyObject;

This is what a Noddy object will contain---in this case, nothing more than every
Python object contains, namely a refcount and a pointer to a type object.  These
are the fields the ``PyObject_HEAD`` macro brings in.  The reason for the macro
is to standardize the layout and to enable special debugging fields in debug
builds.  Note that there is no semicolon after the ``PyObject_HEAD`` macro; one
is included in the macro definition.  Be wary of adding one by accident; it's
easy to do from habit, and your compiler might not complain, but someone else's
probably will!  (On Windows, MSVC is known to call this an error and refuse to
compile the code.)

For contrast, let's take a look at the corresponding definition for standard
Python floats::

   typedef struct {
       PyObject_HEAD
       double ob_fval;
   } PyFloatObject;

Moving on, we come to the crunch --- the type object. ::

   static PyTypeObject noddy_NoddyType = {
       PyObject_HEAD_INIT(NULL)
       0,                         /*ob_size*/
       "noddy.Noddy",             /*tp_name*/
       sizeof(noddy_NoddyObject), /*tp_basicsize*/
       0,                         /*tp_itemsize*/
       0,                         /*tp_dealloc*/
       0,                         /*tp_print*/
       0,                         /*tp_getattr*/
       0,                         /*tp_setattr*/
       0,                         /*tp_compare*/
       0,                         /*tp_repr*/
       0,                         /*tp_as_number*/
       0,                         /*tp_as_sequence*/
       0,                         /*tp_as_mapping*/
       0,                         /*tp_hash */
       0,                         /*tp_call*/
       0,                         /*tp_str*/
       0,                         /*tp_getattro*/
       0,                         /*tp_setattro*/
       0,                         /*tp_as_buffer*/
       Py_TPFLAGS_DEFAULT,        /*tp_flags*/
       "Noddy objects",           /* tp_doc */
   };

Now if you go and look up the definition of :ctype:`PyTypeObject` in
:file:`object.h` you'll see that it has many more fields that the definition
{'fraction_non_alphanumeric': 0.08626353570994039, 'fraction_numerical': 0.00511011071906558, 'mean_word_length': 2.135011441647597, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 34, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5738748', 'n_tokens_mistral': 2196, 'n_tokens_neox': 2110, 'n_words': 1007}