<commit_before>===================================================
django-content-editor -- Editing structured content
===================================================

.. image:: https://travis-ci.org/matthiask/django-content-editor.svg?branch=master
    :target: https://travis-ci.org/matthiask/django-content-editor

.. warning::

   Everything is really bare bones here, but works alright. The abstract
   plugin base model and the administration interface are practically
   finished, but the parts pertaining to views and templates need some
   additional thinking. Also, docs and more tests.

**Tagline: The component formerly known as FeinCMS' ItemEditor.**

The item editor was primarily a system to work with lists of content blocks
which can be assigned to arbitrary other objects. Whether those are elements of
a hierarchical page structure, weblog articles or anything else is irrelevant.
The idea of putting content together in small manageable pieces is interesting
for various use cases.


Why ``django-content-editor``?
==============================

``django-content-editor`` does what FeinCMS_ should have been all along. A
really thin layer around ``django.contrib.admin``'s inlines without any magical
behavior, with proper separation between models, rendering and administration
as Django did with the `newforms admin`_ a long long time ago.

Also, there is **absolutely no magic** going on behind the scenes, no
dynamic model generation or anything similar. The FeinCMS code's dynamic
behavior required workarounds such as ``feincms_item_editor_inline`` for
simple use cases such as specifying ``InlineModelAdmin`` options.
Furthermore, my long gone fondness for monkey patching made the code even
more prone to breakage.

With the next version of django-mptt_ supporting a draggable tree admin
(formerly FeinCMS' TreeEditor) it made lots of sense extracting the content
editor of FeinCMS into its own package, and thereby paving the way for a more
modular Django-based CMF.

Content first! The interface matters, but data matters more.

---

FeinCMS_ actually was trying to **be** a really simple and stupid content
management system. Simply looking at the amount of code in FeinCMS_ is enough
to decide that we did worse than we should have done. While FeinCMS_ still
contains less code than `comparable CMS systems`_ it still consists of
hundreds of lines of comparatively complicated and needlessly dynamic code.
Reasons are mostly historical of course -- Django_'s administration interface
did it fact not even have dynamic JavaScript-backed inlines when FeinCMS_ was
first released, and the necessary events which we require for adding custom
JavaScript-backed widgets (such as rich text editors) to plugins are only
available starting with Django_ 1.9 -- the latest release at the time of
writing.

---


.. _Django: https://www.djangoproject.com/
.. _FeinCMS: https://github.com/feincms/feincms/
.. _newforms admin: https://code.djangoproject.com/wiki/NewformsAdminBranch
.. _django-mptt: https://github.com/django-mptt/django-mptt/
.. _comparable CMS systems: https://www.djangopackages.com/grids/g/cms/


Example: articles with rich text plugins
========================================

``app/models.py``::

    from django.db import models

    from content_editor.models import Template, Region, create_plugin_base


    class Article(models.Model):
        title = models.CharField(max_length=200)
        pub_date = models.DateField(blank=True, null=True)

        regions = [
            Region(key='main', title='main region'),
            # Region(key='sidebar', title='sidebar region', inherited=False),
        ]

        def __str__(self):
            return self.title


    # create_plugin_base does nothing outlandish, it only defines an abstract
    # base model with the following attributes:
    # - a parent ForeignKey with a related_name the rest of the code expects
    # - a region CharField containing the region key defined above
    # - an ordering IntegerField for ordering plugin items
    # - a get_queryset() classmethod returning a queryset for the
    #   ContentProxy classes (a good place to add select_related and
    #   prefetch_related calls or anything similar)
    # That's all. Really!
    ArticlePlugin = create_plugin_base(Article)


    class RichText(ArticlePlugin):
        text = models.TextField(blank=True)

        class Meta:
            verbose_name = 'rich text'
            verbose_name_plural = 'rich texts'


    class Download(ArticlePlugin):
        file = models.FileField(upload_to='downloads/%Y/%m/')

        class Meta:
            verbose_name = 'download'
            verbose_name_plural = 'downloads'


``app/admin.py``::

    from django import forms
    from django.contrib import admin
    from django.db import models

    from content_editor.admin import ContentEditor, ContentEditorInline

    from .models import Article, Richtext, Download


    class RichTextarea(forms.Textarea):
        def __init__(self, attrs=None):
            default_attrs = {'class': 'richtext'}
            if attrs:
                default_attrs.update(attrs)
            super(RichTextarea, self).__init__(default_attrs)


    class RichTextInline(ContentEditorInline):
        model = RichText
        formfield_overrides = {
            models.TextField: {'widget': RichTextarea},
        }

        class Media:
            js = (
                '//cdn.ckeditor.com/4.5.6/standard/ckeditor.js',
                'app/plugin_ckeditor.js',
            )

    admin.site.register(
        Article,
        ContentEditor,
        inlines=[
            RichTextInline,
            ContentEditorInline.create(model=Download),
        ],
    )


``app/static/app/plugin_ckeditor.js``::

    /* global django, CKEDITOR */
    (function($) {

        /* Improve spacing */
        var style = document.createElement('style');
        style.type = 'text/css';
        style.innerHTML = "div[id*='cke_id_'] { margin-left: 170px; }";
        $('head').append(style);

        // Activate and deactivate the CKEDITOR because it does not like
        // getting dragged or its underlying ID changed

        CKEDITOR.config.width = '787';
        CKEDITOR.config.height= '300';
        CKEDITOR.config.format_tags = 'p;h1;h2;h3;h4;pre';
        CKEDITOR.config.toolbar = [[
            'Maximize','-',
            'Format','-',
            'Bold','Italic','Underline','Strike','-',
            'Subscript','Superscript','-',
            'NumberedList','BulletedList','-',
            'Anchor','Link','Unlink','-',
            'Source'
        ]];

        $(document).on(
            'content-editor:activate',
            function(event, $row, formsetName) {
                $row.find('textarea.richtext').each(function() {
                    CKEDITOR.replace(this.id, CKEDITOR.config);
                });
            }
        ).on(
            'content-editor:deactivate',
            function(event, $row, formsetName) {
                $row.find('textarea.richtext').each(function() {
                    CKEDITOR.instances[this.id] && CKEDITOR.instances[this.id].destroy();
                });
            }
        );
    })(django.jQuery);


``app/views.py``::

    from django.views import generic

    from content_editor.models import ContentProxy

    from .models import Article, RichText, Download


    class ArticleView(generic.DetailView):
        model = Article

        def get_context_data(self, **kwargs):
            return super(ArticleView, self).get_context_data(
                content=ContentProxy(
                    self.object,
                    plugins=[RichText, Download],
                ),
                **kwargs)


``app/templates/app/article_detail.html``::

    {% extends "base.html" %}

    {% block title %}{{ article }} - {{ block.super }}{% endblock %}

    {% block content %}
    <h1>{{ article }}</h1>
    {{ article.pub_date }}

    {# Yes, not generic at all. And also does not render downloads. #}
    {% for plugin in content.main %}{{ plugin.text|safe }}{% endfor %}
    {% endblock %}

Finally, ensure that ``content_editor`` and ``app`` are added to your
``INSTALLED_APPS`` setting, and you're good to go.

IF you also want nice icons to add new items, you might want to use
`font awesome`_ and the following snippets:

``app/admin.py``::

    class ArticleAdmin(ContentEditor):
        inlines = [
            RichTextInline,
            ContentEditorInline.create(model=Download),
        ]

        class Media:
            css = {'all': (
                'https://maxcdn.bootstrapcdn.com/font-awesome/4.5.0/css/font-awesome.min.css',  # noqa
            )}
            js = (
                'app/plugin_buttons.js',
            )


``app/plugin_buttons.js``::

    (function($) {
        $(document).on('content-editor:ready', function() {
            ContentEditor.addPluginButton(
                'app_richtext',
                '<i class="fa fa-pencil"></i>'
            );
            ContentEditor.addPluginButton(
                'app_download',
                '<i class="fa fa-download"></i>'
            );
        });
    })(django.jQuery);


.. _font awesome: https://fortawesome.github.io/Font-Awesome/


Conventions
===========

Regions
~~~~~~~

The included ``ContentProxy`` classes and the ``ContentEditor`` admin class
expect a ``regions`` attribute or property (**not** a method) on their model
(the ``Article`` model above) which returns a list of ``Region`` instances.

Regions have the following attributes:

* ``title``: Something nice, will be visible in the content editor.
* ``key``: The region key, used in the content proxy as attribute name for
  the list of plugins. Must contain a valid Python identifier.
* ``inherited``: Only has an effect if you are using the bundled
  ``MPTTContentProxy``: Models inherit content from their ancestor chain if a
  region with ``inherited = True`` is emtpy.

You are free to define additional attributes -- simply pass them when
instantiating a new region.


Templates
~~~~~~~~~

Various classes will expect the main model to have a ``template`` attribute or
property which returns a ``Template`` instance. Nothing of the sort is
implemented yet.

Templates have the following attributes:

* ``title``: Something nice.
* ``key``: The template key. Must contain a valid Python identifier.
* ``template_name``: A template path.
* ``regions``: A list of region instances.

As with the regions above, you are free to define additional attributes.


Design decisions
==============================

About rich text editors
~~~~~~~~~~~~~~~~~~~~~~~

We have been struggling with rich text editors for a long time. To be honest, I
do not think it was a good idea to add that many features to the rich text
editor. Resizing images uploaded into a rich text editor is a real pain, and
what if you'd like to reuse these images or display them using a lightbox
script or something similar? You have to resort to writing loads of JavaScript
code which will only work on one browser. You cannot really filter the HTML
code generated by the user to kick out ugly HTML code generated by copy-pasting
from word. The user will upload 10mb JPEGs and resize them to 50x50 pixels in
the rich text editor.

All of this convinced me that offering the user a rich text editor with too
much capabilities is a really bad idea. The rich text editor in FeinCMS only
has bold, italic, bullets, link and headlines activated (and the HTML code
button, because that's sort of inevitable -- sometimes the rich text editor
messes up and you cannot fix it other than going directly into the HTML code.
Plus, if someone really knows what they are doing, I'd still like to give them
the power to shot their own foot).

If this does not seem convincing you can always add your own rich text content
type with a different configuration (or just override the rich text editor
initialization template in your own project). We do not want to force our world
view on you, it's just that we think that in this case, more choice has the
bigger potential to hurt than to help.


Content blocks
~~~~~~~~~~~~~~

Images and other media files are inserted via objects; the user can only select
a file and a display mode (f.e. float/block for images or something...). An
article's content could look like this:

* Rich Text
* Floated image
* Rich Text
* YouTube Video Link, embedding code is automatically generated from the link
* Rich Text

It's of course easier for the user to start with only a single rich text field,
but I think that the user already has too much confusing possibilities with an
enhanced rich text editor. Once the user grasps the concept of content blocks
which can be freely added, removed and reordered using drag/drop, I'd say it's
much easier to administer the content of a webpage. Plus, the content blocks
can have their own displaying and updating logic; implementing dynamic content
inside the CMS is not hard anymore, on the contrary. Since content blocks are
Django models, you can do anything you want inside them.
<commit_msg>Update the docs with the new rendering code<commit_after>===================================================
django-content-editor -- Editing structured content
===================================================

.. image:: https://travis-ci.org/matthiask/django-content-editor.svg?branch=master
    :target: https://travis-ci.org/matthiask/django-content-editor

.. warning::

   Everything is really bare bones here, but works alright. The abstract
   plugin base model and the administration interface are practically
   finished, but the parts pertaining to views and templates need some
   additional thinking. Also, docs and more tests.

**Tagline: The component formerly known as FeinCMS' ItemEditor.**

The item editor was primarily a system to work with lists of content blocks
which can be assigned to arbitrary other objects. Whether those are elements of
a hierarchical page structure, weblog articles or anything else is irrelevant.
The idea of putting content together in small manageable pieces is interesting
for various use cases.


Why ``django-content-editor``?
==============================

``django-content-editor`` does what FeinCMS_ should have been all along. A
really thin layer around ``django.contrib.admin``'s inlines without any magical
behavior, with proper separation between models, rendering and administration
as Django did with the `newforms admin`_ a long long time ago.

Also, there is **absolutely no magic** going on behind the scenes, no
dynamic model generation or anything similar. The FeinCMS code's dynamic
behavior required workarounds such as ``feincms_item_editor_inline`` for
simple use cases such as specifying ``InlineModelAdmin`` options.
Furthermore, my long gone fondness for monkey patching made the code even
more prone to breakage.

With the next version of django-mptt_ supporting a draggable tree admin
(formerly FeinCMS' TreeEditor) it made lots of sense extracting the content
editor of FeinCMS into its own package, and thereby paving the way for a more
modular Django-based CMF.

Content first! The interface matters, but data matters more.

---

FeinCMS_ actually was trying to **be** a really simple and stupid content
management system. Simply looking at the amount of code in FeinCMS_ is enough
to decide that we did worse than we should have done. While FeinCMS_ still
contains less code than `comparable CMS systems`_ it still consists of
hundreds of lines of comparatively complicated and needlessly dynamic code.
Reasons are mostly historical of course -- Django_'s administration interface
did it fact not even have dynamic JavaScript-backed inlines when FeinCMS_ was
first released, and the necessary events which we require for adding custom
JavaScript-backed widgets (such as rich text editors) to plugins are only
available starting with Django_ 1.9 -- the latest release at the time of
writing.

---


.. _Django: https://www.djangoproject.com/
.. _FeinCMS: https://github.com/feincms/feincms/
.. _newforms admin: https://code.djangoproject.com/wiki/NewformsAdminBranch
.. _django-mptt: https://github.com/django-mptt/django-mptt/
.. _comparable CMS systems: https://www.djangopackages.com/grids/g/cms/


Example: articles with rich text plugins
========================================

``app/models.py``::

    from django.db import models

    from content_editor.models import Template, Region, create_plugin_base


    class Article(models.Model):
        title = models.CharField(max_length=200)
        pub_date = models.DateField(blank=True, null=True)

        regions = [
            Region(key='main', title='main region'),
            # Region(key='sidebar', title='sidebar region', inherited=False),
        ]

        def __str__(self):
            return self.title


    # create_plugin_base does nothing outlandish, it only defines an abstract
    # base model with the following attributes:
    # - a parent ForeignKey with a related_name the rest of the code expects
    # - a region CharField containing the region key defined above
    # - an ordering IntegerField for ordering plugin items
    # - a get_queryset() classmethod returning a queryset for the
    #   ContentProxy classes (a good place to add select_related and
    #   prefetch_related calls or anything similar)
    # That's all. Really!
    ArticlePlugin = create_plugin_base(Article)


    class RichText(ArticlePlugin):
        text = models.TextField(blank=True)

        class Meta:
            verbose_name = 'rich text'
            verbose_name_plural = 'rich texts'


    class Download(ArticlePlugin):
        file = models.FileField(upload_to='downloads/%Y/%m/')

        class Meta:
            verbose_name = 'download'
            verbose_name_plural = 'downloads'


``app/admin.py``::

    from django import forms
    from django.contrib import admin
    from django.db import models

    from content_editor.admin import ContentEditor, ContentEditorInline

    from .models import Article, Richtext, Download


    class RichTextarea(forms.Textarea):
        def __init__(self, attrs=None):
            default_attrs = {'class': 'richtext'}
            if attrs:
                default_attrs.update(attrs)
            super(RichTextarea, self).__init__(default_attrs)


    class RichTextInline(ContentEditorInline):
        model = RichText
        formfield_overrides = {
            models.TextField: {'widget': RichTextarea},
        }

        class Media:
            js = (
                '//cdn.ckeditor.com/4.5.6/standard/ckeditor.js',
                'app/plugin_ckeditor.js',
            )

    admin.site.register(
        Article,
        ContentEditor,
        inlines=[
            RichTextInline,
            ContentEditorInline.create(model=Download),
        ],
    )


``app/static/app/plugin_ckeditor.js``::

    /* global django, CKEDITOR */
    (function($) {

        /* Improve spacing */
        var style = document.createElement('style');
        style.type = 'text/css';
        style.innerHTML = "div[id*='cke_id_'] { margin-left: 170px; }";
        $('head').append(style);

        // Activate and deactivate the CKEDITOR because it does not like
        // getting dragged or its underlying ID changed

        CKEDITOR.config.width = '787';
        CKEDITOR.config.height= '300';
        CKEDITOR.config.format_tags = 'p;h1;h2;h3;h4;pre';
        CKEDITOR.config.toolbar = [[
            'Maximize','-',
            'Format','-',
            'Bold','Italic','Underline','Strike','-',
            'Subscript','Superscript','-',
            'NumberedList','BulletedList','-',
            'Anchor','Link','Unlink','-',
            'Source'
        ]];

        $(document).on(
            'content-editor:activate',
            function(event, $row, formsetName) {
                $row.find('textarea.richtext').each(function() {
                    CKEDITOR.replace(this.id, CKEDITOR.config);
                });
            }
        ).on(
            'content-editor:deactivate',
            function(event, $row, formsetName) {
                $row.find('textarea.richtext').each(function() {
                    CKEDITOR.instances[this.id] && CKEDITOR.instances[this.id].destroy();
                });
            }
        );
    })(django.jQuery);


``app/views.py``::

    from django.utils.html import format_html, mark_safe
    from django.views import generic

    from content_editor.renderer import PluginRenderer
    from content_editor.utils import collect_contents_for_mptt_item

    from .models import Article, RichText, Download


    class ArticleView(generic.DetailView):
        model = Article

        def get_context_data(self, **kwargs):
            renderer = PluginRenderer()
            renderer.register(
                RichText,
                lambda plugin: mark_safe(plugin.text),
            )
            renderer.register(
                Download,
                lambda plugin: format_html(
                    '<a href="{}">{}</a>',
                    plugin.file.url,
                    plugin.file.name,
                ),
            )
            contents = collect_contents_for_mptt_item(
                self.object,
                [RichText, Download])

            return super(ArticleView, self).get_context_data(
                content={
                    region.key: renderer.render(contents[region.key])
                    for region in self.objects.regions
                },
                **kwargs)


``app/templates/app/article_detail.html``::

    {% extends "base.html" %}

    {% block title %}{{ article }} - {{ block.super }}{% endblock %}

    {% block content %}
    <article>
        <h1>{{ article }}</h1>
        {{ article.pub_date }}

        {{ content.main }}
    </article>
    {% comment %}
        <aside>{{ content.sidebar }}</aside>
    {% endcomment %}
    {% endblock %}

Finally, ensure that ``content_editor`` and ``app`` are added to your
``INSTALLED_APPS`` setting, and you're good to go.

IF you also want nice icons to add new items, you might want to use
`font awesome`_ and the following snippets:

``app/admin.py``::

    class ArticleAdmin(ContentEditor):
        inlines = [
            RichTextInline,
            ContentEditorInline.create(model=Download),
        ]

        class Media:
            css = {'all': (
                'https://maxcdn.bootstrapcdn.com/font-awesome/4.5.0/css/font-awesome.min.css',  # noqa
            )}
            js = (
                'app/plugin_buttons.js',
            )


``app/plugin_buttons.js``::

    (function($) {
        $(document).on('content-editor:ready', function() {
            ContentEditor.addPluginButton(
                'app_richtext',
                '<i class="fa fa-pencil"></i>'
            );
            ContentEditor.addPluginButton(
                'app_download',
                '<i class="fa fa-download"></i>'
            );
        });
    })(django.jQuery);


.. _font awesome: https://fortawesome.github.io/Font-Awesome/


Parts
=====

Regions
~~~~~~~

The included ``Contents`` class and its helpers (``collect_contents_*``) and
the ``ContentEditor`` admin class expect a ``regions`` attribute or property
(**not** a method) on their model (the ``Article`` model above) which returns
a list of ``Region`` instances.

Regions have the following attributes:

* ``title``: Something nice, will be visible in the content editor.
* ``key``: The region key, used in the content proxy as attribute name for
  the list of plugins. Must contain a valid Python identifier.
* ``inherited``: Only has an effect if you are using the bundled
  ``collect_contents_for_mptt_item`` or anything comparable: Models inherit
  content from their ancestor chain if a region with ``inherited = True`` is
  emtpy.

You are free to define additional attributes -- simply pass them when
instantiating a new region.


Templates
~~~~~~~~~

Various classes will expect the main model to have a ``template`` attribute or
property which returns a ``Template`` instance. Nothing of the sort is
implemented yet.

Templates have the following attributes:

* ``title``: Something nice.
* ``key``: The template key. Must contain a valid Python identifier.
* ``template_name``: A template path.
* ``regions``: A list of region instances.

As with the regions above, you are free to define additional attributes.


``Contents`` class and helpers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

TODO

``collect_contents_for_items``
------------------------------

``collect_contents_for_item``
-----------------------------

``collect_contents_for_mptt_item``
----------------------------------


``PluginRenderer`` class
~~~~~~~~~~~~~~~~~~~~~~~~

TODO


Design decisions
==============================

About rich text editors
~~~~~~~~~~~~~~~~~~~~~~~

We have been struggling with rich text editors for a long time. To be honest, I
do not think it was a good idea to add that many features to the rich text
editor. Resizing images uploaded into a rich text editor is a real pain, and
what if you'd like to reuse these images or display them using a lightbox
script or something similar? You have to resort to writing loads of JavaScript
code which will only work on one browser. You cannot really filter the HTML
code generated by the user to kick out ugly HTML code generated by copy-pasting
from word. The user will upload 10mb JPEGs and resize them to 50x50 pixels in
the rich text editor.

All of this convinced me that offering the user a rich text editor with too
much capabilities is a really bad idea. The rich text editor in FeinCMS only
has bold, italic, bullets, link and headlines activated (and the HTML code
button, because that's sort of inevitable -- sometimes the rich text editor
messes up and you cannot fix it other than going directly into the HTML code.
Plus, if someone really knows what they are doing, I'd still like to give them
the power to shot their own foot).

If this does not seem convincing you can always add your own rich text content
type with a different configuration (or just override the rich text editor
initialization template in your own project). We do not want to force our world
view on you, it's just that we think that in this case, more choice has the
bigger potential to hurt than to help.


Content blocks
~~~~~~~~~~~~~~

Images and other media files are inserted via objects; the user can only select
a file and a display mode (f.e. float/block for images or something...). An
article's content could look like this:

* Rich Text
* Floated image
* Rich Text
* YouTube Video Link, embedding code is automatically generated from the link
* Rich Text

It's of course easier for the user to start with only a single rich text field,
but I think that the user already has too much confusing possibilities with an
enhanced rich text editor. Once the user grasps the concept of content blocks
which can be freely added, removed and reordered using drag/drop, I'd say it's
much easier to administer the content of a webpage. Plus, the content blocks
can have their own displaying and updating logic; implementing dynamic content
inside the CMS is not hard anymore, on the contrary. Since content blocks are
Django models, you can do anything you want inside them.

{'fraction_non_alphanumeric': 0.10927963159241255, 'fraction_numerical': 0.002339095793282409, 'mean_word_length': 2.935279735366029, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 18, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 24, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2170521', 'n_tokens_mistral': 7382, 'n_tokens_neox': 6918, 'n_words': 2821}