<commit_before># Spinneret

In the crowded space of Common Lisp HTML generators, SPINNERET
occupies the following coordinates:

- Modern. Targets HTML5. Does not treat XML and HTML as the same
  problem. Assumes you will be serving your documents as UTF-8.

- Composable. Makes it easy to refactor HTML generation into separate
  functions and macros.

- Pretty. Treats HTML as a document format, not a serialization.
  Output is idiomatic and readable, following the coding style of the
  HTML5 specification.

- Aggressive. If something can be interpreted as HTML, then it will
  be, meaning that some Lisp forms can't be mixed with HTML syntax. In
  the trade-off between 90% convenience and 10% correctness SPINNERET
  is on the side of convenience.

- Bilingual. Spinneret has the same semantics in Lisp and Parenscript.

HTML generation with SPINNERET looks like this:

     (in-package #:spinneret)

     (defparameter *shopping-list*
       '("Atmospheric ponds"
         "Electric gumption socks"
         "Mrs. Leland's embyronic television combustion"
         "Savage gymnatic aggressors"
         "Pharmaceutical pianos"
         "Intravenous retribution champions"))

     (defparameter *user-name* "<NAME>")

     (defparameter *last-login* "12th Never")

     (defmacro with-page ((&key title) &body body)
       `(with-html
          (:doctype)
          (:html
            (:head
             (:title ,title))
            (:body ,@body))))

     (defun shopping-list ()
       (with-page (:title "Home page")
         (:header
          (:h1 "Home page"))
         (:section
          ("~A, here is *your* shopping list: " *user-name*)
          (:ol (dolist (item *shopping-list*)
                 (:li (1+ (random 10)) item))))
         (:footer ("Last login: ~A" *last-login*))))

Which produces:

     <!DOCTYPE html>
     <html lang=en>
      <head>
       <meta charset=UTF-8>
       <title>Home page</title>
      </head>
      <body>
       <header>
        <h1>Home page</h1>
       </header>
       <section>
        <NAME>, here is <em>your</em> shopping list:
        <ol>
         <li>10 Atmospheric ponds
         <li>6 Electric gumption socks
         <li>4 Mrs. Leland's embyronic television combustion
         <li>9 Savage gymnatic aggressors
         <li>6 Pharmaceutical pianos
         <li>9 Intravenous retribution champions
        </ol>
       </section>
       <footer>
        Last login: 12th Never
       </footer>
      </body>
     </html>

(Pretty-printing is pretty fast, but SPINNERET obeys `*print-pretty*`
should you want to turn it off.)

## Syntax

The rules for WITH-HTML are these:

- All generated forms write to `*html*`.

- A keyword in function position is interpreted as a tag name. If the
  name is not valid as a tag, it is ignored.

  Certain keywords are recognized as pseudo-tags and given special
  treatment:

  :RAW :DOCTYPE :!DOCTYPE :CDATA :!-- :COMMENT :HTML :HEAD

  The value of the LANG attribute of HTML is controlled by
  `*html-lang*`; the value of the meta charset attribute is controlled
  by `*html-charset*`.

  Constant classes and ids can be specified with a selector-like
  syntax. E.g.:

     (:div#wrapper (:div.section ...))
     ≡ (:div :id "wrapper" (:div :class "section" ...))

- Keyword-value pairs following a tag are interpreted as attributes.
  HTML syntax may not be used in attribute values. Attributes with nil
  values are omitted from the output. Boolean attributes with non-nil
  values are minimized.

  Duplicate attributes are handled like duplicate keyword arguments:
  all values are evaluated, but only the leftmost value is used. The
  exception is the class attribute: the class of a tag is the union of
  all its :CLASS arguments.

  The argument :DATASET introduces a list of :DATA-FOO arguments:

     (:p :dataset (:duck (dolomphious) :fish 'fizzgigious
                         :spoon "runcible"))
     ≡ (:p :data-duck (dolomphious) :data-fish 'fizzgigious
           :data-spoon "runcible")

  For flexibility, even at the cost of efficiency, the argument :ATTRS
  introduces a form to evaluate at run time for a plist of extra
  attributes and values.

- Forms after the attributes are treated as arguments. Each non-nil
  (primary) value returned by an argument to a tag is written to the
  stream by HTML, a generic function on which you can define your own
  methods. By default only literal arguments are printed. Literal
  arguments are strings, characters, numbers and symbols beside NIL.

WITH-HTML-STRING is like WITH-HTML, but intercepts the generated HTML
at run time and returns a string.
<commit_msg>Minor: not just class, but all tokenized attributes.<commit_after># Spinneret

In the crowded space of Common Lisp HTML generators, SPINNERET
occupies the following coordinates:

- Modern. Targets HTML5. Does not treat XML and HTML as the same
  problem. Assumes you will be serving your documents as UTF-8.

- Composable. Makes it easy to refactor HTML generation into separate
  functions and macros.

- Pretty. Treats HTML as a document format, not a serialization.
  Output is idiomatic and readable, following the coding style of the
  HTML5 specification.

- Aggressive. If something can be interpreted as HTML, then it will
  be, meaning that some Lisp forms can't be mixed with HTML syntax. In
  the trade-off between 90% convenience and 10% correctness SPINNERET
  is on the side of convenience.

- Bilingual. Spinneret has the same semantics in Lisp and Parenscript.

HTML generation with SPINNERET looks like this:

     (in-package #:spinneret)

     (defparameter *shopping-list*
       '("Atmospheric ponds"
         "Electric gumption socks"
         "Mrs. Leland's embyronic television combustion"
         "Savage gymnatic aggressors"
         "Pharmaceutical pianos"
         "Intravenous retribution champions"))

     (defparameter *user-name* "<NAME>")

     (defparameter *last-login* "12th Never")

     (defmacro with-page ((&key title) &body body)
       `(with-html
          (:doctype)
          (:html
            (:head
             (:title ,title))
            (:body ,@body))))

     (defun shopping-list ()
       (with-page (:title "Home page")
         (:header
          (:h1 "Home page"))
         (:section
          ("~A, here is *your* shopping list: " *user-name*)
          (:ol (dolist (item *shopping-list*)
                 (:li (1+ (random 10)) item))))
         (:footer ("Last login: ~A" *last-login*))))

Which produces:

     <!DOCTYPE html>
     <html lang=en>
      <head>
       <meta charset=UTF-8>
       <title>Home page</title>
      </head>
      <body>
       <header>
        <h1>Home page</h1>
       </header>
       <section>
        <NAME>, here is <em>your</em> shopping list:
        <ol>
         <li>10 Atmospheric ponds
         <li>6 Electric gumption socks
         <li>4 Mrs. Leland's embyronic television combustion
         <li>9 Savage gymnatic aggressors
         <li>6 Pharmaceutical pianos
         <li>9 Intravenous retribution champions
        </ol>
       </section>
       <footer>
        Last login: 12th Never
       </footer>
      </body>
     </html>

(Pretty-printing is pretty fast, but SPINNERET obeys `*print-pretty*`
should you want to turn it off.)

## Syntax

The rules for WITH-HTML are these:

- All generated forms write to `*html*`.

- A keyword in function position is interpreted as a tag name. If the
  name is not valid as a tag, it is ignored.

  Certain keywords are recognized as pseudo-tags and given special
  treatment:

  :RAW :DOCTYPE :!DOCTYPE :CDATA :!-- :COMMENT :HTML :HEAD

  The value of the LANG attribute of HTML is controlled by
  `*html-lang*`; the value of the meta charset attribute is controlled
  by `*html-charset*`.

  Constant classes and ids can be specified with a selector-like
  syntax. E.g.:

     (:div#wrapper (:div.section ...))
     ≡ (:div :id "wrapper" (:div :class "section" ...))

- Keyword-value pairs following a tag are interpreted as attributes.
  HTML syntax may not be used in attribute values. Attributes with nil
  values are omitted from the output. Boolean attributes with non-nil
  values are minimized.

  Duplicate attributes are handled like duplicate keyword arguments:
  all values are evaluated, but only the leftmost value is used. The
  exception is the handling of tokenized attributes, such as :CLASS or
  :REL. The class of a tag is the union of all its :CLASS arguments.

  The argument :DATASET introduces a list of :DATA-FOO arguments:

     (:p :dataset (:duck (dolomphious) :fish 'fizzgigious
                         :spoon "runcible"))
     ≡ (:p :data-duck (dolomphious) :data-fish 'fizzgigious
           :data-spoon "runcible")

  For flexibility, even at the cost of efficiency, the argument :ATTRS
  introduces a form to evaluate at run time for a plist of extra
  attributes and values.

- Forms after the attributes are treated as arguments. Each non-nil
  (primary) value returned by an argument to a tag is written to the
  stream by HTML, a generic function on which you can define your own
  methods. By default only literal arguments are printed. Literal
  arguments are strings, characters, numbers and symbols beside NIL.

WITH-HTML-STRING is like WITH-HTML, but intercepts the generated HTML
at run time and returns a string.

{'fraction_non_alphanumeric': 0.0864911153928495, 'fraction_numerical': 0.005352172982230786, 'mean_word_length': 2.7237943403746514, 'pattern_counts': {'":': 0, '<': 63, '<?xml version=': 0, '>': 63, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4273832', 'n_tokens_mistral': 2797, 'n_tokens_neox': 2598, 'n_words': 1154}