<commit_before># The Magic Book Project

*This project is still working towards a 0.1.0 release, which means that the API is in active development. EPUB and MOBI formats are still not supported. We encourage developers to try the releases and report any issues in the issue tracker.*

The Magic Book Project is an open source project funded by New York University's Interactive Telecommunications Program. It aims to be the best free tool for creating print and digital books from a single source.

This project is for you, if:

- You want to write your book in plain text (Markdown or HTML)
- You want to export to many different formats (static website, pdf, epub, mobi)
- You want your source to be completely free of format-specific hacks
- You want to use CSS to design the look of your book
- You want to use JavaScript to add interactivity to digital formats
- You want the ability to define custom elements that behave differently in every format
- You want to use a command-line tool for all of this
- You want that command-line tool be be written in Node-only. No more XSLT.

Although a small number of open source publishing frameworks already exists, it's hard to find any that are flexible enough to create modern, interactive books for the web while also doing print-ready PDF export.

## Getting Started

First install the `magicbook` package:

```bash
npm install magicbook -g
```

Then run the new project generator:

```bash
magicbook new myproject
```

This will give you a very basic project folder with a `magicbook.json` configuration file. Now `cd` into the new project and build the book.

```bash
cd myproject
magicbook build
```

You now have a `myproject/build` directory with two builds: a website and a PDF. This is of course a very basic setup. Consult the rest of this README for all the available functionality.

## Configuration

To specify configuration for your project, `magicbook` uses a file called `magicbook.json` in your project folder. When you run `magicbook build`, the configuration file will automatically be detected. If you wish to have a custom name and location for the configuration file, you can use the `--config` command line argument.

```bash
magicbook build --config=myfolder/myconfig.json
```

## Source files

You can write your book in `.md`, `.html`, or both. `magicbook` uses a very simple layer on top of HTML5 called [HTMLBook](http://oreillymedia.github.io/HTMLBook) to define the various elements in a book. This mostly means using a few `data-type` attributes to specify chapters and sections, and it's very easy to learn. It is also what makes it possible `magicbook` to do its magic when generating table of contents, etc.

### Writing in Markdown

If you chose to write your book in Markdown, `magicbook` will automatically convert your markdown to HTMLBook. A simple file like the following...

```md
# Chapter title

## Sect 1

### Sect 2
```

... will be converted to the following HTMLBook markup.

```html
<section data-type="chapter">
  <h1>Chapter Title</h1>
  <section data-type="sect1">
    <h1>Sect 1</h1>
    <section data-type="sect2">
      <h2>Sect 2</h2>
    </section>
  </section>
</section>
```

### Writing in HTML

If you choose to write in HTML, you will need to make sure that you're using the HTMLBook `data-type` attributes. `magicbook` will not break if you don't use them, but it won't be possible to generate a table of contents, etc.

### Building source files

You can specify the files to build by adding a `files` array to your `magicbook.json` file. If you do not have a `files` array, it will look for all markdown files in `content/*.md`.

You can set the files property to be a single glob.

```json
{
  "files" : "content/*.md"
}
```

You can set the files array to be an array of globs.

```json
{
  "files" : [
    "content/chapter1/*.md",
    "content/chapter2/*.md"
  ]
}
```

Using an array, you can also specify each of the files you want to build.

```json
{
  "files" : [
    "content/first-file.md",
    "content/second-file.md",
    "content/third-file.md"
  ]
}
```

*This setting is also available in the configuration for each build*, which you can read more about below.

## Builds

You must add a `builds` array to your configuration that as a minimum defines the format of each build. Here's a simple example with the bare minimum configuration to build your book into a website.

```json
{
  "builds" : [
    { "format" : "html" }
  ]
}
```

`magicbook` has the following built-in formats:

- `html` will save all source files as separate `.html` files as a static website.
- `pdf` will combine all source files, bundle them into a single `.html` file, and generate a PDF in the format destination folder. Currently this process uses Prince XML for PDF generation, as it's one of the few applications that can do print-ready PDF files from HTML. You will need a Prince XML license to use it without a watermark.
- `epub` TODO
- `mobi` TODO

The `builds` array is a very powerful concept, as it further allows you to specify specific settings for each build. For example, here's a book that uses a different introduction for the HTML and PDF builds.

```json
{
  "builds" : [
    {
      "format" : "pdf",<commit_msg>more readme<commit_after># The Magic Book Project

*This project is still working towards a 0.1.0 release, which means that the API is in active development. EPUB and MOBI formats are still not supported. We encourage developers to try the releases and report any issues in the issue tracker.*

The Magic Book Project is an open source project funded by New York University's Interactive Telecommunications Program. It aims to be the best free tool for creating print and digital books from a single source.

This project is for you, if:

- You want to write your book in plain text (Markdown or HTML)
- You want to export to many different formats (static website, pdf, epub, mobi)
- You want your source to be completely free of format-specific hacks
- You want to use CSS to design the look of your book
- You want to use JavaScript to add interactivity to digital formats
- You want the ability to define custom elements that behave differently in every format
- You want to use a command-line tool for all of this
- You want that command-line tool be be written in Node-only. No more XSLT.

Although a small number of open source publishing frameworks already exists, it's hard to find any that are flexible enough to create modern, interactive books for the web while also doing print-ready PDF export.

## Getting Started

First install the `magicbook` package:

```bash
npm install magicbook -g
```

Then run the new project generator:

```bash
magicbook new myproject
```

This will give you a very basic project folder with a `magicbook.json` configuration file. Now `cd` into the new project and build the book.

```bash
cd myproject
magicbook build
```

You now have a `myproject/build` directory with two builds: a website and a PDF. This is of course a very basic setup. Consult the rest of this README for all the available functionality.

## Configuration

To specify configuration for your project, `magicbook` uses a file called `magicbook.json` in your project folder. When you run `magicbook build`, the configuration file will automatically be detected. If you wish to have a custom name and location for the configuration file, you can use the `--config` command line argument.

```bash
magicbook build --config=myfolder/myconfig.json
```

## Source files

You can write your book in `.md`, `.html`, or both. `magicbook` uses a very simple layer on top of HTML5 called [HTMLBook](http://oreillymedia.github.io/HTMLBook) to define the various elements in a book. This mostly means using a few `data-type` attributes to specify chapters and sections, and it's very easy to learn. It is also what makes it possible `magicbook` to do its magic when generating table of contents, etc.

### Writing in Markdown

If you chose to write your book in Markdown, `magicbook` will automatically convert your markdown to HTMLBook. A simple file like the following...

```md
# Chapter title

## Sect 1

### Sect 2
```

... will be converted to the following HTMLBook markup.

```html
<section data-type="chapter">
  <h1>Chapter Title</h1>
  <section data-type="sect1">
    <h1>Sect 1</h1>
    <section data-type="sect2">
      <h2>Sect 2</h2>
    </section>
  </section>
</section>
```

### Writing in HTML

If you choose to write in HTML, you will need to make sure that you're using the HTMLBook `data-type` attributes. `magicbook` will not break if you don't use them, but it won't be possible to generate a table of contents, etc.

### The files array

You can specify the files to build by adding a `files` array to your `magicbook.json` file. If you do not have a `files` array, it will look for all markdown files in `content/*.md`.

You can set the files property to be a single glob.

```json
{
  "files" : "content/*.md"
}
```

You can set the files array to be an array of globs.

```json
{
  "files" : [
    "content/chapter1/*.md",
    "content/chapter2/*.md"
  ]
}
```

Using an array, you can also specify each of the files you want to build.

```json
{
  "files" : [
    "content/first-file.md",
    "content/second-file.md",
    "content/third-file.md"
  ]
}
```

Globs are an important concept to learn, as your files array dictates whether or not files in subfolders are transferred to the build folder. This, combined with the fact that the build process will **remove leading numbers, dashes and underscores from folders and filenames**, can help you create just the folder structure you want for your website.

As an example, consider the following folder structure.

```
contents/
  01-first-chapter/
    01-first-file.md
    02-second-file.md
```

This files array:

```json
{
  "files" : [
    "contents/**/*.md"
  ]
}
```

... will create a `html` build that looks like this:

```
build1/
  first-chapter/
    first-file.html
    second-file.html
```

Because the files glob does not set any ordering, you can use numbers in files and folders to decide the order of the files. Because of the wildcard glob `**`, the subfolder is transfered to the build. However, this files array:

```json
{
  "files" : [
    "contents/01-first-chapter/01-first-file.md",
    "contents/01-first-chapter/02-second-file.md"
  ]
}
```

... will create a `html` build that looks like this, without a subfolder:

```
build1/
  first-file.html
  second-file.html
```

*This setting is also available in the configuration for each build*, which you can read more about below.

## Builds

You must add a `builds` array to your configuration that as a minimum defines the format of each build. Here's a simple example with the bare minimum configuration to build your book into a website.

```json
{
  "builds" : [
    { "format" : "html" }
  ]
}
```

`magicbook` has the following built-in formats:

- `html` will save all source files as separate `.html` files as a static website.
- `pdf` will combine all source files, bundle them into a single `.html` file, and generate a PDF in the format destination folder. Currently this process uses Prince XML for PDF generation, as it's one of the few applications that can do print-ready PDF files from HTML. You will need a Prince XML license to use it without a watermark.
- `epub` TODO
- `mobi` TODO

The `builds` array is a very powerful concept, as it further allows you to specify specific settings for each build. For example, here's a book that uses a different introduction for the HTML and PDF builds.

```json
{
  "builds" : [
    {
      "format" : "pdf",
{'fraction_non_alphanumeric': 0.0820684332390018, 'fraction_numerical': 0.004459308807134894, 'mean_word_length': 3.773638968481375, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '32583', 'n_tokens_mistral': 3326, 'n_tokens_neox': 3168, 'n_words': 1735}