<jupyter_start><jupyter_text>CLI<jupyter_code>import os, pathlib, tempfile, shutil, atexit, hashlib, pprint
from IPython.display import *
from IPython import get_ipython # needed for `jupyter_execute` because magics?
RTD = os.environ.get("READTHEDOCS")<jupyter_output><empty_output><jupyter_text>The `jupyter lite` (or `jupyter-lite`) CLI provides tools for lifecycle of combining...- the core JupyterLite **static assets** - extra **application features** like _Lab Extensions_ and settings- **user-authored content** like _Notebooks_... into a **ready-to-[deploy](./deploying.md)** (and optionally **reproducible**) Jupyter sites which require an HTTP server, but no _application_ server.  Installation<jupyter_code>!pip install jupyterlite
!jupyter lite --version<jupyter_output><empty_output><jupyter_text>Extras Some extra features of different _addons_ have additional dependencies.```bashpip install jupyterlite[contents]   jupyter_server for better contents APIpip install jupyterlite[serve]   tornado for better local previewpip install jupyterlite[lab]   a known-compatible jupyterlab (entails `contents` and `serve`) TODO: [archive]   use libarchive```  Quick StartOnce installed, one can get to a locally-hosted, empty JupyterLite site with:```bashjupyter lite serve```This will:- [`init`](init) the baseline assets- [`build`](build) all extra assets    - if a `files/` folder is found, those will be copied into `_output/files`        - ... and _indexed_ if `jupyter_server` is installed- [`serve`](serve) the site locally and print out a URL  The Lite DirWhen you run `jupyter lite` commands, it assumes your current working directory is the partial contents of a JupyterLite site. You can override this with `--lite-dir`. By default, the built site will be created in `_output`, but can be overridden with `--output-dir`.<jupyter_code>if "TMP_DIR" not in globals():
    TMP_DIR = pathlib.Path(tempfile.mkdtemp(prefix="_my_lite_dir_"))
    def clean():
        shutil.rmtree(TMP_DIR)
    atexit.register(clean)
os.chdir(TMP_DIR)
print(pathlib.Path.cwd())<jupyter_output><empty_output><jupyter_text>Well-known FilesSome files in your `--lite-dir` that have special meaning: | look in path...               | ... for file named          | ... and if found                                                ||-------------------------------|-----------------------------|-----------------------------------------------------------------|| `.``./lab``./retro` | `jupyter-lite.{json,ipynb}` | merge with static in `_output/{path}/jupyter-lite.{json,ipynb}` || `.``./lab``./retro` | `overrides.json`            | merge with static in `_output/{*}/jupyter-lite.json`            || `./files/`                    | `*`                         | copy verbatim to `_output/files/*` and index in `/api/contents` |> See more about [merging contents](merging-contents).  Usage  Common Parameters| parameter                | description                                                                          | default                       | environment variable        || ------------------------ | ------------------------------------------------------------------------------------ | ----------------------------- | --------------------------- || `--lite-dir`             | configuration and content for the site                                               | current working directory     | `JUPYTERLITE_DIR`           || `--output-dir`           | where the hostable site will be created                                              | `_output`                     | `JUPYTERLITE_OUTPUT_DIR`    || `--cache-dir`            | a cache directory for downloads                                                      | `/.cache`           | `JUPYTERLITE_CACHE_DIR`     || `--app-archive`          | an alternate site to base off of                                                     | bundled                       |                             || `--contents`             | directory to copy to `_output/files/` and available as _Contents_                    | `./files`                     |                             || `--ignore-contents`      | patterns that should _never_ be included in `/files/` (even if found in `lite-dir`)  | various                       |                             || `--output-archive`       | the path to the archive                                                              | `-jupyterlite.tgz` | `JUPYTERLAB_OUTPUT_ARCHIVE` || `--port`                 | port on `127.0.0.1` to serve the test server                                         | `8000`                        | `JUPYTERLITE_PORT`          || `--base-url`             | the URL prefix to include before the site                                            | `/`                           | `JUPYTERLITE_BASE_URL`      || `--source-date-epoch`    | optionally enable additional reproducible build measures (best-effort!)              |                               | `SOURCE_DATE_EPOCH`         || `--federated-extensions` | paths to folders, `pip`/`conda` packages with extensions [see note](conda-packages) |                               |                             || `--ignore-sys-prefix`    | don't copy any contents, such as install labextensions, from `sys.prefix`            | `False`                       |                             || `--settings-overrides`   | additional settings overrides to enable                                              |                               |                             | All parameters may be configured via a `jupyter_lite_config.json` in the directory where `jupyter lite` is launched, or given via `--config`. ```{hint}For an advanced example, see the [configuration](https://github.com/jupyterlite/jupyterlite/tree/main/examples) used for this documentation.```  HelpThe CLI provides its own documentation, under `--help` (or `-h`).<jupyter_code>!jupyter lite --help<jupyter_output><empty_output><jupyter_text>StatusAlways safe to run, this command provides an overview of what JupyterLite has been doing.<jupyter_code>!jupyter lite status<jupyter_output><empty_output><jupyter_text>ListAlways safe to run, this command provides an overview of what JupyterLite _might_ do.> _TODO: improve on default output_<jupyter_code>!jupyter lite list<jupyter_output><empty_output><jupyter_text>InitCopy all the static data to the `--output-dir`.<jupyter_code>!jupyter lite init<jupyter_output><empty_output><jupyter_text>BuildCopy all the **user-authored content** to the `--output-dir`, and applies appropriate changes to e.g. generated Contents API responses.Special well-known files will be _merged_ appropriately, but generally, files that exist in the user directory will overwrite any existing content.<jupyter_code>!jupyter lite build<jupyter_output><empty_output><jupyter_text>ServeServe the `--output-dir` on `http://127.0.0.1:{--port=8000}{--base-url=/}`.```{warning}This is _not_ a production server. Please consider _any_ of the [deployment](./deploying.md) optionsbefore trying to make this something it isn't.```<jupyter_code>!jupyter lite serve --help<jupyter_output><empty_output><jupyter_text>CheckUse all available mechanisms to verify that the build folder conforms to schema, etc.<jupyter_code>!jupyter lite check<jupyter_output><empty_output><jupyter_text>ArchiveTurn the _output directory_ into a `.tgz` file. This is usually easier to move around than (sometimes) hundreds of files, and can be used as the baseline for future sites.> This command is _relatively_ expensive, and is skipped for documentation purposes<jupyter_code>!jupyter lite archive --help
if not RTD:
    !jupyter lite archive<jupyter_output><empty_output><jupyter_text>But let's talk about a more _reproducible_ asset.<jupyter_code># we clean out the TMP_DIR for the reproducibility examples
shutil.rmtree(TMP_DIR / "_output", ignore_errors=True)<jupyter_output><empty_output><jupyter_text>Reproducible Archives> _🛠️ This feature is a **work-in-progress**, and should not be relied upon by any production workflows **Just Yet**._ If `--source-date-epoch` is given, a number of measures will be taken to *try to ensure* that the output of `jupyter lite archive`, an npm-compatible `tgz` package, always returns a bit-for-bit [reproducible build](https://reproducible-builds.org).The most obvious change is that the modified time of each file "clamped" to that time. Some other changes:- file ownership will be reset- predictable sorting will be used- additional checks will be applied```{note}This is a shortcut for setting the environment variable `SOURCE_DATE_EPOCH`:| platform         | command                                               ||------------------|-------------------------------------------------------|| LinuxMacOS  | `export SOURCE_DATE_EPOCH=`              || Windows          | `set SOURCE_DATE_EPOCH=`                 || Python           | `os.environ.update(SOURCE_DATE_EPOCH, )` |```<jupyter_code>if not "source_date_epoch" in globals():
    from datetime import datetime
    source_date_epoch = int(datetime.utcnow().timestamp())

print("SOURCE_DATE_EPOCH is", source_date_epoch)
if not RTD:
    !jupyter lite archive --source-date-epoch {source_date_epoch} --output-archive ./a.tgz<jupyter_output><empty_output><jupyter_text>If we clear out our `_output`...<jupyter_code>if not RTD:
    shutil.rmtree(TMP_DIR / "_output", ignore_errors=True)
    pprint.pprint([*TMP_DIR.rglob("*")])<jupyter_output><empty_output><jupyter_text>...and rebuild, we should always get the same file.<jupyter_code>if not RTD:
    !jupyter lite archive --source-date-epoch {source_date_epoch} --output-archive ./b.tgz
if not RTD:
    a, b = [
        hashlib.sha256((TMP_DIR / f"{x}.tgz").read_bytes()).hexdigest() 
        for x in "ab"
    ]
    print("We built app archives with the SHA256SUMS of:\n", a, "\n", b)
    try:
        assert a == b, "We did not reproducibly build today.\n- {}\n- {}\n\n".format(a, b)
    except AssertionError as err:
        if shutil.which("diffoscope"):
            print("We did NOT reproducibly build today, checking in with `diffoscope`...")
            !diffoscope a.tgz b.tgz
        print("...but at least we tried REALLY hard!\n")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13829787234042554, 'fraction_numerical': 0.002537575639273863, 'mean_word_length': 2.280089628681178, 'pattern_counts': {'":': 0, '<': 61, '<?xml version=': 0, '>': 65, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 42, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '103687', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2908, 'n_tokens_neox': 2803, 'n_words': 852, 'path': 'docs/cli.ipynb', 'repo_name': 'MattiasFestin/jupyterlite', 'size': '10246'}