## Elevator Pitch

Enable file system persistence with IndexedDB (`IDBFS`), Node.js (`NODEFS`), blobs (`WORKERFS`), proxy (`PROXYFS`).

![Screenshot from 2021-05-19 23-34-01](https://user-images.githubusercontent.com/45380/118983166-238b9200-b94a-11eb-8bb7-11ea96bc5674.png)

## Motivation

Continuing the discussion from #328, this would allow a pyodide instance to work more naturally with persistent files. 

Some external use cases:
- persisting files between user sessions
- sharing files between concurrent browser tabs

Some internal use cases:
- caching downloaded wheels and API responses?

## Changes

- [x] add ` -l*fs.js` to `Makefile` to restore all backends
  - ~~considered [other backends](https://emscripten.org/docs/api_reference/Filesystem-API.html#persistent-data), but as this has some impact on build time and size without a clear use case, I have not included them for now, but an eventual API should probably plan for supporting them explicitly~~
    - [x] #1689 enabled node, and i've already though about uses cases for WORKERFS
- [x] #1692 ~~hoist `FS.mount`, `FS.mkdir` and `FS.sync` and `IDBFS` up to the `pyodide` API?~~
  - rather than doing something fancy, this has the highest likelihood of working with other language runtimes
  - as discussed below, the preferred approach is to hoist `FS` to the `pyodide` namespace, pending a better name...
- [x] ~~add a convenience API in e.g. `loadPyodide({mount: "/home/user1"})`?~~
  - a lot of use cases would be covered by "simple" persistence
  - could also add it to `sys.prefix`
  - moved to https://github.com/pyodide/pyodide/issues/1715
- [x] tests?
  - haven't looked into what this will take...
- [x] docs?
  - depends on what features we end up landing...
  - focus  on documenting notional API https://github.com/pyodide/pyodide/issues/1715

## Findings

- you can make as many mount points as you want
- they all get synced into the `MEMFS` (which must exist) with `FS.sync(true, (err)=>{...})` and likewise back out with `FS.sync(false, (err) => {...})`
  - the example guidance suggests sync-in at the start of the process and sync-out before exiting, but this is pretty limiting!

## Dead-ends

The image above was gotten by hacking `main.c`, emulating a naive/broken approach for polling. A better path is being investigated!
```c

  // Create and mount userfs immediately.
	EM_ASM({
		FS.mkdir('/home/jo');
		FS.mount(IDBFS, {}, '/home/jo');
    FS.syncfs(true, function (err) {
      err && console.error('error syncing FROM IndexedDB', err);
    });
    setInterval(function() {
        FS.syncfs(false, function (err) {
          err && console.error('error syncing TO IndexedDB', err);
      });
    }, 5000);
	});

```
Enable all emscripten file systems: IDBFS, NODEFS, PROXYFS and WORKERFS. ## Elevator Pitch

Enable file system persistence with IndexedDB (`IDBFS`), Node.js (`NODEFS`), blobs (`WORKERFS`), proxy (`PROXYFS`).

![Screenshot from 2021-05-19 23-34-01](https://user-images.githubusercontent.com/45380/118983166-238b9200-b94a-11eb-8bb7-11ea96bc5674.png)

## Motivation

Continuing the discussion from #328, this would allow a pyodide instance to work more naturally with persistent files. 

Some external use cases:
- persisting files between user sessions
- sharing files between concurrent browser tabs

Some internal use cases:
- caching downloaded wheels and API responses?

## Changes

- [x] add ` -l*fs.js` to `Makefile` to restore all backends
  - ~~considered [other backends](https://emscripten.org/docs/api_reference/Filesystem-API.html#persistent-data), but as this has some impact on build time and size without a clear use case, I have not included them for now, but an eventual API should probably plan for supporting them explicitly~~
    - [x] #1689 enabled node, and i've already though about uses cases for WORKERFS
- [x] #1692 ~~hoist `FS.mount`, `FS.mkdir` and `FS.sync` and `IDBFS` up to the `pyodide` API?~~
  - rather than doing something fancy, this has the highest likelihood of working with other language runtimes
  - as discussed below, the preferred approach is to hoist `FS` to the `pyodide` namespace, pending a better name...
- [x] ~~add a convenience API in e.g. `loadPyodide({mount: "/home/user1"})`?~~
  - a lot of use cases would be covered by "simple" persistence
  - could also add it to `sys.prefix`
  - moved to https://github.com/pyodide/pyodide/issues/1715
- [x] tests?
  - haven't looked into what this will take...
- [x] docs?
  - depends on what features we end up landing...
  - focus  on documenting notional API https://github.com/pyodide/pyodide/issues/1715

## Findings

- you can make as many mount points as you want
- they all get synced into the `MEMFS` (which must exist) with `FS.sync(true, (err)=>{...})` and likewise back out with `FS.sync(false, (err) => {...})`
  - the example guidance suggests sync-in at the start of the process and sync-out before exiting, but this is pretty limiting!

## Dead-ends

The image above was gotten by hacking `main.c`, emulating a naive/broken approach for polling. A better path is being investigated!
```c

  // Create and mount userfs immediately.
	EM_ASM({
		FS.mkdir('/home/jo');
		FS.mount(IDBFS, {}, '/home/jo');
    FS.syncfs(true, function (err) {
      err && console.error('error syncing FROM IndexedDB', err);
    });
    setInterval(function() {
        FS.syncfs(false, function (err) {
          err && console.error('error syncing TO IndexedDB', err);
      });
    }, 5000);
	});

```
Pyodide
=======

Python with the scientific stack, compiled to WebAssembly.

Pyodide may be used in any context where you want to run Python inside a web
browser.

Pyodide brings the Python 3.9 runtime to the browser via WebAssembly, along
with the Python scientific stack including NumPy, Pandas, Matplotlib, SciPy, and
scikit-learn. Over 75 packages are currently available. In addition it's
possible to install pure Python wheels from PyPi.

Pyodide provides transparent conversion of objects between Javascript and
Python. When used inside a browser, Python has full access to the Web APIs.

Using Pyodide
=============

.. toctree::
   :maxdepth: 2

   usage/quickstart.md
   usage/webworker.md
   usage/serving-pyodide-packages.md
   usage/loading-packages.md
   usage/type-conversions.md
   usage/wasm-constraints.md
   usage/api-reference.md
   usage/faq.md

Development
===========

The Development section help Pyodide contributors to find information about the
development process including making packages to support third party libraries
and understanding type conversions between Python and JavaScript.

.. toctree::
   :maxdepth: 1
   :caption: Development

   development/building-from-sources.md
   development/new-packages.md
   development/contributing.md
   development/testing.md
   development/debugging.md


Project
=======

The Project section helps contributors get started and gives additional
information about the project's organization.

.. toctree::
   :maxdepth: 1
   :caption: Project

   project/about.md
   project/roadmap.md
   project/code-of-conduct.md
   project/governance.md
   project/changelog.md
   project/related-projects.md

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

Pyodide
=======

Python with the scientific stack, compiled to WebAssembly.

Pyodide may be used in any context where you want to run Python inside a web
browser.

Pyodide brings the Python 3.9 runtime to the browser via WebAssembly, along
with the Python scientific stack including NumPy, Pandas, Matplotlib, SciPy, and
scikit-learn. Over 75 packages are currently available. In addition it's
possible to install pure Python wheels from PyPi.

Pyodide provides transparent conversion of objects between Javascript and
Python. When used inside a browser, Python has full access to the Web APIs.

Using Pyodide
=============

.. toctree::
   :maxdepth: 2

   usage/quickstart.md
   usage/webworker.md
   usage/serving-pyodide-packages.md
   usage/loading-packages.md
   usage/type-conversions.md
   usage/wasm-constraints.md
   usage/api-reference.md
   usage/faq.md

Development
===========

The Development section help Pyodide contributors to find information about the
development process including making packages to support third party libraries
and understanding type conversions between Python and JavaScript.

.. toctree::
   :maxdepth: 1
   :caption: Development

   development/building-from-sources.md
   development/new-packages.md
   development/contributing.md
   development/testing.md
   development/debugging.md


Project
=======

The Project section helps contributors get started and gives additional
information about the project's organization.

.. toctree::
   :maxdepth: 1
   :caption: Project

   project/about.md
   project/roadmap.md
   project/code-of-conduct.md
   project/governance.md
   project/changelog.md
   project/related-projects.md

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
