## Elevator Pitch

Enable file system persistence with IndexedDB (`IDBFS`), Node.js (`NODEFS`), blobs (`WORKERFS`), proxy (`PROXYFS`).

![Screenshot from 2021-05-19 23-34-01](https://user-images.githubusercontent.com/45380/118983166-238b9200-b94a-11eb-8bb7-11ea96bc5674.png)

## Motivation

Continuing the discussion from #328, this would allow a pyodide instance to work more naturally with persistent files. 

Some external use cases:
- persisting files between user sessions
- sharing files between concurrent browser tabs

Some internal use cases:
- caching downloaded wheels and API responses?

## Changes

- [x] add ` -l*fs.js` to `Makefile` to restore all backends
  - ~~considered [other backends](https://emscripten.org/docs/api_reference/Filesystem-API.html#persistent-data), but as this has some impact on build time and size without a clear use case, I have not included them for now, but an eventual API should probably plan for supporting them explicitly~~
    - [x] #1689 enabled node, and i've already though about uses cases for WORKERFS
- [x] #1692 ~~hoist `FS.mount`, `FS.mkdir` and `FS.sync` and `IDBFS` up to the `pyodide` API?~~
  - rather than doing something fancy, this has the highest likelihood of working with other language runtimes
  - as discussed below, the preferred approach is to hoist `FS` to the `pyodide` namespace, pending a better name...
- [x] ~~add a convenience API in e.g. `loadPyodide({mount: "/home/user1"})`?~~
  - a lot of use cases would be covered by "simple" persistence
  - could also add it to `sys.prefix`
  - moved to https://github.com/pyodide/pyodide/issues/1715
- [x] tests?
  - haven't looked into what this will take...
- [x] docs?
  - depends on what features we end up landing...
  - focus  on documenting notional API https://github.com/pyodide/pyodide/issues/1715

## Findings

- you can make as many mount points as you want
- they all get synced into the `MEMFS` (which must exist) with `FS.sync(true, (err)=>{...})` and likewise back out with `FS.sync(false, (err) => {...})`
  - the example guidance suggests sync-in at the start of the process and sync-out before exiting, but this is pretty limiting!

## Dead-ends

The image above was gotten by hacking `main.c`, emulating a naive/broken approach for polling. A better path is being investigated!
```c

  // Create and mount userfs immediately.
	EM_ASM({
		FS.mkdir('/home/jo');
		FS.mount(IDBFS, {}, '/home/jo');
    FS.syncfs(true, function (err) {
      err && console.error('error syncing FROM IndexedDB', err);
    });
    setInterval(function() {
        FS.syncfs(false, function (err) {
          err && console.error('error syncing TO IndexedDB', err);
      });
    }, 5000);
	});

```
Enable all emscripten file systems: IDBFS, NODEFS, PROXYFS and WORKERFS. ## Elevator Pitch

Enable file system persistence with IndexedDB (`IDBFS`), Node.js (`NODEFS`), blobs (`WORKERFS`), proxy (`PROXYFS`).

![Screenshot from 2021-05-19 23-34-01](https://user-images.githubusercontent.com/45380/118983166-238b9200-b94a-11eb-8bb7-11ea96bc5674.png)

## Motivation

Continuing the discussion from #328, this would allow a pyodide instance to work more naturally with persistent files. 

Some external use cases:
- persisting files between user sessions
- sharing files between concurrent browser tabs

Some internal use cases:
- caching downloaded wheels and API responses?

## Changes

- [x] add ` -l*fs.js` to `Makefile` to restore all backends
  - ~~considered [other backends](https://emscripten.org/docs/api_reference/Filesystem-API.html#persistent-data), but as this has some impact on build time and size without a clear use case, I have not included them for now, but an eventual API should probably plan for supporting them explicitly~~
    - [x] #1689 enabled node, and i've already though about uses cases for WORKERFS
- [x] #1692 ~~hoist `FS.mount`, `FS.mkdir` and `FS.sync` and `IDBFS` up to the `pyodide` API?~~
  - rather than doing something fancy, this has the highest likelihood of working with other language runtimes
  - as discussed below, the preferred approach is to hoist `FS` to the `pyodide` namespace, pending a better name...
- [x] ~~add a convenience API in e.g. `loadPyodide({mount: "/home/user1"})`?~~
  - a lot of use cases would be covered by "simple" persistence
  - could also add it to `sys.prefix`
  - moved to https://github.com/pyodide/pyodide/issues/1715
- [x] tests?
  - haven't looked into what this will take...
- [x] docs?
  - depends on what features we end up landing...
  - focus  on documenting notional API https://github.com/pyodide/pyodide/issues/1715

## Findings

- you can make as many mount points as you want
- they all get synced into the `MEMFS` (which must exist) with `FS.sync(true, (err)=>{...})` and likewise back out with `FS.sync(false, (err) => {...})`
  - the example guidance suggests sync-in at the start of the process and sync-out before exiting, but this is pretty limiting!

## Dead-ends

The image above was gotten by hacking `main.c`, emulating a naive/broken approach for polling. A better path is being investigated!
```c

  // Create and mount userfs immediately.
	EM_ASM({
		FS.mkdir('/home/jo');
		FS.mount(IDBFS, {}, '/home/jo');
    FS.syncfs(true, function (err) {
      err && console.error('error syncing FROM IndexedDB', err);
    });
    setInterval(function() {
        FS.syncfs(false, function (err) {
          err && console.error('error syncing TO IndexedDB', err);
      });
    }, 5000);
	});

```
<div align="center">
  <a href="https://github.com/pyodide/pyodide">
  <img src="./docs/_static/img/pyodide-logo-readme.png" alt="Pyodide">
  </a>
</div>


[![Build Status](https://circleci.com/gh/pyodide/pyodide.png)](https://circleci.com/gh/pyodide/pyodide)
[![Documentation Status](https://readthedocs.org/projects/pyodide/badge/?version=stable)](https://pyodide.readthedocs.io/?badge=stable)

Python with the scientific stack, compiled to WebAssembly.

## What is Pyodide?

Pyodide may be used in any context where you want to run Python inside a web
browser.

Pyodide brings the Python 3.9 runtime to the browser via WebAssembly, along with
the Python scientific stack including NumPy, Pandas, Matplotlib, SciPy, and
scikit-learn. The [packages directory](packages) lists over 75 packages which
are currently available. In addition it's possible to install pure Python wheels
from PyPi.

Pyodide provides transparent conversion of objects between Javascript and
Python. When used inside a browser, Python has full access to the Web APIs.

## Try Pyodide (no installation needed)

Try Pyodide in a
[REPL](https://pyodide.org/en/stable/console.html) directly in
your browser. For further information, see the
[documentation](https://pyodide.org/en/stable/).

## Getting Started

Pyodide offers three different ways to get started depending on your needs and
technical resources. These include:

- Use a hosted distribution of Pyodide: see the [Getting
  Started](https://pyodide.org/en/stable/usage/quickstart.html) documentation.
- Download a version of Pyodide from the [releases
  page](https://github.com/pyodide/pyodide/releases/) and serve it
  with a web server.
- [Build Pyodide from source](https://pyodide.org/en/stable/development/building-from-sources.html)
  - Build natively with `make`: primarily for Linux users who want to
    experiment or contribute back to the project.
  - [Use a Docker image](https://pyodide.org/en/stable/development/building-from-sources.html#using-docker):
    recommended for Windows and macOS users and for Linux users who prefer a
    Debian-based Docker image with the dependencies already installed.


## History
Pyodide was created in 2018 by [Michael Droettboom](https://github.com/mdboom)
at Mozilla as part of the [Iodide
project](https://github.com/iodide-project/iodide). Iodide is an experimental
web-based notebook environment for literate scientific computing and
communication.

Iodide is no longer maintained. If you want to use Pyodide in an interactive
client-side notebook, see [Pyodide notebook
environments](https://pyodide.org/en/stable/project/related-projects.html#notebook-environments-ides-repls).

## Contributing

Please view the [contributing
guide](https://pyodide.org/en/stable/development/contributing.html) for tips
on filing issues, making changes, and submitting pull requests. Pyodide is an
independent and community-driven open-source project. The decision making
process is outlined in the [Project
governance](https://pyodide.org/en/stable/project/governance.html).

## Communication

- Mailing list: [mail.python.org/mailman3/lists/pyodide.python.org/](https://mail.python.org/mailman3/lists/pyodide.python.org/)
- Gitter: [gitter.im/pyodide/community](https://gitter.im/pyodide/community)
- Twitter: [twitter.com/pyodide](https://twitter.com/pyodide)
- Stack Overflow: [stackoverflow.com/questions/tagged/pyodide](https://stackoverflow.com/questions/tagged/pyodide)

## License

Pyodide uses the [Mozilla Public License Version
2.0](https://choosealicense.com/licenses/mpl-2.0/).

<div align="center">
  <a href="https://github.com/pyodide/pyodide">
  <img src="./docs/_static/img/pyodide-logo-readme.png" alt="Pyodide">
  </a>
</div>


[![Build Status](https://circleci.com/gh/pyodide/pyodide.png)](https://circleci.com/gh/pyodide/pyodide)
[![Documentation Status](https://readthedocs.org/projects/pyodide/badge/?version=stable)](https://pyodide.readthedocs.io/?badge=stable)

Python with the scientific stack, compiled to WebAssembly.

## What is Pyodide?

Pyodide may be used in any context where you want to run Python inside a web
browser.

Pyodide brings the Python 3.9 runtime to the browser via WebAssembly, along with
the Python scientific stack including NumPy, Pandas, Matplotlib, SciPy, and
scikit-learn. The [packages directory](packages) lists over 75 packages which
are currently available. In addition it's possible to install pure Python wheels
from PyPi.

Pyodide provides transparent conversion of objects between Javascript and
Python. When used inside a browser, Python has full access to the Web APIs.

## Try Pyodide (no installation needed)

Try Pyodide in a
[REPL](https://pyodide.org/en/stable/console.html) directly in
your browser. For further information, see the
[documentation](https://pyodide.org/en/stable/).

## Getting Started

Pyodide offers three different ways to get started depending on your needs and
technical resources. These include:

- Use a hosted distribution of Pyodide: see the [Getting
  Started](https://pyodide.org/en/stable/usage/quickstart.html) documentation.
- Download a version of Pyodide from the [releases
  page](https://github.com/pyodide/pyodide/releases/) and serve it
  with a web server.
- [Build Pyodide from source](https://pyodide.org/en/stable/development/building-from-sources.html)
  - Build natively with `make`: primarily for Linux users who want to
    experiment or contribute back to the project.
  - [Use a Docker image](https://pyodide.org/en/stable/development/building-from-sources.html#using-docker):
    recommended for Windows and macOS users and for Linux users who prefer a
    Debian-based Docker image with the dependencies already installed.


## History
Pyodide was created in 2018 by [Michael Droettboom](https://github.com/mdboom)
at Mozilla as part of the [Iodide
project](https://github.com/iodide-project/iodide). Iodide is an experimental
web-based notebook environment for literate scientific computing and
communication.

Iodide is no longer maintained. If you want to use Pyodide in an interactive
client-side notebook, see [Pyodide notebook
environments](https://pyodide.org/en/stable/project/related-projects.html#notebook-environments-ides-repls).

## Contributing

Please view the [contributing
guide](https://pyodide.org/en/stable/development/contributing.html) for tips
on filing issues, making changes, and submitting pull requests. Pyodide is an
independent and community-driven open-source project. The decision making
process is outlined in the [Project
governance](https://pyodide.org/en/stable/project/governance.html).

## Communication

- Mailing list: [mail.python.org/mailman3/lists/pyodide.python.org/](https://mail.python.org/mailman3/lists/pyodide.python.org/)
- Gitter: [gitter.im/pyodide/community](https://gitter.im/pyodide/community)
- Twitter: [twitter.com/pyodide](https://twitter.com/pyodide)
- Stack Overflow: [stackoverflow.com/questions/tagged/pyodide](https://stackoverflow.com/questions/tagged/pyodide)

## License

Pyodide uses the [Mozilla Public License Version
2.0](https://choosealicense.com/licenses/mpl-2.0/).
