## Elevator Pitch

Enable file system persistence with IndexedDB (`IDBFS`), Node.js (`NODEFS`), blobs (`WORKERFS`), proxy (`PROXYFS`).

![Screenshot from 2021-05-19 23-34-01](https://user-images.githubusercontent.com/45380/118983166-238b9200-b94a-11eb-8bb7-11ea96bc5674.png)

## Motivation

Continuing the discussion from #328, this would allow a pyodide instance to work more naturally with persistent files. 

Some external use cases:
- persisting files between user sessions
- sharing files between concurrent browser tabs

Some internal use cases:
- caching downloaded wheels and API responses?

## Changes

- [x] add ` -l*fs.js` to `Makefile` to restore all backends
  - ~~considered [other backends](https://emscripten.org/docs/api_reference/Filesystem-API.html#persistent-data), but as this has some impact on build time and size without a clear use case, I have not included them for now, but an eventual API should probably plan for supporting them explicitly~~
    - [x] #1689 enabled node, and i've already though about uses cases for WORKERFS
- [x] #1692 ~~hoist `FS.mount`, `FS.mkdir` and `FS.sync` and `IDBFS` up to the `pyodide` API?~~
  - rather than doing something fancy, this has the highest likelihood of working with other language runtimes
  - as discussed below, the preferred approach is to hoist `FS` to the `pyodide` namespace, pending a better name...
- [x] ~~add a convenience API in e.g. `loadPyodide({mount: "/home/user1"})`?~~
  - a lot of use cases would be covered by "simple" persistence
  - could also add it to `sys.prefix`
  - moved to https://github.com/pyodide/pyodide/issues/1715
- [x] tests?
  - haven't looked into what this will take...
- [x] docs?
  - depends on what features we end up landing...
  - focus  on documenting notional API https://github.com/pyodide/pyodide/issues/1715

## Findings

- you can make as many mount points as you want
- they all get synced into the `MEMFS` (which must exist) with `FS.sync(true, (err)=>{...})` and likewise back out with `FS.sync(false, (err) => {...})`
  - the example guidance suggests sync-in at the start of the process and sync-out before exiting, but this is pretty limiting!

## Dead-ends

The image above was gotten by hacking `main.c`, emulating a naive/broken approach for polling. A better path is being investigated!
```c

  // Create and mount userfs immediately.
	EM_ASM({
		FS.mkdir('/home/jo');
		FS.mount(IDBFS, {}, '/home/jo');
    FS.syncfs(true, function (err) {
      err && console.error('error syncing FROM IndexedDB', err);
    });
    setInterval(function() {
        FS.syncfs(false, function (err) {
          err && console.error('error syncing TO IndexedDB', err);
      });
    }, 5000);
	});

```
Enable all emscripten file systems: IDBFS, NODEFS, PROXYFS and WORKERFS. ## Elevator Pitch

Enable file system persistence with IndexedDB (`IDBFS`), Node.js (`NODEFS`), blobs (`WORKERFS`), proxy (`PROXYFS`).

![Screenshot from 2021-05-19 23-34-01](https://user-images.githubusercontent.com/45380/118983166-238b9200-b94a-11eb-8bb7-11ea96bc5674.png)

## Motivation

Continuing the discussion from #328, this would allow a pyodide instance to work more naturally with persistent files. 

Some external use cases:
- persisting files between user sessions
- sharing files between concurrent browser tabs

Some internal use cases:
- caching downloaded wheels and API responses?

## Changes

- [x] add ` -l*fs.js` to `Makefile` to restore all backends
  - ~~considered [other backends](https://emscripten.org/docs/api_reference/Filesystem-API.html#persistent-data), but as this has some impact on build time and size without a clear use case, I have not included them for now, but an eventual API should probably plan for supporting them explicitly~~
    - [x] #1689 enabled node, and i've already though about uses cases for WORKERFS
- [x] #1692 ~~hoist `FS.mount`, `FS.mkdir` and `FS.sync` and `IDBFS` up to the `pyodide` API?~~
  - rather than doing something fancy, this has the highest likelihood of working with other language runtimes
  - as discussed below, the preferred approach is to hoist `FS` to the `pyodide` namespace, pending a better name...
- [x] ~~add a convenience API in e.g. `loadPyodide({mount: "/home/user1"})`?~~
  - a lot of use cases would be covered by "simple" persistence
  - could also add it to `sys.prefix`
  - moved to https://github.com/pyodide/pyodide/issues/1715
- [x] tests?
  - haven't looked into what this will take...
- [x] docs?
  - depends on what features we end up landing...
  - focus  on documenting notional API https://github.com/pyodide/pyodide/issues/1715

## Findings

- you can make as many mount points as you want
- they all get synced into the `MEMFS` (which must exist) with `FS.sync(true, (err)=>{...})` and likewise back out with `FS.sync(false, (err) => {...})`
  - the example guidance suggests sync-in at the start of the process and sync-out before exiting, but this is pretty limiting!

## Dead-ends

The image above was gotten by hacking `main.c`, emulating a naive/broken approach for polling. A better path is being investigated!
```c

  // Create and mount userfs immediately.
	EM_ASM({
		FS.mkdir('/home/jo');
		FS.mount(IDBFS, {}, '/home/jo');
    FS.syncfs(true, function (err) {
      err && console.error('error syncing FROM IndexedDB', err);
    });
    setInterval(function() {
        FS.syncfs(false, function (err) {
          err && console.error('error syncing TO IndexedDB', err);
      });
    }, 5000);
	});

```
package:
  name: zarr
  version: 2.6.1
source:
  sha256: fa7eac1e4ff47ff82d09c42bb4679e18e8a05a73ee81ce59cee6a441a210b2fd
  url: https://files.pythonhosted.org/packages/c2/40/1f3f01bc78aebce5d05535c74ad9d79fe514895062af4826b1d96d43e3d7/zarr-2.6.1.tar.gz
  patches:
    - patches/fix-zarrsync.patch
requirements:
  run:
    - numpy
    - asciitree
    - numcodecs
test:
  imports:
  - zarr

package:
  name: zarr
  version: 2.6.1
source:
  sha256: fa7eac1e4ff47ff82d09c42bb4679e18e8a05a73ee81ce59cee6a441a210b2fd
  url: https://files.pythonhosted.org/packages/c2/40/1f3f01bc78aebce5d05535c74ad9d79fe514895062af4826b1d96d43e3d7/zarr-2.6.1.tar.gz
  patches:
    - patches/fix-zarrsync.patch
requirements:
  run:
    - numpy
    - asciitree
    - numcodecs
test:
  imports:
  - zarr
