<commit_before># Overview and Roadmap

This document provides an overview and roadmap for different parts of the ZetaVM project.

## ZetaVM Prototype

The current implementation of ZetaVM is at the prototype stage. There is currently an interpeter, but no garbage collector and no JIT compiler. The current goal is to get the system running quickly with a core set of features, so that we can  prototype, properly the system, iron out bugs, and alter the design as necessary.

## Plush

The first language implemented on top of ZetaVM is called Plush. This language is inspired by JavaScript and Lua, and serves both to bootstrap the system, and to demonstrate to beginners how they can build their own language targetting ZetaVM. The Plush language may eventually have optional extensions, but the core language itself will intentionally be kept simple so that its implementation remains simple and accessible to beginners.

## Core Libraries

A set of core libraries will be provided as part of ZetaVM. These will be written in C++ and implement a set of intput/output APIs to interface with the outside world. The APIs provided will intentionally be kept low-level, simple and minimalistic to minimize the risk of introducing corner cases and undefined behaviors. It is expected that higher-level libraries will be implemented on top of these and provide more user-friendly interfaces.

The core libraries will cover services such as file I/O, console I/O, basic 2D graphics (pixel plotting and blitting), mouse and keyboard input as well as raw PCM audio output. The early prototype version of the VM will implement only the most essential libraries.

## Language Extensions

The Plush language will eventually make it possible for people to write parser extensions. This will allow the addition of new operators and new expression types to the language. I would like to implement closures as a language extension, but also possibly template strings, regular expressions, and pattern matching.

## Immutable Package Manager

The VM will eventually ship with a package manager. This package manager will make it trivial to immediately upload code you have written from the command-line and make it available to anyone. The package manager will also be used to upload your own language implementation package, making it instantly accessible to anyone using ZetaVM.

Packages will be versioned and immutable. That is, once a package is uploaded, it will be assigned a version number, e.g. `"john/imagelib/56"`. This package version will then be frozen and unchangeable. The aim is to make it so that once code relies on a specific package version, the dependencies can never be changed and broken. Hence, by freezing the core VM semantics, and freezing submitted packages, we will hopefully make it possible to write software that doesn't break.

## JIT Compiler

Once ZetaVM is past the initial prototyping stage, a JIT compiler will be implemented to improve performance. This JIT will likely be based on basic block versioning. The [current plan](https://pointersgonewild.com/2017/06/11/zetas-jitterpreter/) is that the JIT will use the same internal representation as the interpreter, that is, the interpreter will do part of the compilation work.
<commit_msg>Update roadmap.md<commit_after># Overview and Roadmap

This document provides an overview and roadmap for different parts of the ZetaVM project.

## ZetaVM Prototype

The current implementation of ZetaVM is at the prototype stage. There is currently an interpeter, but no garbage collector and no JIT compiler. The current goal is to get the system running quickly with a core set of features, so that we can  prototype, properly the system, iron out bugs, and alter the design as necessary.

## Plush

The first language implemented on top of ZetaVM is called Plush. This language is inspired by JavaScript and Lua, and serves both to bootstrap the system, and to demonstrate to beginners how they can build their own language targetting ZetaVM. The Plush language may eventually have optional extensions, but the core language itself will intentionally be kept simple so that its implementation remains simple and accessible to beginners.

## Core Libraries

A set of core libraries will be provided as part of ZetaVM. These will be written in C++ and implement a set of intput/output APIs to interface with the outside world. The APIs provided will intentionally be kept low-level, simple and minimalistic to minimize the risk of introducing corner cases and undefined behaviors. It is expected that higher-level libraries will be implemented on top of these and provide more user-friendly interfaces.

The core libraries will cover services such as file I/O, console I/O, basic 2D graphics (pixel plotting and blitting), mouse and keyboard input as well as raw PCM audio output. The early prototype version of the VM will implement only the most essential libraries.

## Language Extensions

The Plush language will eventually make it possible for people to write parser extensions. This will allow the addition of new operators, new expression types, and new statement types to the language.

Some possible Plush extensions include:
- Closures
- Template strings
- Regular expressions
- Pattern matching
- Gradual typing
- Variable argument count functions, optional arguments

## Immutable Package Manager

The VM will eventually ship with a package manager. This package manager will make it trivial to immediately upload code you have written from the command-line and make it available to anyone. The package manager will also be used to upload your own language implementation package, making it instantly accessible to anyone using ZetaVM.

Packages will be versioned and immutable. That is, once a package is uploaded, it will be assigned a version number, e.g. `"john/imagelib/56"`. This package version will then be frozen and unchangeable. The aim is to make it so that once code relies on a specific package version, the dependencies can never be changed and broken. Hence, by freezing the core VM semantics, and freezing submitted packages, we will hopefully make it possible to write software that doesn't break.

## JIT Compiler

Once ZetaVM is past the initial prototyping stage, a JIT compiler will be implemented to improve performance. This JIT will likely be based on basic block versioning. The [current plan](https://pointersgonewild.com/2017/06/11/zetas-jitterpreter/) is that the JIT will use the same internal representation as the interpreter, that is, the interpreter will do part of the compilation work.

{'fraction_non_alphanumeric': 0.03226295062102393, 'fraction_numerical': 0.003332323538321721, 'mean_word_length': 4.672680412371134, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6837512', 'n_tokens_mistral': 1500, 'n_tokens_neox': 1438, 'n_words': 1012}