<commit_before>
<< basic idea is to compile to be able to compile to C >>
<< tables are a combination of vector of type A, hashmap/unsorted_map/dictionary of type B, and a linked list of boxed variants? >>
<< tables are the only data structure not copied by value and are a single ui64 address pointing to contiguous memory which contains the vecotr and hashmap portion >>
<< tables contain sizebytes, size, and an ownership flag so have at least 24 bytes over overhead - ownership flag is on the pointer's highest bit? >>  
<< <> is a compile time expression, <<>> is a line independant comment >>
<< should be able to call C nativly >>
<< compile time expressions are used to express any meta programming or function overloading >>
<< compile time expressions take strings and a context and return strings >>
<< CTEs have compiler functions exposed to them and can parse the incoming string into an ast >>
<< variables at compile time are a dictionary containing their symbol string and any other key value pairs given to them (such as type) >> 
<< all compilation must be able to be performed by compiling the file with the entry point. there must be exactly one global expression with the key 'entry' mapped to 'true' ? >>

<mod 'io' 'str' 'net'>                 << include io string and network modules using the compile time function mod >>

mainCRTStartup :: entry:t = {          << make the entry point for the executable _CRT_main >> 
  io_out('hello world');               << ';' marks the end of an expression if it is not contained by {} >>
  0;                                   << last value is the return value if 'ret' keyword is not used     >>
};                                     << } is the end of an expression, but the whole expression is the assignment to mainCRTStartup and so needs to end with ';' >> 

{cp mainCRTStartup::entry:t {io_out('hello world'); 0;} }  << alternate syntax without using any infix operators >>


{x les y|io_out('x is less than y')}   << branching with basic conditional execution >>
{x les y | t | f}                      << branching >>
{x les y
| 'less'
| {x eq y|'equal'|'more'}}             << if elseif else >>

std = { add={x,y| x + y} };            << effectively a namespace since the expression takes no arguments >>
{io_out {std_add 1 2}}                 << prints 3 >>
std_add(1,2) :> io_out;                << prints 3 >>
{|io_out('wat')} <: {x les y};         << expression conditionally evaluated - passing a boolean to a lazy expression dictates if it is evaluated  >>
imgbuf!{x|x/2}                         << divides each element in the table 'imgbuf' by 2  >>
loop { io_out('hello world\n'); }      << prints 'hello world' lines forever >>
loop io_out('hello world\n');          << does the same since loop is a unary operator >>
<to 10 {x|io_out('line ',x,'\n')}>     << prints 10 lines using the compile time function 'to' >>

sizeof   operator giving the size in bytes of a type - same as C
addr     operator giving the address of a variable as a ui64 - like & in C
deref    operator giving the value at an ui64 address - like * in C
mv       operator that returns a table but sets the old table address to have the ownership flag set to false

{x=0,y=0,z=0;|}                                                   << closures and structs are the same thing and have 8 bytes of overhead for the attached function address - this closure's (or struct's) function would have its address as 0 and return 0 >>
a_vec = {x=0.0f,y=0.0f,z=0.0f; | {x*x + y*y + z*z} :> sqrt}       << closure that returns length - this function would be ran without arguments >>


----------------
Older ideas
----------------
Variables have any number of :metadata symbols after them

<>    means compile time expression - use this to declare cdecl expressions ? 
{}    means expression  - {|} means closure / delayed expression 
[]    means data declaration
()    means alternate syntax for argument passing
=     copies data and declaration?
<-    moves  data or copies a declaration
!     means mutate?
.     looks up from the struct part of a table if used as a value - tbl.{expr}
.     becomes a binary operator to make the left side be the first value to a function when used as a function a.b becomes {b a}
->    looks up from the map part of a table
~     means next in linked list - pass an expression to loop over the whole list
&     means reference / pointer
@     means copy
$     means atomic reference counted shared pointer
:>    means forward to first parameter of next function
:>>   means forward to first parameter of function after next
::>>  means forward to second parameter of function after next
,     means separation of data
;     means separation of expressions
|     means separation of sections in a data declaration or expression
[]    means look up from the array of a table while [|] means data declaration?

:     means compile time symbol? - can <> be used to return compile time tables instead?
``    means external function declaration
:>    means move ?
<=    copies data and a declaration ? not needed?

< [| compile_time_symbol="wat", compile_time_flag=true |] >        // a compile time table instead of less flexible :symbol declarations
<cdecl "i32" "func_name" "f32" "ui32" "ui64">                      // ui64 is used where a 64 bit pointer would be 

#img ||>> {flip img} |> {a,b|{over a b}}

{flip img} :>> {blur img} ::> {over} :> flipOverBlur
  -OR-
flipOverBlur = {over {flip img} {blur img}}
  -OR-
blurred = blur(img);
flipped = flip(img);
flipOverBlur = over(flipped, blurred);

{LoadImg file} |> img |> {blur} |> {sub _ img} |> {abs} |> {luminance _ AVG} |> {ChanImg} |> edges
{LoadImg file} :> ::>> {blur} :> {sub} :> {abs} :> {luminance _ AVG} :> {ChanImg} :> edges





<mod 'io' 'str' 'net'>            << include io string and network modules >>
{x les y | t | f}                 << branching >>
{x les y
| 'less'
| {x eq y|'equal'|'more'}}        << if elseif else >>

<< all branching expressions must be the same type >>
{ x les y -> 'less'               << same as above >>
| x eq  y -> 'equal'              << passing a boolean into an expression that takes no arguments selectivly executes the expression >>
| 'more'}                         << an expression executes segments until one does not return null, or false >> 

<< all branching expressions must be the same type >>
  x les y -> 'less'               << same as above >>
| x eq  y -> 'equal'              << passing a boolean into an expression that takes no arguments selectivly executes the expression >>
| 'more';                         << an expression executes segments until one does not return null, or false >> 

std = { add={x,y| x + y} };       << effectively a namespace since the expression takes no arguments >>
{io_out {std_add 1 2}}            << prints 3 >>
std_add(1,2) -> io_out;           << prints 3 - equivilent to io_out(std_add(1,2)) >>
{|io_out('wat')} if x les y;      << if operator combines an expression and another expression that evaluates to a boolean >>

<< take an idea from klong an use label::{expression} for declarations and/or anything that will go into the binary? >>
main::{ "hello world" -> out }
add::{x,y|x+y}
std::{ add::{x,y|x+y} sub::{x,y|x-y} }           << expression without arguments attached to a label acts as a namespace, but just uses '_' as a delimiter >>
addsub::{x,y| std_add(x,y) -> std_sub(x,y) }     << last expression doesn't need ';' to end >>
dataseg::(10; {x|x*x})            << creates a vector of length 10 of the the squares of 0..9 embedded in the initialized data segment of the .exe >> 
SQR::<x|x*x>                      << SQR(8f32)  evaluates to 64f32 at compile time >>

<| structs, closures, and deconstructors are all together >
<| a lazy compile time function doesn't exist because a normal expression could be used, so instead it is a comment >
v3f::{x=0f32, y=0f32, z=0f32 | null | out('vector deconstructed') }         << a struct/closure with a pointless deconstructor  
main::{ stk pt=v3f; pt.x=1; pt.y=1; pt.z=1; pt ~= null; 0; }                << make a v3f on the stack, set all its values to 1, set its destructor to null, and finally return 0 
v2f::{x=0f32, y=0f32; _x,_y | stk r=v2f; r.x=_x; r.y=_y; r; | null }        << a struct/closure with its function acting as a constructor and no destructor 
main::{ stk pt=v2f(1,1); out("pt: " <- pt.x <- " " <- pt.y <- "\n"); 0; }   << a v2f on the stack using its function as a constructor, then output the value using the <- concatenation operator

<< stk is value based and runs the destructor on scope exit if 1. the destructor has not already ran, and 2. the mv operator has not been applied
<< instead of mv operator use <= ? use  <=>  for creating another shared reference? stk is always a move unless & is used in front to denote a reference or @ is used to denote a copy?
<< glbl is a pointer to the global heap and uses atomic reference counted memory - because it is only a pointer it is always just a copy? copying between threads increments the atomic reference, but copying within threads increments a separate, non-atomic reference count?
<< thrd is a pointer to a thread local heap and uses non-atomic reference counted memory



<commit_msg>more brainstorms<commit_after>
<< basic idea is to compile to be able to compile to C >>
<< tables are a combination of vector of type A, hashmap/unsorted_map/dictionary of type B, and a linked list of boxed variants? >>
<< tables are the only data structure not copied by value and are a single ui64 address pointing to contiguous memory which contains the vecotr and hashmap portion >>
<< tables contain sizebytes, size, and an ownership flag so have at least 24 bytes over overhead - ownership flag is on the pointer's highest bit? >>  
<< <> is a compile time expression, <<>> is a line independant comment >>
<< should be able to call C nativly >>
<< compile time expressions are used to express any meta programming or function overloading >>
<< compile time expressions take strings and a context and return strings >>
<< CTEs have compiler functions exposed to them and can parse the incoming string into an ast >>
<< variables at compile time are a dictionary containing their symbol string and any other key value pairs given to them (such as type) >> 
<< all compilation must be able to be performed by compiling the file with the entry point. there must be exactly one global expression with the key 'entry' mapped to 'true' ? >>

<mod 'io' 'str' 'net'>                 << include io string and network modules using the compile time function mod >>

mainCRTStartup :: entry:t = {          << make the entry point for the executable _CRT_main >> 
  io_out('hello world');               << ';' marks the end of an expression if it is not contained by {} >>
  0;                                   << last value is the return value if 'ret' keyword is not used     >>
};                                     << } is the end of an expression, but the whole expression is the assignment to mainCRTStartup and so needs to end with ';' >> 

{cp mainCRTStartup::entry:t {io_out('hello world'); 0;} }  << alternate syntax without using any infix operators >>


{x les y|io_out('x is less than y')}   << branching with basic conditional execution >>
{x les y | t | f}                      << branching >>
{x les y
| 'less'
| {x eq y|'equal'|'more'}}             << if elseif else >>

std = { add={x,y| x + y} };            << effectively a namespace since the expression takes no arguments >>
{io_out {std_add 1 2}}                 << prints 3 >>
std_add(1,2) :> io_out;                << prints 3 >>
{|io_out('wat')} <: {x les y};         << expression conditionally evaluated - passing a boolean to a lazy expression dictates if it is evaluated  >>
imgbuf!{x|x/2}                         << divides each element in the table 'imgbuf' by 2  >>
loop { io_out('hello world\n'); }      << prints 'hello world' lines forever >>
loop io_out('hello world\n');          << does the same since loop is a unary operator >>
<to 10 {x|io_out('line ',x,'\n')}>     << prints 10 lines using the compile time function 'to' >>

sizeof   operator giving the size in bytes of a type - same as C
addr     operator giving the address of a variable as a ui64 - like & in C
deref    operator giving the value at an ui64 address - like * in C
mv       operator that returns a table but sets the old table address to have the ownership flag set to false

{x=0,y=0,z=0;|}                                                   << closures and structs are the same thing and have 8 bytes of overhead for the attached function address - this closure's (or struct's) function would have its address as 0 and return 0 >>
a_vec = {x=0.0f,y=0.0f,z=0.0f; | {x*x + y*y + z*z} :> sqrt}       << closure that returns length - this function would be ran without arguments >>


----------------
Older ideas
----------------
Variables have any number of :metadata symbols after them

<>    means compile time expression - use this to declare cdecl expressions ? 
{}    means expression  - {|} means closure / delayed expression 
[]    means data declaration
()    means alternate syntax for argument passing
=     copies data and declaration?
<-    moves  data or copies a declaration
!     means mutate?
.     looks up from the struct part of a table if used as a value - tbl.{expr}
.     becomes a binary operator to make the left side be the first value to a function when used as a function a.b becomes {b a}
->    looks up from the map part of a table
~     means next in linked list - pass an expression to loop over the whole list
&     means reference / pointer
@     means copy
$     means atomic reference counted shared pointer
:>    means forward to first parameter of next function
:>>   means forward to first parameter of function after next
::>>  means forward to second parameter of function after next
,     means separation of data
;     means separation of expressions
|     means separation of sections in a data declaration or expression
[]    means look up from the array of a table while [|] means data declaration?

:     means compile time symbol? - can <> be used to return compile time tables instead?
``    means external function declaration
:>    means move ?
<=    copies data and a declaration ? not needed?

< [| compile_time_symbol="wat", compile_time_flag=true |] >        // a compile time table instead of less flexible :symbol declarations
<cdecl "i32" "func_name" "f32" "ui32" "ui64">                      // ui64 is used where a 64 bit pointer would be 

#img ||>> {flip img} |> {a,b|{over a b}}

{flip img} :>> {blur img} ::> {over} :> flipOverBlur
  -OR-
flipOverBlur = {over {flip img} {blur img}}
  -OR-
blurred = blur(img);
flipped = flip(img);
flipOverBlur = over(flipped, blurred);

{LoadImg file} |> img |> {blur} |> {sub _ img} |> {abs} |> {luminance _ AVG} |> {ChanImg} |> edges
{LoadImg file} :> ::>> {blur} :> {sub} :> {abs} :> {luminance _ AVG} :> {ChanImg} :> edges





<mod 'io' 'str' 'net'>            << include io string and network modules >>
{x les y | t | f}                 << branching >>
{x les y
| 'less'
| {x eq y|'equal'|'more'}}        << if elseif else >>

<< all branching expressions must be the same type >>
{ x les y -> 'less'               << same as above >>
| x eq  y -> 'equal'              << passing a boolean into an expression that takes no arguments selectivly executes the expression >>
| 'more'}                         << an expression executes segments until one does not return null, or false >> 

<< all branching expressions must be the same type >>
  x les y -> 'less'               << same as above >>
| x eq  y -> 'equal'              << passing a boolean into an expression that takes no arguments selectivly executes the expression >>
| 'more';                         << an expression executes segments until one does not return null, or false >> 

std = { add={x,y| x + y} };       << effectively a namespace since the expression takes no arguments >>
{io_out {std_add 1 2}}            << prints 3 >>
std_add(1,2) -> io_out;           << prints 3 - equivilent to io_out(std_add(1,2)) >>
{|io_out('wat')} if x les y;      << if operator combines an expression and another expression that evaluates to a boolean >>

<< take an idea from klong an use label::{expression} for declarations and/or anything that will go into the binary? >>
main::{ "hello world" -> out }
add::{x,y|x+y}
std::{ add::{x,y|x+y} sub::{x,y|x-y} }           << expression without arguments attached to a label acts as a namespace, but just uses '_' as a delimiter >>
addsub::{x,y| std_add(x,y) -> std_sub(x,y) }     << last expression doesn't need ';' to end >>
dataseg::(10; {x|x*x})            << creates a vector of length 10 of the the squares of 0..9 embedded in the initialized data segment of the .exe >> 
SQR::<x|x*x>                      << SQR(8f32)  evaluates to 64f32 at compile time >>

<| structs, closures, and deconstructors are all together >
<| a lazy compile time function doesn't exist because a normal expression could be used, so instead it is a comment >
v3f::{x=0f32, y=0f32, z=0f32 | null | out('vector deconstructed') }         << a struct/closure with a pointless deconstructor  
main::{ stk pt=v3f; pt.x=1; pt.y=1; pt.z=1; pt ~= null; 0; }                << make a v3f on the stack, set all its values to 1, set its destructor to null, and finally return 0 
v2f::{x=0f32, y=0f32; _x,_y | stk r=v2f; r.x=_x; r.y=_y; r; | null }        << a struct/closure with its function acting as a constructor and no destructor 
main::{ stk pt=v2f(1,1); out("pt: " <- pt.x <- " " <- pt.y <- "\n"); 0; }   << a v2f on the stack using its function as a constructor, then output the value using the <- concatenation operator

<< stk is value based and runs the destructor on scope exit if 1. the destructor has not already ran, and 2. the mv operator has not been applied
<< instead of mv operator use <= ? use  <=>  for creating another shared reference? stk is always a move unless & is used in front to denote a reference or @ is used to denote a copy?
<< glbl is a pointer to the global heap and uses atomic reference counted memory - because it is only a pointer it is always just a copy? copying between threads increments the atomic reference, but copying within threads increments a separate, non-atomic reference count?
<< thrd is a pointer to a thread local heap and uses non-atomic reference counted memory? this would mean that the tbl structure would always be reference counted if not on the stack? is this a problem if the reference count is not atomic and the increment and decrement can be skipped with mv ?
<< = always copies numbers and structs but moves tbls? functions are pointers to memory maps and never deallocated, so they are copied as well
<< reference counting has a thread local reference, an atomic reference count for multiple cores, and a numa reference count for multiple nodes? 


<< code completion in an ide could be done in reverse, with arguments, then functions that would be valid with those arguments
<< tables can have vectors of structs by declaring the struct as the vector type?
{'fraction_non_alphanumeric': 0.10092602226615337, 'fraction_numerical': 0.010404744563520965, 'mean_word_length': 2.9512846865364852, 'pattern_counts': {'":': 0, '<': 289, '<?xml version=': 0, '>': 307, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 14, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6313863', 'n_tokens_mistral': 5445, 'n_tokens_neox': 5200, 'n_words': 2643}