<commit_before># Table Maker

Table maker is an extremely simple compiler that reads in a database spec file 
containing a list of tables, then generates output that can be used to create 
those tables in MySQL.

It is not yet intended to provide full support for all features available in 
MySQL but is certainly enough for most projects. It can be automated as part of
a build process with a simple Makefile.

All tables by default will use the utf8 character set, the MyISAM engine, and
output will include `DROP IF EXISTS` statements for each table.


## Build Script

The build.php script compiles the specified filename and sends the resulting
SQL to STDOUT. Building the accompanying sample spec file into compiled MySQL
file can be performed with the following command:

    $ ./build.php sample.in > sample.sql


## Spec File Format

Spec files are plain text files that, at minimum, contain a table name on its
own line, followed by a series of consecutive tab-indented lines each 
representing column data types and names. Each column line is written as the
data type shorthand preceded by optional modifier characters (see lists below), 
followed by a colon, and the name of the field. No spaces should separate each
element of the field spec line, but spaces can be used in the column name and
will be converted to underscores during compilation.


### Column Data Types (Shorthand)
The following shorthand data types should be used in the spec file.

    Shorthand     MySQL Data Type    Default Modifiers
    -----------   ----------------   -----------------
    bi            bigint             Unsigned, non-nullable
    ti            tinyint            Unsigned, non-nullable
    i             int                Unsigned, non-nullable
    d             decimal            Unsigned, non-nullable
    c             char               Non-nullable
    v             varchar            Non-nullable
    t             text               Non-nullable
    dt            datetime           Non-nullable


### Column Modifiers

You may optionally use these column modifiers before a to specify various MySQL 
column options.

    Shorthand      MySQL Behavior
    ------------   --------------
    *              Sets column as the primary key
    +              Enables auto-increment for the column
    -              Turns integer type into signed (all are unsigned by default)
    ^              Allows NULL values (all are non-nullable by default)
    x              Sets column length to x (accepts any integer)
    x.y            Sets decimal column length to allow for x digits before 
                   decimal point, with y decimal places after
    (abc)          Sets the default value for the column to 'abc'


### Macros

Table Maker supports the use of macros to define data types and modifiers that
are used repeatedly throughout the spec file. This allows for easier maintenance
later when one needs to expand the length of columns containing certain kinds
of data (emails, names, monetary amounts, etc).

Macros follow the syntax `#define x y`, where x is the alias you wish to use,
and y is the data type and modifier shorthands `x` should map to. Macros should 
be defined at the top of the spec file thusly:

    #define id *+bi
    #define hash c32
    #define name v30
    #define email v50

They can then be used as the entire data type name of a column, mixed with other 
non-macro types within the same table and modifiers within the same line.



## Examples
A simple database with a `user` table might have one such spec file:

    #define id *+bi
    #define hash c32
    #define email v50
    #define credit bi
    
    user
    	id:user id
    	email:email
    	hash:password
    	v30:first name
    	v30:last name
    	-credit:credits
    	-d5.4:average
    	dt:create date

When compiled by the `build.php` script, the resulting SQL is generated:

    DROP TABLE IF EXISTS `user`;
    CREATE TABLE `user` (
    	`user_id` bigint unsigned NOT NULL AUTO_INCREMENT,
    	`email` varchar(50) NOT NULL,
    	`password` char(32) NOT NULL,
    	`first_name` varchar(30) NOT NULL,
    	`last_name` varchar(30) NOT NULL,
    	`credits` bigint NOT NULL,
    	`average` decimal(9,4) NOT NULL,
    	`create_date` datetime NOT NULL,
    	PRIMARY KEY (`user_id`)
    ) ENGINE=MyISAM DEFAULT CHARSET=utf8;

## Other Notes
<commit_msg>Add example use of default value<commit_after># Table Maker

Table maker is an extremely simple compiler that reads in a database spec file 
containing a list of tables, then generates output that can be used to create 
those tables in MySQL.

It is not yet intended to provide full support for all features available in 
MySQL but is certainly enough for most projects. It can be automated as part of
a build process with a simple Makefile.

All tables by default will use the utf8 character set, the MyISAM engine, and
output will include `DROP IF EXISTS` statements for each table.


## Build Script

The build.php script compiles the specified filename and sends the resulting
SQL to STDOUT. Building the accompanying sample spec file into compiled MySQL
file can be performed with the following command:

    $ ./build.php sample.in > sample.sql


## Spec File Format

Spec files are plain text files that, at minimum, contain a table name on its
own line, followed by a series of consecutive tab-indented lines each 
representing column data types and names. Each column line is written as the
data type shorthand preceded by optional modifier characters (see lists below), 
followed by a colon, and the name of the field. No spaces should separate each
element of the field spec line, but spaces can be used in the column name and
will be converted to underscores during compilation.


### Column Data Types (Shorthand)
The following shorthand data types should be used in the spec file.

    Shorthand     MySQL Data Type    Default Modifiers
    -----------   ----------------   -----------------
    bi            bigint             Unsigned, non-nullable
    ti            tinyint            Unsigned, non-nullable
    i             int                Unsigned, non-nullable
    d             decimal            Unsigned, non-nullable
    c             char               Non-nullable
    v             varchar            Non-nullable
    t             text               Non-nullable
    dt            datetime           Non-nullable


### Column Modifiers

You may optionally use these column modifiers before a to specify various MySQL 
column options.

    Shorthand      MySQL Behavior
    ------------   --------------
    *              Sets column as the primary key
    +              Enables auto-increment for the column
    -              Turns integer type into signed (all are unsigned by default)
    ^              Allows NULL values (all are non-nullable by default)
    x              Sets column length to x (accepts any integer)
    x.y            Sets decimal column length to allow for x digits before 
                   decimal point, with y decimal places after
    (abc)          Sets the default value for the column to 'abc'


### Macros

Table Maker supports the use of macros to define data types and modifiers that
are used repeatedly throughout the spec file. This allows for easier maintenance
later when one needs to expand the length of columns containing certain kinds
of data (emails, names, monetary amounts, etc).

Macros follow the syntax `#define x y`, where x is the alias you wish to use,
and y is the data type and modifier shorthands `x` should map to. Macros should 
be defined at the top of the spec file thusly:

    #define id *+bi
    #define hash c32
    #define name v30
    #define email v50

They can then be used as the entire data type name of a column, mixed with other 
non-macro types within the same table and modifiers within the same line.



## Examples
A simple database with a `user` table might have one such spec file:

    #define id *+bi
    #define hash c32
    #define email v50
    #define credit bi
    
    user
    	id:user id
    	email:email
    	hash:password
    	v30:first name
    	v30:last name
    	-credit(50):credits
    	-d5.4:average
    	dt:create date

When compiled by the `build.php` script, the resulting SQL is generated:

    DROP TABLE IF EXISTS `user`;
    CREATE TABLE `user` (
    	`user_id` bigint unsigned NOT NULL AUTO_INCREMENT,
    	`email` varchar(50) NOT NULL,
    	`password` char(32) NOT NULL,
    	`first_name` varchar(30) NOT NULL,
    	`last_name` varchar(30) NOT NULL,
    	`credits` bigint NOT NULL DEFAULT '50',
    	`average` decimal(9,4) NOT NULL,
    	`create_date` datetime NOT NULL,
    	PRIMARY KEY (`user_id`)
    ) ENGINE=MyISAM DEFAULT CHARSET=utf8;

## Other Notes

{'fraction_non_alphanumeric': 0.05866240584341475, 'fraction_numerical': 0.006847751654873316, 'mean_word_length': 2.399146625290923, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5782247', 'n_tokens_mistral': 2331, 'n_tokens_neox': 2230, 'n_words': 1181}