<commit_before># Cypher to SQL Translator Tool - _Reagan_

>_version 4.0_

_Reagan_ v4.0 is a tool that will perform automatic translation of both 
graph schemas and graph queries in Neo4J and Cypher respectively, 
to a relational schema and relational query language (SQL).
This is an individual project for the Part II Tripos of 
Computer Science at the University of Cambridge. 

## Modules
### Schema Converter
- Takes as input a dump from the Neo4J shell of an existing graph database.
- Dump needs to be obtained manually by the user.
- The current schema translator converts the dump to two relations at a minimum: nodes and edges. The translator will attempt to optimise, if possible, by creating smaller relations with fewer NULLs. A separate metadata file is also created containing all the possible labels of the node store.
 
### Cypher parsing unit
- Built using ANTLRv4 tool.
- Grammar: https://s3.amazonaws.com/artifacts.opencypher.org/Cypher.g4
- Produces a token list which can be parsed more in the Cyp2SQL tool.

### Query translator unit
- Maps the Cypher query input to an internal representation in Java.
- This is then used to build up the SQL piece by piece.

### Output Module
- Fills in two text files - one is the results from Neo4J based on the Cypher input; the other is the results from Postgres based on the SQL generated from the Cypher input.
- Program outputs true/false depending on whether or not the files match.
- NOTE: this is not wholly accurate due to the following issues: encoding differences, differences in ways that sorting occurs, treatment of NULL.
- Thus, the files also contain an indicator at the bottom of the file as to how many records were returned. This is a quick way of checking that the translator was successful.
- The times of execution of both databases is also displayed by the tool.<commit_msg>Temp change to README - major changes to come Sept 2017!<commit_after># Cypher to SQL Translator Tool - _Reagan_

>_version 4.0_

>Please note that this tool is currently being redeveloped! The current state of the tool is not stable - major changes will be published by September 2017. 

_Reagan_ v4.0 is a tool that will perform automatic translation of both 
graph schemas and graph queries in Neo4J and Cypher respectively, 
to a relational schema and relational query language (SQL).
This is an individual project for the Part II Tripos of 
Computer Science at the University of Cambridge. 

## Modules
### Schema Converter
- Takes as input a dump from the Neo4J shell of an existing graph database.
- Dump needs to be obtained manually by the user.
- The current schema translator converts the dump to two relations at a minimum: nodes and edges. The translator will attempt to optimise, if possible, by creating smaller relations with fewer NULLs. A separate metadata file is also created containing all the possible labels of the node store.
 
### Cypher parsing unit
- Built using ANTLRv4 tool.
- Grammar: https://s3.amazonaws.com/artifacts.opencypher.org/Cypher.g4
- Produces a token list which can be parsed more in the Cyp2SQL tool.

### Query translator unit
- Maps the Cypher query input to an internal representation in Java.
- This is then used to build up the SQL piece by piece.

### Output Module
- Fills in two text files - one is the results from Neo4J based on the Cypher input; the other is the results from Postgres based on the SQL generated from the Cypher input.
- Program outputs true/false depending on whether or not the files match.
- NOTE: this is not wholly accurate due to the following issues: encoding differences, differences in ways that sorting occurs, treatment of NULL.
- Thus, the files also contain an indicator at the bottom of the file as to how many records were returned. This is a quick way of checking that the translator was successful.
- The times of execution of both databases is also displayed by the tool.
{'fraction_non_alphanumeric': 0.0408371618172537, 'fraction_numerical': 0.007656967840735069, 'mean_word_length': 4.069857697283312, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 6, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6755079', 'n_tokens_mistral': 1013, 'n_tokens_neox': 948, 'n_words': 616}