<commit_before># cheatsheets
Its tough remembering all the codes one wrote, especially if one have not use it for awhile. This is a collection of cheat codes I compiled to assist in generating faster scripts. They are updated whenever I encounter any lines of code which I forsee to be used again in the future. 

  * __Avro__: A serialised file format based on json `avro.md`
  * __Command Line__: Bash & Command Line `cmd.md`
  * __D3.js__: Intro to D3, Dynamic Driven Documents
  * __Docker__: Docker container commands `docker.md`
  * __Git__: Git & GitHub/GitLab `git.md`
  * __HDFS__: Hadoop File System & Impala commands `hdfs.md`
  * __Kafka__: Kafka, a distributed streaming platform `kafka.md`
  * __MongoDB__: A popular NoSQL database based on json format `mongodb.md`
  * __Networkx__: Python's Networkx package `network_basics.ipynb`
  * __Numpy__: python numpy package `numpy.py`
  * __PostgreSQL__: sql samples for postgresql `postgres.sql`
  * __PostGIS__: GIS database in PostgreSQL `postgis.rst`
  * __Sphinx__: Documentation creation using sphinx `sphinx.md`
  * __SQL__: Some basic SQL queries `sql_basic.sql`
  * __Social Media__: Folder containing python codes for various social media APIs, including Slack, Instagram, Facebook, Twitter
  * __Spark__: Installation of Spark in Linux
  * __Tableau__: Sql queries for Tableau `tableau.sql`
  * __Web Development__: Folder containing web development languages cheatsheets like html, css & php
  * __Web Scraping__: Web-scraping using beautiful soup and selenium `webscraping.py`
  * __Python__
    * __Matplotlib__: Python's matplotlib package `matplotlib.py`
    * __OS__: python os package and file handling `os.py`
    * __Pandas__: python pandas package `pandas.py`
    * __Parallel Processing__: parallel processing using python `parallel_python.py`
    * __Python Installation__: tips on python installation `py_install.md`
    * __PySpark__: pyspark package `pyspark.py`
    * __Regular Expression__: python re package `re.py`
    * __Seaborn__: Seaborn package `seaborn.py`<commit_msg>updated new organisation of cheatsheets<commit_after># cheatsheets
Its tough remembering all the codes one wrote, especially if one have not use it for awhile. This is a collection of cheat codes I compiled to assist in generating faster scripts. They are updated whenever I encounter any lines of code which I forsee to be used again in the future. 

  * __Avro__: A serialised file format based on json `avro.md`
  * __Command Line__: Bash & Command Line `cmd.md`
  * __D3.js__: Intro to D3, Dynamic Driven Documents
  * __Docker__: Docker container commands `docker.md`
  * __Git__: Git & GitHub/GitLab `git.md`
  * __HDFS__: Hadoop File System & Impala commands `hdfs.md`
  * __Kafka__: Kafka, a distributed streaming platform `kafka.md`
  * __MongoDB__: A popular NoSQL database based on json format `mongodb.md`
  * __Networkx__: Python's Networkx package `network_basics.ipynb`
  * __PostgreSQL__: sql samples for postgresql `postgres.sql`
  * __PostGIS__: GIS database in PostgreSQL `postgis.rst`
  * __Sphinx__: Documentation creation using sphinx `sphinx.md`
  * __SQL__: Some basic SQL queries `sql_basic.sql`
  * __Social Media__: Folder containing python codes for various social media APIs, including Slack, Instagram, Facebook, Twitter
  * __Spark__: Installation of Spark in Linux
  * __Tableau__: Sql queries for Tableau `tableau.sql`
  * __Web Development__: Folder containing web development languages cheatsheets like html, css & php
  * __Python__
    * __Matplotlib__: Python's matplotlib package `matplotlib.py`
    * __Numpy__: python numpy package `numpy.py`
    * __OS__: python os package and file handling `os.py`
    * __Pandas__: python pandas package `pandas.py`
    * __Parallel Processing__: parallel processing using python `parallel_python.py`
    * __Python Installation__: tips on python installation `py_install.md`
    * __PySpark__: pyspark package `pyspark.py`
    * __Regular Expression__: python re package `re.py`
    * __Seaborn__: Seaborn package `seaborn.py`
    * __Web Scraping__: Web-scraping using beautiful soup and selenium `webscraping.py`
{'fraction_non_alphanumeric': 0.0712223291626564, 'fraction_numerical': 0.0009624639076034649, 'mean_word_length': 4.505960264900662, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2574083', 'n_tokens_mistral': 1328, 'n_tokens_neox': 1260, 'n_words': 481}