<commit_before>---
layout: post
title: "AWK is a Turing-Complete Language"
tags: [Software, Languages, AWK]
summary: So Let's Learn It
---
{% include JB/setup %}

AWK is a Turing-complete language. So let's dive in.

The approach I'm going to take here it to introduce all the major language features via example. So, no, this isn't a complete language reference. If you want that, look [here](https://www.grymoire.com/Unix/Awk.html).


## Awk vs Mawk vs Gawk

There are various flavors of `awk`.

If you're nearly any flavor of linux, `awk` almost certainly comes installed. Try typing "`man awk`" to see what flavor of awk you have. You probably have MAWK or GAWK (GNU AWK). They have a very similar API and both are fine. I *think* GAWK is somewhat more consistent across versions and platforms.


## Example 1 - Field Separators

For our first example, let's parse a little CSV file. This one shows the radiation dosage from various sources in different units (`rad.csv`):

    source,dose (Sv),dose (BED)
    eating a banana,1.0E-7,1
    arm x-ray,1.0e-6,10
    dental x-ray,5.0e-6,50
    one day natural background,1.0e-5,100
    flight from LA to New york,4.e-5,400
    mammogram,4e-4,4000
    head CT scan,2.0e-3,20000
    chest CT scan,7.0e-3,70000
    max yearly radiation worker dose,5.0e-2,5e5
    lowest yearly dose linked to cancer,0.1,1e6
    usually fatal dase,4.0,4e7

The first column in this file is the source of radiation, let's grab that on the commandline:

    $ awk -F, '{print $1}' rad.csv
    source
    eating a banana
    arm x-ray
    dental x-ray
    one day natural background
    flight from LA to New york
    mammogram
    head CT scan
    chest CT scan
    max yearly radiation worker dose
    lowest yearly dose linked to cancer
    usually fatal dase

The second column in the file is the radiation from this source in the scientific unit Sieverts (Sv). Boring. The third column is the amount of radiation from each source in the far more useful Banana Equivalent Dosages (BED). How many bananas would you need to eat to get that high a dosage? Let's find out!

    $ awk -F "," '{print $1,"\t\t",$3}' rad.csv
    source 		 dose (BED)
    eating a banana 		 1
    arm x-ray 		 10
    dental x-ray 		 50
    one day natural background 		 100
    flight from LA to New york 		 400
    mammogram 		 4000
    head CT scan 		 20000
    chest CT scan 		 70000
    max yearly radiation worker dose 		 5e5
    lowest yearly dose linked to cancer 		 1e6
    usually fatal dase 		 4e7

Okay, a dental x-ray is 50 times as much radiation as eating a banana. That seems safe. A chest CT is 70,000 BEDs; probably don't do that too often.

But you would need to each 40 million bananas to get a lethal dose of radiation. Yeah, I feel pretty safe on that one.


## Example 2 - Arithmetic Expressions

TODO


## Example 3 - Arrays

TODO

## Example 4 - printf

TODO


## Example 5 - Numerical Functions

TODO


## Example 6 - String Functions

TODO
<commit_msg>#104 down the rabbit hole we go<commit_after>---
layout: post
title: "AWK is a Turing-Complete Language"
tags: [Software, Languages, AWK]
summary: So Let's Learn It
---
{% include JB/setup %}

AWK is a Turing-complete language. So let's dive in.

The approach I'm going to take here it to introduce all the major language features via example. So, no, this isn't a complete language reference. If you want that, look [here](https://www.grymoire.com/Unix/Awk.html).


## Awk vs Mawk vs Gawk

There are various flavors of `awk`.

If you're nearly any flavor of linux, `awk` almost certainly comes installed. Try typing "`man awk`" to see what flavor of awk you have. You probably have MAWK or GAWK (GNU AWK). They have a very similar API and both are fine. I *think* GAWK is somewhat more consistent across versions and platforms.


## Example 1 - Field Separators

For our first example, let's parse a little CSV file. This one shows the radiation dosage from various sources in different units (`rad.csv`):

    source,dose (Sv),dose (BED)
    eating a banana,1.0E-7,1
    arm x-ray,1.0e-6,10
    dental x-ray,5.0e-6,50
    one day natural background,1.0e-5,100
    flight from LA to New york,4.e-5,400
    mammogram,4e-4,4000
    head CT scan,2.0e-3,20000
    chest CT scan,7.0e-3,70000
    max yearly radiation worker dose,5.0e-2,5e5
    lowest yearly dose linked to cancer,0.1,1e6
    usually fatal dase,4.0,4e7

The first column in this file is the source of radiation, let's grab that on the commandline:

    $ awk -F, '{print $1}' rad.csv
    source
    eating a banana
    arm x-ray
    dental x-ray
    one day natural background
    flight from LA to New york
    mammogram
    head CT scan
    chest CT scan
    max yearly radiation worker dose
    lowest yearly dose linked to cancer
    usually fatal dase

The second column in the file is the radiation from this source in the scientific unit Sieverts (Sv). Boring. The third column is the amount of radiation from each source in the far more useful Banana Equivalent Dosages (BED). How many bananas would you need to eat to get that high a dosage? Let's find out!

    $ awk -F "," '{print $1,"\t\t",$3}' rad.csv
    source 		 dose (BED)
    eating a banana 		 1
    arm x-ray 		 10
    dental x-ray 		 50
    one day natural background 		 100
    flight from LA to New york 		 400
    mammogram 		 4000
    head CT scan 		 20000
    chest CT scan 		 70000
    max yearly radiation worker dose 		 5e5
    lowest yearly dose linked to cancer 		 1e6
    usually fatal dase 		 4e7

Okay, a dental x-ray is 50 times as much radiation as eating a banana. That seems safe. A chest CT is 70,000 BEDs; probably don't do that too often.

But you would need to each 40 million bananas to get a lethal dose of radiation. Yeah, I feel pretty safe on that one.


## Example 2 - Arithmetic Expressions

Well, how does one Banana Equivalent Dose (BED) relate to one Sievert (Sv)? Let's do a little math:

    $ awk -F, '{print $3 / $2}' rad.csv 
    -nan
    10000000
    10000000
    10000000
    10000000
    10000000
    10000000
    10000000
    10000000
    10000000
    10000000
    10000000

Oh, right, that first row in the file is a header without any numbers in it. Let's skip that:

    $ awk -F, 'NR > 1 {print $3 / $2}' rad.csv 
    10000000
    10000000
    10000000
    10000000
    10000000
    10000000
    10000000
    10000000
    10000000
    10000000
    10000000

Ah, the difference is exactly 10 million. Easy.

Well, AWK can do some basic math. Cool. Let's make a little script (`f2c.awk`):

    BEGIN {
      print(f2c(32))
      print(f2c(72))
      print(f2c(100))
    }

    function f2c(f) {
            return (f - 32.0) / 1.8
    }

Which we can run from the command line super easy:

    $ awk -f f2c.awk 
    0
    22.2222
    37.7778

So, neat. You can write AWK scripts and run them using `awk -f script.awk`. They need a `BEGIN` function, which is the `main` function from most other languages. And, yeah, you can even create your own functions and do some basic math. That's starting to seem more like a real language than just a little commnanline utility.

But this rabbit hole goes a lot deeper.


## Example 3 - Arrays

TODO

## Example 4 - printf

TODO


## Example 5 - Numerical Functions

TODO


## Example 6 - String Functions

TODO

{'fraction_non_alphanumeric': 0.06975789905621665, 'fraction_numerical': 0.05949938448912597, 'mean_word_length': 2.8123044838373303, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 4, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 22, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '890150', 'n_tokens_mistral': 2658, 'n_tokens_neox': 2284, 'n_words': 1076}