<jupyter_start><jupyter_text>Data structure interface and matrix algebra examples<NAME> | 13.11.2020 Core Analysis Goal(s)1. Settle for a data structure that can well represent the playing field at all times of the game.2. Find a suitable data structure for defining the "plots" or "blocks" on whichplayers can manage their land3. Define the interface: what are the dimensions (x, y, z) Key Insight(s)1. Numpy masked arrays (np.ma) are perfectly fine. We should use those.2. A simple numpy function (np.block) may be enough to work with these blocks Explanations1. in python, numpy matrix algebra should always be preferred to hard-coded for-loops2. Python itself can be quite slow, which is why numpy is written in a lower-levellanguages (i think the most prominent one is written in C). If you use the numpy matrix computations, for-loops still need to beused to calculate things, but they will be executed in C, which is MUCH faster.This is the crucial difference. Does not matter much for 80 x 80 matrix,but I would wish to not close the door to being able to the game on a 800 x 800matrix as well! (:<jupyter_code>import os
import numpy as np
import numpy.ma as ma

%load_ext autoreload
%autoreload 2

import matplotlib as mpl
import matplotlib.pyplot as plt
%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import seaborn as sns
sns.set_context("poster")
sns.set(rc={'figure.figsize': (16, 9.)})
sns.set_style("ticks")

import pandas as pd
pd.set_option("display.max_rows", 120)
pd.set_option("display.max_columns", 120)<jupyter_output><empty_output><jupyter_text>Define size of playing field as a function of the number of plots<jupyter_code>n_plots = 4
rows = cols = 80

# check if playing field size and plot number works together
try:
    # modulo must be zero
    assert rows % n_plots == 0
except AssertionError as e:
    print("playing field size and plot number dont work together. field size must be divideable by plot number without rest.")<jupyter_output><empty_output><jupyter_text>Create a dummy "plot" matrix<jupyter_code>plot_length = int(rows/n_plots)
dummy_plot = np.ones(plot_length**2).reshape((plot_length, plot_length))
dummy_plot.shape<jupyter_output><empty_output><jupyter_text>Create a few meaningful block matrixesExample matrix:  1 2A x yB z x<jupyter_code>A1 = dummy_plot
A2 = dummy_plot * 2
B1 = dummy_plot * 3
B2 = dummy_plot * 4<jupyter_output><empty_output><jupyter_text>Create a bigger (40 x 40) matrix from the 4 smaller block matrices<jupyter_code>large_matrix = np.block([[A1, A2], [B1, B2]])
large_matrix
large_matrix.shape<jupyter_output><empty_output><jupyter_text>Works like a charm! We can now distinguish the various plots on our playing fieldby using np.block.Now, let's figure out how to manipulate the blocks separately. For this, I foundsome suggestions here: https://stackoverflow.com/questions/39104999/multiply-each-block-of-a-block-matrix-with-different-coefficient-in-numpy.I will follow those and will try to use numpy's Kronecker Delta function to solve our problem.1. The number of plots is determined by $n_{plots}$, in our case, $n_{plots} := 4$.2. Assuming our playing field is a $(n * n)$ quadratic matrix, we also need to define $n$. For now, let $n := 80$.3. We can define $length_{plot}$ as $\frac{n_{plots}}{n}$, in our case, $length_{plot} = 20$.4. For updating the blocks/plots separately, we need to define a $n_{plots} * n_{plots}$ coefficient matrixand from thereon construct a $(length_{plot} * n) * (length_{plot} * n)$ block coefficient matrix.5. We can then exploit the Kronecker Delta function, and multiply the small, $n_{plots} * n_{plots}$ coefficient matrixwith a $length_{plot} * length_{plot}$ unit matrix (create with np.ones).6. In the end, we use this $(length_{plot} * n) * (length_{plot} * n)$ block coefficient matrixto update our $(n * n)$ playing field matrix.Et voila! Is this understandable?<jupyter_code>large_dummy_matrix = np.ones_like(large_matrix)
large_dummy_matrix.shape
n = plot_length
coef_matrix = np.array([[1, 2], [3, 4]])

result = np.multiply(large_dummy_matrix, np.kron(coef_matrix, np.ones((n,n))))
result<jupyter_output><empty_output><jupyter_text>Again, works like a charm! Let's now put everything together in a neat datastructure tailored to our playing field problem.Unfortunately, mixing characters (A-D) and numbers (1-4) is pretty badfor the coding team for many reasons, so let's please just use numbers(just like in linalg courses) to specify plots/blocks on our map.Both Rows and Columns are characterised by numbers from 1-4 which when combineduniquely identify each block/plot as follows:| - | 1   | 2   | 3   | 4  |--- | --- | --- | --- | ---| 1 | 11  | 12  | 13  | 14 || 2 | 21  | 22  | 23  | 24 || 3 | 31  | 32  | 33  | 34 || 4 | 41  | 42  | 43  | 44 |Hope that is fine - thanks! Use index matrix and value concatenation to dynamically create plot/blockdefinitions based on playing field size and plot number (ideally, we want tostay flexible here).https://stackoverflow.com/questions/36700914/how-to-merge-2-numpy-arrays-and-concatenate-their-values<jupyter_code>n_plots
plot_length
n = rows
# define blocks/plots via unique integers
matrix_indizes = np.indices((n_plots, n_plots), dtype="uint8") + 1
row_indizes, column_indizes = matrix_indizes[0], matrix_indizes[1]
plot_definition_matrix = np.char.add(row_indizes.astype(np.str), column_indizes.astype(np.str)).astype(np.uint8)
plot_definition_matrix<jupyter_output><empty_output><jupyter_text>Now, let's use the small "plot_definition_matrix" to create the definitionsfor the real world, large playing field matrix consisting of many small blockmatrices.<jupyter_code>dummy_playing_field_matrix = np.ones(shape=(rows, cols), dtype=np.uint8)
dummy_playing_field_matrix.shape
large_plot_definition_matrix = np.multiply(
    dummy_playing_field_matrix,
    np.kron(plot_definition_matrix, np.ones(shape=(plot_length, plot_length)))
)

large_plot_definition_matrix
large_plot_definition_matrix.shape<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09563340527976091, 'fraction_numerical': 0.02324423044994189, 'mean_word_length': 5.301255230125523, 'pattern_counts': {'":': 0, '<': 34, '<?xml version=': 0, '>': 34, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '444318', 'license': 'MIT', 'n_tokens_mistral': 2003, 'n_tokens_neox': 1847, 'n_words': 685, 'path': 'notebooks/01-fz-data-structure-interface.ipynb', 'repo_name': 'martinabuck/gecm', 'size': '6033'}