<jupyter_start><jupyter_text>Boilerplate<jupyter_code>import copy
import os

import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf
import tensorflow_hub as hub

import spiral.agents.default as default_agent
import spiral.agents.utils as agent_utils
import spiral.environments.libmypaint as libmypaint


nest = tf.contrib.framework.nest

# The path to libmypaint brushes.
BRUSHES_BASEDIR = os.path.join(os.getcwd(), "..", "third_party/mypaint-brushes-1.3.0")
BRUSHES_BASEDIR = os.path.abspath(BRUSHES_BASEDIR)
# The path to a TF-Hub module.
MODULE_PATH = "https://tfhub.dev/deepmind/spiral/default-wgangp-celebahq64-gen-19steps/agent4/1"

# Disable TensorFlow debug output.
tf.logging.set_verbosity(tf.logging.ERROR)<jupyter_output><empty_output><jupyter_text>Sample from the modelThis section demonstrates the most basic usage of the package, i.e., sampling from a pre-trained model.  Create the environmentFirst, we need to create the environment.<jupyter_code>env_settings = dict(
    episode_length=20,                 # Number of frames in each episode.
    canvas_width=64,                   # The width of the canvas in pixels.
    grid_width=32,                     # The width of the action grid.
    brush_type="classic/dry_brush",    # The type of the brush.
    brush_sizes=[1, 2, 4, 8, 12, 24],  # The sizes of the brush to use.
    use_color=True,                    # Color or black & white output?
    use_pressure=True,                 # Use pressure parameter of the brush?
    use_alpha=False,                   # Drop or keep the alpha channel of the canvas?
    background="white",                # Background could either be "white" or "transparent".
    brushes_basedir=BRUSHES_BASEDIR,   # The location of libmypaint brushes.
)
env = libmypaint.LibMyPaint(**env_settings)<jupyter_output><empty_output><jupyter_text>Create the agentWe provide a convenience function `get_module_wrappers` that returns two `python` functions implementing the agent.The first one, `initial_state`, is used to get the initial state of the agent (specifically, the state of the LSTM).The second, `step`, takes an observation from the environment and performs a single agent step.<jupyter_code>initial_state, step = agent_utils.get_module_wrappers(MODULE_PATH)<jupyter_output><empty_output><jupyter_text>Run sampling<jupyter_code>state = initial_state()<jupyter_output><empty_output><jupyter_text>Running the following cell will return a sample from the model. You can execute it multiple times until you get the one that you like. In addition to showing the final state of the canvas we also record all the agent's actions (note `actions` variable) needed to reproduce it.<jupyter_code>noise_sample = np.random.normal(size=(10,)).astype(np.float32)

time_step = env.reset()
actions = []
for t in range(19):
    time_step.observation["noise_sample"] = noise_sample
    action, state = step(time_step.step_type, time_step.observation, state)
    time_step = env.step(action)
    actions.append(action)
    
plt.close("all")
plt.figure(figsize=(5, 5))
plt.imshow(time_step.observation["canvas"], interpolation="nearest")<jupyter_output><empty_output><jupyter_text>Manipulate the sampleLet's now do something the obtained sample. Since we have the corresponding sequence of actions we can re-render the image in higher resolution. To that end, we will need to create one more environment with modified settings.  Create a high-resolution version of the environment<jupyter_code># Let's make the canvas 8x8 times bigger.
SCALE_FACTOR = 8

# Patch the environments setting for higher resolution.
hires_env_settings = copy.deepcopy(env_settings)
hires_env_settings["canvas_width"] *= SCALE_FACTOR
hires_env_settings["brush_sizes"] = [
    s * SCALE_FACTOR for s in hires_env_settings["brush_sizes"]]

env = libmypaint.LibMyPaint(**hires_env_settings)<jupyter_output><empty_output><jupyter_text>Execute pre-recorded actions<jupyter_code>env.reset()
for t in range(19):
    time_step = env.step(actions[t])
    
plt.close("all")
plt.figure(figsize=(5, 5))
plt.imshow(time_step.observation["canvas"], interpolation="nearest")<jupyter_output><empty_output><jupyter_text>Change the thickness of the brush strokesIn addition to changing the resolution of the images, let's in introduce some more subtle structural changes. We could, for example, change the thickness of all the strokes.<jupyter_code>modified_actions = copy.deepcopy(actions)
for action in modified_actions:
    action["size"] = np.array(0, dtype=np.int32)
    action["pressure"] = np.array(2, dtype=np.int32)

env.reset()
for t in range(19):
    time_step = env.step(modified_actions[t])
    
plt.close("all")
plt.figure(figsize=(5, 5))
plt.imshow(time_step.observation["canvas"], interpolation="nearest")<jupyter_output><empty_output><jupyter_text>Change the brush typeFinally, let's re-render the image above using a different brush type.<jupyter_code># Patch the environments setting for a different brush.
pen_env_settings = copy.deepcopy(hires_env_settings)
pen_env_settings["brush_type"] = "classic/pen"

env = libmypaint.LibMyPaint(**pen_env_settings)
env.reset()
for t in range(19):
    time_step = env.step(modified_actions[t])
    
plt.close("all")
plt.figure(figsize=(5, 5))
plt.imshow(time_step.observation["canvas"], interpolation="nearest")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09757462686567164, 'fraction_numerical': 0.009701492537313432, 'mean_word_length': 5.112884834663626, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '222820', 'license': 'Apache-2.0', 'n_tokens_mistral': 1737, 'n_tokens_neox': 1650, 'n_words': 470, 'path': 'notebooks/spiral-demo.ipynb', 'repo_name': 'GitHub30/spiral', 'size': '5360'}