<jupyter_start><jupyter_text>Photo-z posteriors through normalizing flow This notebook shows how to load a normalizing flow from pzflow, wrap it for rail.creation, and draw galaxy samples with redshift posteriors.<jupyter_code>from pzflow.examples import example_flow
from rail.creation import Creator, engines
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.patches as mpl_patches<jupyter_output>Found classifier FZBoost
Found classifier randomPZ
Found classifier simpleNN
Found classifier trainZ<jupyter_text>Let's load the example galaxy redshift flow from pzflow. To see the construction of this flow, look at this pzflow [example notebook](https://github.com/jfcrenshaw/pzflow/blob/main/examples/redshift_example.ipynb). This flow will be wrapped in a `FlowEngine`, which is a wrapper so that the methods of the flow match the methods expected by the `Creator` object<jupyter_code>flow = engines.FlowEngine(example_flow())
creator = Creator(flow)<jupyter_output>WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)<jupyter_text>Now we can draw samples from the creator:<jupyter_code>samples = creator.sample(n_samples=10000, seed=0)
samples<jupyter_output><empty_output><jupyter_text>These samples don't include redshift posteriors. If we want posteriors, we need to set`include_pdf==True`:<jupyter_code>samples_w_pdfs = creator.sample(n_samples=10000, seed=0, include_pdf=True)
samples_w_pdfs<jupyter_output><empty_output><jupyter_text>The last column of the samples are the true redshift posteriors. You can also access the grid over which the posteriors were calculated:<jupyter_code>samples_w_pdfs.attrs['pz_grid']<jupyter_output><empty_output><jupyter_text>This is the default redshift grid, but we can also define our own grid:<jupyter_code>grid = np.arange(0, 2.5, 0.5)
samples_w_pdfs2 = creator.sample(n_samples=10000, seed=0, include_pdf=True, pz_grid=grid)
samples_w_pdfs2
samples_w_pdfs2.attrs['pz_grid']<jupyter_output><empty_output><jupyter_text>Let's plot a few of the pdf's<jupyter_code>fig,axes = plt.subplots(4, 3, figsize=(15,20))

axes = axes.flatten()

for idx, ax in enumerate(axes):
    
    zs = samples_w_pdfs.attrs['pz_grid']
    redshift, u, g, r, i, z, y, pz = samples_w_pdfs.loc[idx]
    ax.plot(zs, pz)
    
    ax.axvline(redshift, c='C3', ls='--', zorder=0)
    
    # create legend with magnitudes
    handles = [mpl_patches.Rectangle((0, 0), 1, 1, fc="white", ec="white", lw=0, alpha=0)] * 6
    labels = []
    band_labels = ['u', 'g', 'r', 'i', 'z', 'y']
    for band,mag in zip(band_labels, [u,g,r,i,z,y]):
        labels.append(f'{band} = {mag:.2f}')
    ax.legend(handles, labels, loc='best', 
              fancybox=True, framealpha=0.7, 
              handlelength=0, handletextpad=0)
    
    ax.set_xlim(0,2)
    ax.set_ylim(0,30)
    ax.set_xlabel("Redshift")
    ax.set_ylabel("$p(z \,|\, ugrizy)$")<jupyter_output><empty_output><jupyter_text>Let's look at the point estimates<jupyter_code>z_map = []
for pz in samples_w_pdfs['pz_pdf']:
    z_map.append(samples_w_pdfs.attrs['pz_grid'][pz.argmax()])
fig,ax = plt.subplots(figsize=(4,4), constrained_layout=True)
ax.scatter(samples_w_pdfs['redshift'], z_map, s=1)
ax.set_xlabel('True redshift')
ax.set_ylabel('$z_\mathrm{MAP}$')
ax.set_xlim(0,2)
ax.set_ylim(0,2)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12217598097502973, 'fraction_numerical': 0.017538644470868014, 'mean_word_length': 5.8394308943089435, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 31, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '371483', 'license': 'MIT', 'n_tokens_mistral': 1285, 'n_tokens_neox': 1226, 'n_words': 289, 'path': 'examples/creation/basic-creation-demo.ipynb', 'repo_name': 'LSSTDESC/RAIL', 'size': '3364'}