<jupyter_start><jupyter_text>First, move within a simulation directory that contains a sub-directory 'halo/' that contains rockstar halo catalog files. By default, the code assumes that the raw text halo files that Rockstar produces have been converted to hdf5 format and live in halo/rockstar_dm/catalog_hdf5/ Currently, I run Rockstar halo finder using only dark-matter particles. This means that all halo quantities are computed using **only** dark matter particles. I then assign star particles to the dark-matter halos in post-processing. Ensure that rockstar_analysis and utilities directories are in your python path, then...<jupyter_code>import rockstar_analysis as rockstar
import utilities as ut
# you can access the files as named or use the aliases in __init__.py to keep it shorter 
# for example, these are the same:

rockstar.rockstar_io
rockstar.io<jupyter_output><empty_output><jupyter_text>read halo catalog<jupyter_code># read halo catalog at single snapshot (z = 0)

hal = rockstar.io.IO.read_catalogs('redshift', 0)
# halo properties are stored as dictionary

for k in hal.keys():
    print(k)
# you can read halo catalogs at all available snapshots
# by supplying None or 'all' as the input snapshot value.
# stored as a list of dictionaries, with list index same as snapshot index

hals = rockstar.io.IO.read_catalogs('redshift', None)
# halo catalog at z = 0 (snapshot 600)
hal_z0 = hals[600]

# halo catalog at z = 1 (snapshot 277)
hal_z1 = hals[277]
# by default, if you read multiple snapshots it will store so the halo catalog list index = snapshot index
# but if you just want to read a few snapshots and store as short list, you can do the following

hals = rockstar.io.IO.read_catalogs('redshift', [0, 1], all_snapshot_list=False)

hal_z1 = hals[0]
hal_z0 = hals[1]<jupyter_output># in utilities.simulation.Snapshot():
  reading:  snapshot_times.txt
  input redshift = [0, 1]:  using snapshot indices = [600 277]
  redshifts = [0. 1.]

* read 4878 halos from:  halo_277.hdf5
* read 4878 halos with star particles from:  star_277.hdf5


* read 3484 halos from:  halo_600.hdf5
* read 3484 halos with star particles from:  star_600.hdf5<jupyter_text>halo properties<jupyter_code>hal = rockstar.io.IO.read_catalogs('redshift', 0)
# halo properties are stored as dictionary

for k in hal.keys():
    print(k)
# 3-D position (particle_number x dimension_number array) [kpc comoving]

hal['position']
# 3-D velocity (particle_number x dimension_number array) [km/s physical]

hal['velocity']
# DM mass of halo [M_sun]
# I use 200m, 200 x the mean matter density, for the halo overdensity/virial definition

hal['mass']
# but rockstar also stores total halo mass based on different virial definitions [M_sun]

print('{}\n{}\n{}'.format(hal['mass.200m'], hal['mass.vir'], hal['mass.200c']))
# DM mass that is bound to halo [M_sun]

hal['mass.bound']
# halo radius, again using 200m for the overdentiy/virial definition [kpc physical]

hal['radius']
# NFW scale radius [kpc physical]

hal['scale.radius']
# maximum of the circular velocity profile [km/s physical]

print(hal['vel.circ.max'])
# standard deviation of the velocity (velocity disperion) [km/s physical]

print(hal['vel.std'])
# DM mass of low-resolution DM particles within R_200m [M_sun]

hal['mass.lowres']
# the fraction of DM mass within R_200m that is in low-resolution DM
# this is a derived quantity, so you need to call via the .prop() function

hal.prop('lowres.mass.frac')
# index of the primary host halo in catalog list

hal['host.index']
# 3-D distance to the primary host halo [kpc physical]

hal['host.distance']
# total (scalar) distance to the primary host halo [kpc physical]
# this is a derived quantity, so you need to call via the .prop() function

hal.prop('host.distance.total')
# 3-D velocity wrt the primary host halo [kpc physical]
# radial and tangential velocity wrt the primary host halo [kpc physical]

print(hal['host.velocity'])
print(hal['host.velocity.rad'])
print(hal['host.velocity.tan'])
# use .prop() to compute derived quantities, such the log of the mass
# see rockstar.io.HaloDictionaryClass for all options for derived quantities

hal.prop('log mass')<jupyter_output><empty_output><jupyter_text>halo catalogs with baryonic particle properties I assign baryonic particle properties (such as stars) in post-processing, after generating the DM halo catalogs. I store those properties in a separate file, for example, star_600.hdf5. The above reader automatically looks if this file exists at the snapshot of interest and will read and append these properties to the halo catalog. Not all snapshots have baryonic particle properties.Currently I impose a 15 kpc (physical) max radius to assign a star particle to a halo.<jupyter_code># all stellar properties have dictionary keys as 'star.*' (ignore star.mass.rockstar)
# list of star particle properties
for k in hal:
    if 'star.' in k:
        print(k)
# find halos with star particles

his = ut.array.get_indices(hal['star.mass'], [1, Inf])
# mass of all star particles in halo [M_sun]

hal['star.mass'][his]
# number of star particles in halo [M_sun]

hal['star.number'][his]
# radius that encloses 50%, 90% of the stellar mass [kpc physical]

print(hal['star.radius.50'][his])
print(hal['star.radius.90'][his])
# can compute stellar density (within R_50) as a derived property [M_sun / kpc^3]

hal.prop('star.density.50', his)
# stellar velocity dispersion (standard deviation) at R_50 [km / s]

hal['star.vel.std.50'][his]
# center-of-mass position and velocity of star particles [kpc comoving]

print(hal['star.position'][his])
print(hal['star.velocity'][his])
# time (age of Universe) when galaxy formed 50%, 90%, etc of its current stellar mass [Gyr]

print(hal['star.form.time.50'][his])
print(hal['star.form.time.90'][his])
# can convert this to lookback-time via derived property [Gyr]

hal.prop('star.form.time.lookback.50', his)
# indices of member star particles in the particle list at the same snapshot
# for example, star particles assigned to halo 0 would be part_indices = hal['star.indies'][0]
# then if you read in the particle snapshot, you can access member star particles via 
# part['star'][property_name][part_indices]

hal['star.indices'][his[0]]<jupyter_output><empty_output><jupyter_text>additional information stored in sub-dictionaries<jupyter_code># dictionary of useful information about the simulation

hal.info
# dictionary class of information about the cosmology of the simulation

hal.Cosmology
# dictionary of information about this snapshot's index, scale-factor, redshift, time, lookback-time

hal.snapshot
# dictionary of arrays about *all* snapshots of the simulation

print(hal.Snapshot.keys())
print(hal.Snapshot['redshift'])<jupyter_output>dict_keys(['index', 'scalefactor', 'redshift', 'time', 'time.width'])
[9.90000000e+01 1.90000000e+01 1.50000000e+01 1.46059113e+01
 1.42307692e+01 1.38732395e+01 1.35321102e+01 1.32062778e+01
 1.28947372e+01 1.25965662e+01 1.23109245e+01 1.20370369e+01
 1.17741938e+01 1.15217390e+01 1.12790699e+01 1.10456276e+01
 1.08208952e+01 1.06043959e+01 1.03956833e+01 1.01943464e+01
 1.00000000e+01 9.81967163e+00 9.64516163e+00 9.47619057e+00
 9.31250000e+00 9.15384579e+00 9.00000000e+00 8.84375000e+00
 8.69230747e+00 8.54545403e+00 8.40298557e+00 8.26470566e+00
 8.13043499e+00 8.00000000e+00 7.86153841e+00 7.72727251e+00
 7.59701490e+00 7.47058821e+00 7.34782600e+00 7.22857141e+00
 7.11267614e+00 7.00000000e+00 6.89743567e+00 6.79746819e+00
 6.69999981e+00 6.60493851e+00 6.51219511e+00 6.42168665e+00
 6.33333349e+00 6.24705887e+00 6.16279078e+00 6.08045959e+00
 6.00000000e+00 5.92307711e+00 5.84782600e+00 5.77419329e+00
 5.70212793e+00 5.63157892e+00 5.56250000e+00 5.49484539e+00
 5.42857122e+00[...]<jupyter_text>halo merger trees The Rockstar halo finder treats each snapshot as independent, so halos at different snapshots do not know about each other. ConsistentTrees produces halo merger trees that link halos over time.Note: ConsistentTrees applies some smoothing and physical consistency checks on halos over time, which leads to two important differences from the halo catalog: (1) not every halo in the catalog exists in the merger tree (especially those that are marginally resolved) (2) some halos in the merger tree are 'phantom' halos that have been interpolated across snapshots but do not exist in the halo catalog at that snapshot. Like with the halo catalogs, the reader automatically looks if baryonic particle property files exists at each snapshot and will read and append these properties to the halo merger trees. Again, not all snapshots have baryonic particle properties, so not all halos in the tree will have valid values for these.<jupyter_code># read halo merger trees across all snapshots
# this is a concatenated list of all halos across all snaphots, with pointers to progenitor and descendant halos

halt = rockstar.io.IO.read_tree()
# halo merger tree properties are stored as dictionary

for k in halt.keys():
    print(k)
# now each halo has a corredponding snapshot index

halt['snapshot']
# so, get all halos at snapshot 600 (z = 0) and print their masses

hindices = np.where(halt['snapshot'] == 600)[0]
print(halt['mass'][hindices])
# alternately, get catalog of halos only at snapshot 600 from tree

hal = rockstar.io.IO.get_catalog_from_tree(halt, 600)
hal['mass']
# whether halo is 'phantom' interpolation across snapshots

halt['am.phantom']
# tree index of its descendant at a later (usually the next) snapshot
# ConsistentTrees allows only one descendant per halo

print(halt['descendant.index'])

# for example, get descendant of halo index 100 and print its mass
hindex = 100
desc_index = halt['descendant.index'][hindex]
print(desc_index, halt['mass'][desc_index])
# number of progenitor halos (can be arbitrarily large) at an earler (usually the previous) snapshot

halt['progenitor.number']
# tree index of main (most massive) progenitor
# iteratively loop over this to get list of main progenitors going back in time

halt['progenitor.main.index']
# whether I am the main (most massive) progenitor of my descendant

halt['am.progenitor.main']
# tree index of next co-progenitor

halt['progenitor.co.index']
# tree index of my descendant at the final snapshot (z = 0)

halt['final.index']
# example workflow

# start with a halo with tree index 0 (at z = 0)
hindex = 0
print(halt['snapshot'][hindex])
print(halt['mass'][hindex])

# find its progenitors, list their mass
prog_index = halt['progenitor.main.index'][hindex]
prog_indices = []
while prog_index >= 0:
    prog_indices.append(prog_index)
    prog_index = halt['progenitor.co.index'][prog_index]
print(prog_indices)
print(halt['mass'][prog_indices])

# for the main progenitor, find its progenitors, list their mass
hindex = prog_indices[0]
prog_index = halt['progenitor.main.index'][hindex]
prog_indices = []
while prog_index >= 0:
    prog_indices.append(prog_index)
    prog_index = halt['progenitor.co.index'][prog_index]
print(prog_indices)
print(halt['mass'][prog_indices])
# get list of main progenitors as far back as can go

hindex = 0
prog_main_index = hindex
prog_main_indices = []
while prog_main_index >= 0:
    prog_main_indices.append(prog_main_index)
    prog_main_index = halt['progenitor.main.index'][prog_main_index]

print(prog_main_indices)
print(halt['mass'][prog_main_indices])
print(halt['position'][prog_main_indices])
# halt.prop() computes derived quantities and can make walking the tree easier

# for halo with tree index = 0, get its main progenitors going back as far as can (including self)
hindex = 0
prog_indices = halt.prop('progenitor.main.indices', hindex)

# print mass history
print(prog_indices)
print(halt['mass'][prog_indices])
# print stellar mass history - recall that not all snapshots necessarily have star particle information

print(halt['star.mass'][prog_indices])
# for halo with tree index = 1, get all of its progenitors at previous snapshot, print their masses

hindex = 1
prog_indices = halt.prop('progenitor.indices', hindex)

print(prog_indices)
print(halt['mass'][prog_indices])
# for halo with tree index = 123456, get its descendants going forward as far as can (including self)

hindex = 123456
desc_indices = halt.prop('descendant.indices', hindex)

# print future mass growth
print(desc_indices)
print(halt['mass'][desc_indices])<jupyter_output>[123456 122426 121405 120390 119380 118377 117381 116387 115407 114431
 113460 112498 111541 110591 109647 108713 107784 106860 105945 105038
 104137 103244 102353 101469 100593  99722  98855  97999  97142  96289
  95445  94604  93765  92934  92111  91291  90472  89662  88852  88052
  87259  86470  85688  84911  84139  83372  82611  81859  81110  80367
  79625  78890  78162  77435  76713  76002  75299  74601  73905  73213
  72526  71846  71168  70490  69823  69157  68502  67855  67215  66579
  65946  65319  64698  64080  63467  62860  62264  61669  61076  60496
  59918  59349  58792  58244  57704  57171  56642  56120  55600  55085
  54574  54065  53558  53057  52558  52066  51580  51098  50623  50149
  49679  49212  48751  48291  47835  47384  46934  46487  46039  45597
  45162  44731  44308  43887  43471  43061  42653  42248  41847  41448
  41055  40668  40283  39902  39523  39146  38774  38404  38034  37672
  37310  36949  36590  36234  35883  35535  35190  34853  34518  34181
  3385[...]<jupyter_text>interfacting between halo catalog and merger trees Each halo in the merger trees has a pointers to its array index in the halo catalog: 'catalog.index'Conversely, each halo in the catalog has a pointer to its array index in the halo merger trees: 'tree.index'You can use these to go back and forth between the two.Note: tree ids/indices are unique across all snapshots, but halo catalog ids/indices are unique only at a given snapshot<jupyter_code># read halo merger trees across all snapshots
halt = rockstar.io.IO.read_tree()

# read halo catalog at each snapshot
hals = rockstar.io.IO.read_catalogs(snapshot_values='all')
# starting from the halo catalog at z = 0, get index of the halo in the merger trees
hal = hals[600]

cat_index = 100  # random halo of interest
print(hal['mass'][cat_index])
tree_index = hal['tree.index'][cat_index]
print(tree_index)
print(halt['mass'][tree_index])
# conversely, starting from the halo merger trees, get index of the halo in the snapshot catalog
# because halo catalog indices are *not* unique across snapshots, 
# you also need to use which snapshot index the halo points to

tree_index = 100  # random halo of interest
print(halt['mass'][tree_index])
snapshot_index = halt['snapshot'][tree_index]  # index of snapshot in halo catalog
cat_index = halt['catalog.index'][tree_index]  # halo catalog index at snapshot
print(snapshot_index, cat_index)
print(hals[snapshot_index]['mass'][cat_index])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08985741189131019, 'fraction_numerical': 0.10680656443368308, 'mean_word_length': 5.0888615888615885, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 8, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '473305', 'license': 'Apache-2.0', 'n_tokens_mistral': 5883, 'n_tokens_neox': 4722, 'n_words': 1739, 'path': 'students_final_projects/group-f/rockstar_analysis/rockstar_tutorial.ipynb', 'repo_name': 'mdrafiislam/PH482_582', 'size': '14868'}