<jupyter_start><jupyter_text>First - test to see if there are any stars that would be selected by the parallax criterion, but not by the proper motion criterion.<jupyter_code>gaia_dir = "/Users/adamamiller/Desktop/PS1_fits/gaia_stars/"
gaia_df = pd.read_hdf(gaia_dir + "parallax_ps1_gaia_mag_pm_plx.h5")
pxl_not_pm = np.where((gaia_df["parallax_over_error"] >= 8) & 
                      (gaia_df["pm_over_error"] < 7.5))
gaia_df.iloc[pxl_not_pm]<jupyter_output><empty_output><jupyter_text>There are ~579k sources that satisfy the pxl cut but not the pm cut. These now need to be matched to the pxl cut stars in PS1 to see which (if any) of these stars need to be adjusted in the ZTF database.<jupyter_code>pxl_ps1 = pd.read_hdf("parallax_ps1_gaia_cat_merge.h5")
pxl_objid = np.array(gaia_df["objid"].iloc[pxl_not_pm])
pxl_ps1_objid = np.array(pxl_ps1.index)
pxl_in_cat = np.isin(pxl_objid, pxl_ps1_objid)
print("There are {} pxl stars that need to be added to the catalog".format(len(pxl_objid[~pxl_in_cat])))
pxl_objid[~pxl_in_cat]
del pxl_ps1
del gaia_df
gc.collect()<jupyter_output><empty_output><jupyter_text>Get the missing pm stars<jupyter_code>gaia_dir = "/Users/adamamiller/Desktop/PS1_fits/gaia_stars/"
pm_df = pd.read_hdf(gaia_dir + "pm_ps1_gaia_mag_pm_plx.h5")
pm_ps1 = pd.read_hdf("pm_ps1_gaia_cat_merge.h5")
pm_objid = np.array(pm_df["objid"])
pm_ps1_objid =np.array(pm_ps1.index)
del pm_df
del pm_ps1
gc.collect()
pm_in_cat = np.isin(pm_objid, pm_ps1_objid)
pxl_and_pm_objid = np.append(pm_objid[~pm_in_cat],
                             pxl_objid[~pxl_in_cat])
new_stars_df = pd.DataFrame(pxl_and_pm_objid, columns=["objid"])
new_stars_df.to_hdf("objid_for_gaia_stars_for_ztf_database.h5", "d1")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11366245694603903, 'fraction_numerical': 0.016647531572904706, 'mean_word_length': 6.028225806451613, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '569773', 'license': 'MIT', 'n_tokens_mistral': 775, 'n_tokens_neox': 752, 'n_words': 132, 'path': 'gaia/pmStarsForZTFdatabase.ipynb', 'repo_name': 'Hallflower20/PS1_star_galaxy', 'size': '1742'}