<jupyter_start><jupyter_text>We are developing here a package that tries to go to deeper healpix levels when the density of sources is higher.First goal: get a bijective funtion from f(N_1,N_2)  N_3 where N_1 = healpix_level, N_2 = healpix_number, N_3 = cumulative_healpix_number, done (see cumhpx)Second goal: a function that assigns these cumulative healpixels based on the density, e.g. via a pixel threshold, done (see assign_cumhpx)Third goal: transfer this cumhpx structure to other catalogues. Done (see transfer_cumhpx_structure)Fourth goal: Do statistics on the sources from the same cumhpx. Done (see hess_likelihood_per_cumhpx)Fifth goal: Plot the results For comparison we download gdr2 samples and gdr2mock samples.GDR2 ESA random index max = 1692919134GDR2 ESA total count = 1692919135GDR2 ESA total count for stars up to G=20.7 = 1451448033GDR2mock random index max = 1606747035Download sample via:SELECT *FROM gaiadr2.gaia_sourceWHERE phot_g_mean_mag < 20.7AND random_index < 4232298  (total stars/400)saved to output/GDR2_207/GDR2_207_0.0025sampling.fitsAND GDR2mockSELECT *FROM gdr2mock.mainWHERE random_index < 4016868saved to output/GDR2mock_207/GDR2mock_207_0.0025sampling.fits<jupyter_code># Load the data and downsample
new = fits.getdata("../output/mock_cat_old_gdr3_isochrones_00025_sampling/GDR2mock_20.7Gmag.fits")
data = fits.getdata("../output/GDR2_207/GDR2_207_cleaned_0.0025sampling.fits")
old = fits.getdata("../output/GDR2mock_207/GDR2mock_207_0.0025sampling.fits")
print(len(new),len(old), len(data))
downsampling = 1
new = new[::downsampling]
data = data[::downsampling]
old = old[::downsampling]
print(len(new),len(old), len(data))
start = time.time()

data, cumhpx_like_new = assess_likelihood(data,new, threshold = 10000, cmd_bins = 5)
data, cumhpx_like_old = assess_likelihood(data,old, threshold = 10000, cmd_bins = 5)
t1 = time.time()
print('likelihood: ', sum(cumhpx_like_new),'new',sum(cumhpx_like_old), 'old')
hpx_like, hpx_dens = get_hpx_structure_for_plotting(cumhpx_like_new,data)
hpx_like_old, hpx_dens_old = get_hpx_structure_for_plotting(cumhpx_like_old,data)
t2 = time.time()
# looks good. now plotting
print('assess_likelihood took %.1f seconds' %((t1-start)/2))
print('get_hpx_structure_for_plotting took %.1f seconds' %((t2-t1)/2))

plot_mollweide_log(-1*hpx_like,'-lnlike')
plot_mollweide_log(-1*hpx_like_old,'-lnlike')
# Should be the same
plot_mollweide_log(hpx_dens, 'starcounts')
"""
## Nested healpix resolution of GAIA
healpix_resolution = 4096
#phi = x['glon'] * np.pi/180.
#theta = (90.- x['glat']) * np.pi/180.
number = ang2pix(healpix_resolution,x['ra'],x['dec'], nest = True, lonlat = True)
number = number.view(np.int64)
number *= 34359738368
x = append_fields(x,'source_id',number,usemask = False)
print('calculated healpix')
"""<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11284046692607004, 'fraction_numerical': 0.06720905553590378, 'mean_word_length': 6.921568627450981, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '260983', 'license': 'MIT', 'n_tokens_mistral': 1181, 'n_tokens_neox': 1034, 'n_words': 241, 'path': 'notebook/notebook_aida/[3]Comparing Catalogs CMDs per healpix.ipynb', 'repo_name': 'jan-rybizki/Galaxia_wrap', 'size': '2827'}