<jupyter_start><jupyter_text>OSKAR System noise testingThis is an simple test looking at the expected RMS and mean of the uncorrelated system noiseon auto and cross correlations.In this example $n=10^{6}$ samples are generated and correlated in blocks of $m=100$ with the blocks then averaged before measuring the mean and STD.<jupyter_code>%matplotlib inline
from IPython.display import display
import numpy as np
import matplotlib.pyplot as pp
import time

n = 1e7
m = 1000
s = 15
Xp = (np.random.randn(n/m, m) + 1.j*np.random.randn(n/m, m))*s
Xq = (np.random.randn(n/m, m) + 1.j*np.random.randn(n/m, m))*s

ac = np.sum(Xp*np.conj(Xp),1)/m # Auto-correlation
xc = np.sum(Xp*np.conj(Xq),1)/m # Cross-correlation

print 'Cross-correlation: measured (predicted)'
print '  mean : %.4f%+.4fi (0+0i)' % (np.real(np.mean(xc)), np.imag(np.mean(xc)))
print '  STD  : %.4f (%.4f)' % (np.std(xc), 2*s**2/(m**0.5))
print 'Auto-correlation: measured (predicted)'
print '  mean : %.4f (%.4f)' % (np.real(np.mean(ac)), 2*s**2)
print '  STD  : %.4f (%.4f)' % (np.std(ac), 2*s**2/(m**0.5))<jupyter_output>Cross-correlation: measured (predicted)
  mean : -0.0129-0.0541i (0+0i)
  STD  : 14.3638 (14.2302)
Auto-correlation: measured (predicted)
  mean : 449.8842 (450.0000)
  STD  : 14.2708 (14.2302)
{'fraction_non_alphanumeric': 0.17848791893998442, 'fraction_numerical': 0.06858924395947, 'mean_word_length': 4.372384937238493, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '499937', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 580, 'n_tokens_neox': 519, 'n_words': 133, 'path': 'doc/ipython_notebooks/uncorrelated_system_noise.ipynb', 'repo_name': 'davepallot/OSKAR', 'size': '1283'}