<jupyter_start><jupyter_text>Outliers example 2  example of outlier test for the pairs of values, $y=f(x)$<jupyter_code>%pylab inline
import matplotlib as mpl
mpl.rcParams['lines.linewidth']=2
mpl.rcParams['lines.color']='r'
mpl.rcParams['figure.figsize']=(12,8)
mpl.rcParams['font.size']=14
mpl.rcParams['axes.labelsize']=20
x = array([10.0,20.03,30.01,40.02,50.02,60.01,70.00,80.01])
y = array([1.62,2.04,4.05,2.85,3.84,3.81,4.86,5.02])
plot(x,y,'o')
xlim([0,90])
ylim([1,6])
xlabel('$x$'),ylabel('$y$')

# best fit seems to be linear
p = polyfit(x,y,1)
Y = polyval(p,x)
plot(x,Y,'--')
text(10,5,'Y = %4.3f x + %4.3f' % (p[0],p[1]))<jupyter_output><empty_output><jupyter_text>Test 1/2: Use standard error of the estimate:$$ S_{y,x}=\sqrt{\frac{\sum_{i=1}^{i=n}(y_{i}-Y_{i})^{2}}{\nu}},\,\,\,\nu=n-(m+1)$$and the **standardized residuals**$$ \frac{e_i}{S_{y,x}} = \frac{|Y_i-y_i|}{S_{y,x}}$$From compare it with the value of 2 (very close to the t-distribution for 95%), i.e. if:$$\left|e_{i}/S_{y,x}\right|>2$$then it's an **outlier**<jupyter_code>n = len(x)
m = 1 # number of independent variables
nu = n - (m+1)
Syx = sqrt(sum((Y - y)**2)/nu)
print (Syx)
e = (Y-y)
plot(arange(1,n+1),e/Syx,'go')
plot([0,n+1],[-2,-2],'b-',[0,n+1],[2,2],'r-')
plot([3],e[2]/Syx,'rx',mfc='none',markersize=30)
xlim([0,n+1]),ylim([-3,3])
xlabel('$n$'),ylabel('$e/S_{yx}$')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.2828282828282828, 'fraction_numerical': 0.07864357864357864, 'mean_word_length': 6.6208791208791204, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 10, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '569088', 'license': 'CC0-1.0', 'n_tokens_mistral': 721, 'n_tokens_neox': 663, 'n_words': 101, 'path': 'notebooks/statistics/outliers_example_pairs.ipynb', 'repo_name': 'alexlib/engineering_experiments_measurements_course', 'size': '1386'}