<jupyter_start><jupyter_text>Naive Mean-Field approximation[![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/danhtaihoang/network-inference/master?filepath=sphinx%2Fcodesource%2Fnmf.ipynb)In this section, we show the performance of Naive Mean-Field approximation (nMF) to infer couplings $W_{ij}$ from configurations `s`. We will see that nMF method can produce good inference results only in the regimes of large sample sizes and small coupling variability. This method produces bad results in the limit of small sample sizes or large coupling variability.   To start using our jupyter notebook, we first import the nescessary packages.<jupyter_code>import numpy as np
import sys
import matplotlib.pyplot as plt
import simulate
import inference
%matplotlib inline

np.random.seed(1)<jupyter_output><empty_output><jupyter_text>We first use the same parameters as the first example in FEM section above: $N = 100$, $g = 2.0$, and $L=2000$.<jupyter_code># parameter setting:
n = 100
g = 2.0
w0 = np.random.normal(0.0,g/np.sqrt(n),size=(n,n))

l = 2000
s = simulate.generate_data(w0,l)<jupyter_output><empty_output><jupyter_text>Now, we apply nMF method to recover the coupling matrix `w` from variable configurations `s`.<jupyter_code>w = inference.nmf(s)<jupyter_output><empty_output><jupyter_text>We plot the heat map of inferred coupling matrix `w` and compare with the actual couplings `w0`:<jupyter_code>plt.figure(figsize=(11,3.2))

plt.subplot2grid((1,3),(0,0))
plt.title('actual coupling matrix')
plt.imshow(w0,cmap='rainbow',origin='lower')
plt.xlabel('j')
plt.ylabel('i')
plt.clim(-0.5,0.5)
plt.colorbar(fraction=0.045, pad=0.05,ticks=[-0.5,0,0.5])

plt.subplot2grid((1,3),(0,1))
plt.title('predicted coupling matrix')
plt.imshow(w,cmap='rainbow',origin='lower')
plt.xlabel('j')
plt.ylabel('i')
plt.clim(-0.5,0.5)
plt.colorbar(fraction=0.045, pad=0.05,ticks=[-0.5,0,0.5])

plt.subplot2grid((1,3),(0,2))
plt.title('predicted couplings vs. actual couplings')
plt.plot([-1,1],[-1,1],'r--')
plt.scatter(w0,w)
plt.xlabel('actual couplings')
plt.ylabel('inferred couplings')

plt.tight_layout(h_pad=1, w_pad=1.5)
plt.show()<jupyter_output><empty_output><jupyter_text>The mean square error (MSE) between actual couplings and inferred couplings is calculated:<jupyter_code>MSE = np.mean((w-w0)**2)
print('MSE:',MSE)<jupyter_output>('MSE:', 0.016432507950391948)<jupyter_text>The inference result is very poor.Now, if we use a much larger sample size, $L=100000$ for instance.<jupyter_code>l = 100000
s = simulate.generate_data(w0,l)
w = inference.nmf(s)<jupyter_output><empty_output><jupyter_text>The inference result is shown below:<jupyter_code>plt.figure(figsize=(11,3.2))

plt.subplot2grid((1,3),(0,0))
plt.title('actual coupling matrix')
plt.imshow(w0,cmap='rainbow',origin='lower')
plt.xlabel('j')
plt.ylabel('i')
plt.clim(-0.5,0.5)
plt.colorbar(fraction=0.045, pad=0.05,ticks=[-0.5,0,0.5])

plt.subplot2grid((1,3),(0,1))
plt.title('predicted coupling matrix')
plt.imshow(w,cmap='rainbow',origin='lower')
plt.xlabel('j')
plt.ylabel('i')
plt.clim(-0.5,0.5)
plt.colorbar(fraction=0.045, pad=0.05,ticks=[-0.5,0,0.5])

plt.subplot2grid((1,3),(0,2))
plt.title('predicted couplings vs. actual couplings')
plt.plot([-1,1],[-1,1],'r--')
plt.scatter(w0,w)
plt.xlabel('actual couplings')
plt.ylabel('inferred couplings')

plt.tight_layout(h_pad=1, w_pad=1.5)
plt.show()
MSE = np.mean((w-w0)**2)
print('MSE:',MSE)<jupyter_output>('MSE:', 0.016114338928562788)<jupyter_text>The result shows that nMF produces bad prediction results in the limit of large coupling variability, even for large sample sizes. Now, let us consider a very small coupling variability, $g = 0.5$ for instance.<jupyter_code>g = 0.5
w0 = np.random.normal(0.0,g/np.sqrt(n),size=(n,n))<jupyter_output><empty_output><jupyter_text>Data length $L=2000$ is used for our first example.<jupyter_code>l = 2000
s = simulate.generate_data(w0,l)
w = inference.nmf(s)
plt.figure(figsize=(11,3.2))

plt.subplot2grid((1,3),(0,0))
plt.title('actual coupling matrix')
plt.imshow(w0,cmap='rainbow',origin='lower')
plt.xlabel('j')
plt.ylabel('i')
plt.clim(-0.2,0.2)
plt.colorbar(fraction=0.045, pad=0.05,ticks=[-0.2,0,0.2])

plt.subplot2grid((1,3),(0,1))
plt.title('predicted coupling matrix')
plt.imshow(w,cmap='rainbow',origin='lower')
plt.xlabel('j')
plt.ylabel('i')
plt.clim(-0.2,0.2)
plt.colorbar(fraction=0.045, pad=0.05,ticks=[-0.2,0,0.2])

plt.subplot2grid((1,3),(0,2))
plt.title('predicted couplings vs. actual couplings')
plt.plot([-0.2,0.2],[-0.2,0.2],'r--')
plt.scatter(w0,w)
plt.xlabel('actual couplings')
plt.ylabel('inferred couplings')

plt.tight_layout(h_pad=1, w_pad=1.5)
plt.show()
MSE = np.mean((w-w0)**2)
print('MSE:',MSE)<jupyter_output>('MSE:', 0.0005299059905942844)<jupyter_text>Now, let us see if the inference result can be improved with larger data length, $L=100000$ for instance.<jupyter_code>l = 100000
s = simulate.generate_data(w0,l)
w = inference.nmf(s)
plt.figure(figsize=(11,3.2))

plt.subplot2grid((1,3),(0,0))
plt.title('actual coupling matrix')
plt.imshow(w0,cmap='rainbow',origin='lower')
plt.xlabel('j')
plt.ylabel('i')
plt.clim(-0.2,0.2)
plt.colorbar(fraction=0.045, pad=0.05,ticks=[-0.2,0,0.2])

plt.subplot2grid((1,3),(0,1))
plt.title('predicted coupling matrix')
plt.imshow(w,cmap='rainbow',origin='lower')
plt.xlabel('j')
plt.ylabel('i')
plt.clim(-0.2,0.2)
plt.colorbar(fraction=0.045, pad=0.05,ticks=[-0.2,0,0.2])

plt.subplot2grid((1,3),(0,2))
plt.title('predicted couplings vs. actual couplings')
plt.plot([-0.2,0.2],[-0.2,0.2],'r--')
plt.scatter(w0,w)
plt.xlabel('actual couplings')
plt.ylabel('inferred couplings')

plt.tight_layout(h_pad=1, w_pad=1.5)
plt.show()
MSE = np.mean((w-w0)**2)
print('MSE:',MSE)<jupyter_output>('MSE:', 8.53008777227379e-05)
{'fraction_non_alphanumeric': 0.19005544005544006, 'fraction_numerical': 0.0693000693000693, 'mean_word_length': 7.029207232267037, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '294672', 'license': 'MIT', 'n_tokens_mistral': 2679, 'n_tokens_neox': 2363, 'n_words': 416, 'path': 'sphinx/codesource/.ipynb_checkpoints/nmf-checkpoint.ipynb', 'repo_name': 'danhtaihoang/network-inference', 'size': '5772'}