<jupyter_start><jupyter_text>06 Custom Kernels and Mean Functions_[Estimated execution time: 1 min]_In this tutorial we show how custom kernels can be trained with this toolkit as well as how we can train a mean function for our data set. We create an artificial data set, we show how to define a trainable mean function and how to seperate training of the mean function and the kernel.<jupyter_code>import mogptk
import numpy as np<jupyter_output><empty_output><jupyter_text>Let's create a function that is periodical and add a polynomial. In this case we choose a sinusoidal with unit frequency and a second-degree polynomial $2.0x - 0.2x^2$.Remark that loading a `Data` object using `mogptk.LoadFunction` allows us to know the "true" signal to which we add a random variance. This is useful for plotting and also for calculating the "true" prediction error.<jupyter_code>f = lambda x: np.sin(x[:,0]*2.0*np.pi) + 2*x[:,0] - 0.2*x[:,0]**2
data = mogptk.LoadFunction(f, start=0.0, end=10.0, n=100, var=0.5)
data.plot();<jupyter_output><empty_output><jupyter_text>In order to make a trainable mean function where the parameters are automatically picked up by [`model.train`](https://games-uchile.github.io/mogptk/model.htmlmogptk.model.Model.train), we need to create a new class and derive from [`mogptk.gpr.Mean`](https://games-uchile.github.io/mogptk/gpr/mean.htmlmogptk.gpr.mean.Mean). In `__init__` we need to call the base class' initializer and then add parameters by assigning instantiations of [`mogptk.gpr.Parameter`](https://games-uchile.github.io/mogptk/gpr/parameter.htmlmogptk.gpr.parameter.Parameter) to the class' properties.The `__call__` function is called to evaluate the mean function and gets passed an `X` of shape `(data points,input dims)` and should returns the Y values of shape `(datapoints,)`. Make sure to "call" your parameters before using: `coefs = self.coefs()`!<jupyter_code>class Mean(mogptk.gpr.Mean):
    def __init__(self):
        super(Mean, self).__init__()        
        self.coefficients = mogptk.gpr.Parameter([0.0, 0.0, 0.0])
    
    def __call__(self, X):
        coefs = self.coefficients()
        return coefs[0] + coefs[1]*X[:,1] + coefs[2]*X[:,1]**2<jupyter_output><empty_output><jupyter_text>We initialize our mean function, create a periodic kernel, and initialize our model. See all implemented [single output](https://games-uchile.github.io/mogptk/gpr/singleoutput.html) and [multi output](https://games-uchile.github.io/mogptk/gpr/multioutput.html) kernels. Note that we need to create an [`IndependentMultiOutputKernel`](https://games-uchile.github.io/mogptk/gpr/multioutput.htmlmogptk.gpr.multioutput.IndependentMultiOutputKernel) to handle the multi-output nature of MOGPTK, even though in this case we have only one channel.In this case, the random initialization of the parameters of the periodic kernel may give poor results. Setting reasonable values allows training to properly optimize the kernel.<jupyter_code>mean = Mean()
kernel = mogptk.gpr.PeriodicKernel(input_dims=1)
mo_kernel = mogptk.gpr.IndependentMultiOutputKernel(kernel)
model = mogptk.Model(data, mo_kernel, mean=mean, name="Periodic")

# initialize kernel parameters to reasonable values
kernel.l.assign(1.0)
kernel.p.assign(1.0)<jupyter_output><empty_output><jupyter_text>We will first only train the mean function, then the kernel, and then both the mean function and the kernel. Note that we can set `trainable` for kernels, means, and individual parameters to enable or disable training.<jupyter_code>mean.trainable = True
kernel.trainable = False
model.train(method='Adam', lr=0.1, iters=100, plot=True, error='MAE');
mean.trainable = False
kernel.trainable = True
model.train(method='Adam', lr=0.1, iters=100, plot=True, error='MAE');
mean.trainable = True
kernel.trainable = True
model.train(method='Adam', lr=0.1, iters=100, plot=True, error='MAE');
model.predict()
data.plot();

mogptk.error(model, disp=True)<jupyter_output><empty_output><jupyter_text>The trained parameters for the coefficients are close to the used values for creating the data set (`[0.0, 2.0, -0.2]`). Also note that the `IMO.noise` parameter is (usually) close to `0.5`. There are two `noise` parameters, one for the entire model and one for each channel of [`IndependentMultiOutputKernel`](https://games-uchile.github.io/mogptk/gpr/multioutput.htmlmogptk.gpr.multioutput.IndependentMultiOutputKernel) (IMO). The latter allows training different noises per channel which makes the model-wide noise redundant, thus it is fixed to zero.<jupyter_code>model.print_parameters()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10918544194107452, 'fraction_numerical': 0.01451473136915078, 'mean_word_length': 5.502816901408451, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 7, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '36051', 'license': 'MIT', 'n_tokens_mistral': 1512, 'n_tokens_neox': 1426, 'n_words': 471, 'path': 'examples/06_Custom_Kernels_and_Mean_Functions.ipynb', 'repo_name': 'GAMES-UChile/mogptk', 'size': '4616'}