<jupyter_start><jupyter_text>Sequential fitting It is not always clear how to select good hyperparameters for calculations. In the second tutorial "Getting insights about the model" it was shown how to plot spectrums of PCA for all lambda channels and parities. This information along with the other one, such as regression accuracy might be usefull to select better hypers. Particulary, the most straghtforward way is to select number of pca components, in a such a way to cover the most part of variance, and do it successively from block to block. In this case it is very undesirable to fit all parts of the model, including not changed ones from scratch. One possible way around is to do all things by hand, as it was described in the tutorial "Constructor or non standard_sequence", but there would be additional headache with packing resulting blocks into a single model with convenient .transform method. Nice toolbox has capabilities to do it very succinctly.First of all we need to get spherical expansion coefficients the same way as in previous tutorials:<jupyter_code># downloading dataset from https://archive.materialscloud.org/record/2020.110

!wget "https://archive.materialscloud.org/record/file?file_id=b612d8e3-58af-4374-96ba-b3551ac5d2f4&filename=methane.extxyz.gz&record_id=528" -O methane.extxyz.gz
!gunzip -k methane.extxyz.gz

import numpy as np
import ase.io
import tqdm
from nice.blocks import *
from nice.utilities import *
from matplotlib import pyplot as plt
from sklearn.linear_model import BayesianRidge

structures = ase.io.read('methane.extxyz', index='0:1000')

HYPERS = {
    'interaction_cutoff': 6.3,
    'max_radial': 5,
    'max_angular': 5,
    'gaussian_sigma_type': 'Constant',
    'gaussian_sigma_constant': 0.05,
    'cutoff_smooth_width': 0.3,
    'radial_basis': 'GTO'
}

all_species = get_all_species(structures)

coefficients = get_spherical_expansion(structures, HYPERS, all_species)
coefficients = coefficients[1]<jupyter_output>--2020-10-14 21:22:20--  https://archive.materialscloud.org/record/file?file_id=b612d8e3-58af-4374-96ba-b3551ac5d2f4&filename=methane.extxyz.gz&record_id=528
Resolving archive.materialscloud.org (archive.materialscloud.org)... 172.16.17.32
Connecting to archive.materialscloud.org (archive.materialscloud.org)|172.16.17.32|:443... connected.
HTTP request sent, awaiting response... 302 FOUND
Location: https://object.cscs.ch/archive/b6/12/d8e3-58af-4374-96ba-b3551ac5d2f4/data?response-content-type=application%2Foctet-stream&response-content-disposition=attachment%3B%20filename%3Dmethane.extxyz.gz&Expires=1602703401&Signature=eG9uyvXY2PJpAnrZE3Tj2OaWXNM%3D&AWSAccessKeyId=ee64314446074ed3ab5f375a522a4893 [following]
--2020-10-14 21:22:21--  https://object.cscs.ch/archive/b6/12/d8e3-58af-4374-96ba-b3551ac5d2f4/data?response-content-type=application%2Foctet-stream&response-content-disposition=attachment%3B%20filename%3Dmethane.extxyz.gz&Expires=1602703401&Signature=eG9uyvXY2PJpAnrZE3Tj2OaWXN[...]<jupyter_text>coefficients are now spherical expansion coefficients for H centered environments:<jupyter_code>print(coefficients.shape)<jupyter_output>(4000, 10, 6, 11)<jupyter_text>Let's do the first steps from standar sequence:<jupyter_code>even_0, odd_0 = InitialTransformer().transform(coefficients)
initial_pca = IndividualLambdaPCAsBoth()
initial_pca.fit(even_0, odd_0)
even_0_t, odd_0_t = initial_pca.transform(even_0, odd_0)<jupyter_output><empty_output><jupyter_text>Now we can fit couple of standard blocks:<jupyter_code>block_1 = StandardBlock(ThresholdExpansioner(100), None,
                        IndividualLambdaPCAsBoth(20))
block_1.fit(even_0_t, odd_0_t, even_0_t, odd_0_t)
even_1, odd_1, _ = block_1.transform(even_0_t, odd_0_t, even_0_t, odd_0_t)

block_2 = StandardBlock(None, None, None,
                        ThresholdExpansioner(100, mode='invariants'))
block_2.fit(even_1, odd_1, even_0_t, odd_0_t)
_, _, even_invariants = block_2.transform(even_1, odd_1, even_0_t, odd_0_t)<jupyter_output><empty_output><jupyter_text>At his moment we have all parts of this standard sequence fitted:<jupyter_code>nice = StandardSequence(initial_pca=initial_pca, blocks=[block_1, block_2])
print(initial_pca.is_fitted())
print(block_1.is_fitted())
print(block_2.is_fitted())<jupyter_output>True
True
True<jupyter_text>what about full model?<jupyter_code>print(nice.is_fitted())<jupyter_output>False<jupyter_text>Nope. At this point there is very high probability to make a mistake. Particularly one can feed StandardSequence with some fitted initial_pca along with blocks, which were fitted based not on the same initial_pca, or with different initial_normalizer, or even on different data. In order to prevent it, there is requirement to pass additional flag guaranteed_parts_fitted_consistently = True to the model:<jupyter_code>nice = StandardSequence(initial_pca=initial_pca,
                        blocks=[block_1, block_2],
                        guaranteed_parts_fitted_consistently=True)
print(nice.is_fitted())<jupyter_output>True<jupyter_text>Model is considered to be fitted if 1) all parts are fitted and 2) if guaranteed_parts_fitted_consistently is set to be True**Golden rule:** Every time you pass guaranteed_parts_fitted_consistently = True make a pause and think twice. Let's check consistency:<jupyter_code>even_invariants_2 = nice.transform(coefficients,
                                   return_only_invariants=True)[3]
print(np.sum(np.abs(even_invariants - even_invariants_2)))<jupyter_output>0.0<jupyter_text>This also works in other direction:<jupyter_code>initial_pca = IndividualLambdaPCAsBoth()
block_1 = StandardBlock(ThresholdExpansioner(100), None,
                        IndividualLambdaPCAsBoth(20))
block_2 = StandardBlock(None, None, None,
                        ThresholdExpansioner(100, mode='invariants'))

print(initial_pca.is_fitted())
print(block_1.is_fitted())
print(block_2.is_fitted())

nice = StandardSequence(initial_pca=initial_pca, blocks=[block_1, block_2])
nice.fit(coefficients)

print(initial_pca.is_fitted())
print(block_1.is_fitted())
print(block_2.is_fitted())<jupyter_output>False
False
False
True
True
True<jupyter_text>StandardBlock behaves the same way:<jupyter_code>expansioner, pca = ThresholdExpansioner(100), IndividualLambdaPCAsBoth(20)
print(expansioner.is_fitted())
print(pca.is_fitted())

block = StandardBlock(expansioner, None, pca)
block.fit(even_0_t, odd_0_t, even_0_t, odd_0_t)

print(expansioner.is_fitted())
print(pca.is_fitted())
expansioner, pca = ThresholdExpansioner(100), IndividualLambdaPCAsBoth(20)
expansioner.fit(even_0_t, odd_0_t, even_0_t, odd_0_t)
even_1, odd_1 = expansioner.transform(even_0_t, odd_0_t, even_0_t, odd_0_t)
pca.fit(even_1, odd_1)

block = StandardBlock(expansioner,
                      None,
                      pca,
                      guaranteed_parts_fitted_consistently=True)

print(block.is_fitted())<jupyter_output>True
{'fraction_non_alphanumeric': 0.09321789321789321, 'fraction_numerical': 0.047041847041847044, 'mean_word_length': 5.821850393700787, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '66326', 'license': 'MIT', 'n_tokens_mistral': 2551, 'n_tokens_neox': 2329, 'n_words': 521, 'path': 'tutorials/sequential_fitting.ipynb', 'repo_name': 'cosmo-epfl/nice', 'size': '6932'}