<jupyter_start><jupyter_text>Preview the dataBefore we dataset is put into a dataset and load it into the DeePyMoD framework, we can preview the data to get a feeling for the shape.<jupyter_code>data = loadmat("data/advection_diffusion.mat")
usol = np.real(data["Expression1"]).astype("float32")
usol = usol.reshape((51, 51, 61, 4))
x_v= usol[:,:,:,0]
y_v = usol[:,:,:,1]
t_v = usol[:,:,:,2]
u_v = usol[:,:,:,3]
fig, axes = plt.subplots(ncols=3, figsize=(15, 4))

im0 = axes[0].contourf(x_v[:,:,0], y_v[:,:,0], u_v[:,:,0], cmap='coolwarm')
axes[0].set_xlabel('x')
axes[0].set_ylabel('y')
axes[0].set_title('t = 0')

im1 = axes[1].contourf(x_v[:,:,10], y_v[:,:,10], u_v[:,:,10], cmap='coolwarm')
axes[1].set_xlabel('x')
axes[1].set_title('t = 10')

im2 = axes[2].contourf(x_v[:,:,20], y_v[:,:,20], u_v[:,:,20], cmap='coolwarm')
axes[2].set_xlabel('x')
axes[2].set_title('t= 20')

fig.colorbar(im1, ax=axes.ravel().tolist())

plt.show()<jupyter_output><empty_output><jupyter_text>Prepare the dataNext, we actually prepare the dataset. In order to do so, we write a function that loads the data from the matlab file and then reshapes it into a shape with thenumber of samples on the first axis and the coordinates on the next. We do so identically for the data then return both in a (time, x, y, data) format. We could also flatten the data and return it as (number_of_samples, number_of_features).<jupyter_code>def create_data():
    data = loadmat("data/advection_diffusion.mat")
    usol = np.real(data["Expression1"]).astype("float32")
    usol = usol.reshape((51, 51, 61, 4))
    x_v = usol[:,:,:,0]
    y_v = usol[:,:,:,1]
    t_v = usol[:,:,:,2]
    u_v = usol[:,:,:,3]
    coords = torch.from_numpy(np.stack((t_v,x_v, y_v), axis=-1))
    data = torch.from_numpy(usol[:, :, :, 3]).unsqueeze(-1)
    # alternative way of providing the coordinates
    # coords = torch.from_numpy(np.transpose((t_v.flatten(), x_v.flatten(), y_v.flatten())))
    # data = torch.from_numpy(usol[:, :, :, 3].reshape(-1,1))
    print("The coodinates have shape {}".format(coords.shape))
    print("The data has shape {}".format(data.shape))
    return coords, data<jupyter_output><empty_output><jupyter_text>Then we pass this function to the dataset class, which then adds noise to it, normalizes the coordinates and performs random subsampling to it.<jupyter_code>dataset = Dataset(
    create_data,
    preprocess_kwargs={
        "noise_level": 0.01,
        "normalize_coords": True,
        "normalize_data": True,
    },
    subsampler=Subsample_random,
    subsampler_kwargs={"number_of_samples": 2500},
    device=device,
)<jupyter_output>The coodinates have shape torch.Size([51, 51, 61, 3])
The data has shape torch.Size([51, 51, 61, 1])
Dataset is using device:  cuda<jupyter_text>Next we use the get_train_test_loader function to get turn the dataset into PyTorch compatible dataloaders<jupyter_code>train_dataloader, test_dataloader = get_train_test_loader(dataset, train_test_split=0.8)<jupyter_output><empty_output><jupyter_text>Configuration of DeepMoDConfiguration of the function approximator: Here the first argument is the number of input and the last argument the number of output layers.<jupyter_code>network = NN(3, [50, 50, 50, 50], 1)<jupyter_output><empty_output><jupyter_text>Configuration of the library function: We select athe library with a 2D spatial input. Note that that the max differential order has been pre-determined here out of convinience. So, for poly_order 1 the library contains the following 12 terms:* [$1, u_x, u_y, u_{xx}, u_{yy}, u_{xy}, u, u u_x, u u_y, u u_{xx}, u u_{yy}, u u_{xy}$]<jupyter_code>library = Library2D(poly_order=1)<jupyter_output><empty_output><jupyter_text>Configuration of the sparsity estimator and sparsity scheduler used. In this case we use the most basic threshold-based Lasso estimator and a scheduler that asseses the validation loss after a given patience. If that value is smaller than 1e-5, the algorithm is converged.<jupyter_code>estimator = Threshold(0.1)
sparsity_scheduler = TrainTestPeriodic()<jupyter_output><empty_output><jupyter_text>Configuration of the sparsity estimator<jupyter_code>constraint = LeastSquares()<jupyter_output><empty_output><jupyter_text>Now we instantiate the model and select the optimizer<jupyter_code>model = DeepMoD(network, library, estimator, constraint).to(device)
# Defining optimizer
optimizer = torch.optim.Adam(
    model.parameters(), betas=(0.99, 0.99), amsgrad=True, lr=1e-3
)<jupyter_output><empty_output><jupyter_text>Run DeepMoDWe can now run DeepMoD using all the options we have set and the training data:* The directory where the tensorboard file is written (log_dir)* The ratio of train/test set used (split)* The maximum number of iterations performed (max_iterations)* The absolute change in L1 norm considered converged (delta)* The amount of epochs over which the absolute change in L1 norm is calculated (patience)<jupyter_code>train(
    model,
    train_dataloader,
    test_dataloader,
    optimizer,
    sparsity_scheduler,
    log_dir="runs/2DAD/",
    max_iterations=100000,
    delta = 1e-5,
    patience=200,
)<jupyter_output>36475  MSE: 1.58e-05  Reg: 3.47e-05  L1: 1.86e+00 Algorithm converged. Writing model to disk.<jupyter_text>Sparsity masks provide the active and non-active terms in the PDE:<jupyter_code>model.sparsity_masks<jupyter_output><empty_output><jupyter_text>estimator_coeffs gives the magnitude of the active terms:<jupyter_code>model.estimator_coeffs()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12866449511400652, 'fraction_numerical': 0.028773072747014114, 'mean_word_length': 5.732034104750305, 'pattern_counts': {'":': 4, '<': 47, '<?xml version=': 0, '>': 47, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '300546', 'license': 'MIT', 'n_tokens_mistral': 2075, 'n_tokens_neox': 1962, 'n_words': 538, 'path': 'examples/PDE_2D_Advection-Diffusion.ipynb', 'repo_name': 'levandosky-k/DeePyMoD', 'size': '5526'}