<jupyter_start><jupyter_text>Optimizing a Tensor Network using PytorchIn this example we show how a general machine learningstrategy can be used to optimize tensor networks withrespect to some target loss function.We'll take the example of maximizing the overlap of somematrix product state with periodic boundary conditionswith a densely represented state, since this does nothave a simple, deterministic alternative.``quimb`` makes use of ``opt_einsum`` which can contracttensors with a variety of backends. Here we'll use ``pytorch``. Note that pytorch does not yet support complexdata (but that also means we don't need to conjugate usingthe ``.H`` attribute).<jupyter_code>import torch

import quimb as qu
import quimb.tensor as qtn

# perform all contractions with pytorch
qtn.set_contract_backend('torch')<jupyter_output><empty_output><jupyter_text>First, find a (dense) PBC groundstate, $| gs \rangle$:<jupyter_code>L = 16
H = qu.ham_heis(L, sparse=True, cyclic=True)
gs = qu.groundstate(H)<jupyter_output><empty_output><jupyter_text>Then we convert it to a (constant) torch array:<jupyter_code># this converts the dense vector to an effective 1D tensor network
target = qtn.Dense1D(gs)

# this maps the torch.tensor function over all the data arrays, here only one
target.apply_to_arrays(torch.tensor)<jupyter_output><empty_output><jupyter_text>Next we create an initial guess random MPS, $|\psi\rangle$, also converting each of the arrays to torch variables (but now requiring the gradient so that each can be optimized):<jupyter_code>bond_dim = 32
mps = qtn.MPS_rand_state(L, bond_dim, cyclic=True)
mps.apply_to_arrays(lambda t: torch.tensor(t, requires_grad=True))<jupyter_output><empty_output><jupyter_text>Last, we set up a ``pytorch`` optimizer, taking as the loss the normalized target overlap $\dfrac{|\langle gs | \psi \rangle|^2} { \langle \psi | \psi \rangle }$:<jupyter_code># we give the optimizer all the tensors it should optimize
optimizer = torch.optim.Adam([t.data for t in mps], lr=0.01)

# perform 100 steps of optimization
for t in range(1, 101):
    
    # negate the overlap as we a minimizing
    loss = - (mps @ target)**2 / (mps @ mps)
    
    # reset, compute the gradient, and take a optimize step
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    
    if t % 10 == 0:
        print(f"round: {t}, loss: {loss.item()}")<jupyter_output>round: 10, loss: -0.9405585461294388
round: 20, loss: -0.9788279069737648
round: 30, loss: -0.9925598607587998
round: 40, loss: -0.9965893422669746
round: 50, loss: -0.997983439965575
round: 60, loss: -0.9986126769786011
round: 70, loss: -0.9989872559043187
round: 80, loss: -0.9991876671618736
round: 90, loss: -0.9993254027902169
round: 100, loss: -0.9994244319138171<jupyter_text>We now have a pretty good fidelity between our PBC MPS ansatz and the target groundstate.Although the loss was computed with normalization, the MPS still needs to be normalized:<jupyter_code>mps /= (mps @ mps)**0.5
mps @ mps<jupyter_output><empty_output><jupyter_text>And finally we can check that the overlap matches the loss found:<jupyter_code>(mps @ target)**2<jupyter_output><empty_output><jupyter_text>Other things to think about might be:- playing with the optimizer type (here ADAM) and settings (e.g. learning rate)- using single precision data for GPU accelerationWe can also convert the ``pytorch`` arrays back to numpy with:<jupyter_code># the 'detach' unlinks the tensors from the gradient calculator
mps.apply_to_arrays(lambda t: t.detach().numpy())
type(mps[4].data)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09729579035405632, 'fraction_numerical': 0.059938667410091996, 'mean_word_length': 5.453237410071942, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '509217', 'license': 'Apache-2.0', 'n_tokens_mistral': 1322, 'n_tokens_neox': 1168, 'n_words': 405, 'path': 'docs/examples/ex_torch_optimize_pbc_mps.ipynb', 'repo_name': 'paulsbrookes/quimb', 'size': '3587'}