<jupyter_start><jupyter_text>Using GST to test for context dependenceThis example shows how to introduce new operation labels into a GST analysis so as to test for context dependence.  In particular, we'll look at the 1-qubit X, Y, I model.  Suppose a usual GST analysis cannot fit the model well, and that we think this is due to the fact that a "Gi" gate which immediately follows a "Gx" gate is affected by some residual noise that isn't otherwise present.  In this case, we can model the system as having two different "Gi" gates: "Gi" and "Gi2", and model the "Gi" gate as "Gi2" whenever it follows a "Gx" gate.<jupyter_code>from __future__ import print_function
import pygsti
from pygsti.modelpacks.legacy import std1Q_XYI<jupyter_output><empty_output><jupyter_text>First we'll create a mock data set that exhibits this context dependence.  To do this, we add an additional "Gi2" gate to the data-generating model, generate some data using "Gi2"-containing operation sequences, and finally replace all instances of "Gi2" with "Gi" so that it looks like data that was supposed to have just a single "Gi" gate.<jupyter_code># The usual setup: identify the target model, fiducials, germs, and max-lengths
target_model = std1Q_XYI.target_model()
fiducials = std1Q_XYI.fiducials
germs = std1Q_XYI.germs
maxLengths = [1,2,4,8,16,32]

# Create a Model to generate the data - one that has two identity gates: Gi and Gi2
mdl_datagen = target_model.depolarize(op_noise=0.1, spam_noise=0.001)
mdl_datagen["Gi2"] = mdl_datagen["Gi"].copy()
mdl_datagen["Gi2"].depolarize(0.1) # depolarize Gi2 even further
mdl_datagen["Gi2"].rotate( (0,0,0.1), mdl_datagen.basis) # and rotate it slightly about the Z-axis

# Create a set of operation sequences by constructing the usual set of experiments and using 
# "manipulate_circuits" to replace Gi with Gi2 whenever it follows Gx.  Create a 
# DataSet using the resulting Gi2-containing list of sequences.
listOfExperiments = pygsti.construction.create_lsgst_circuits(target_model, fiducials, fiducials, germs, maxLengths)
rules = [ (("Gx","Gi") , ("Gx","Gi2")) ] # a single replacement rule: GxGi => GxGi2 
listOfExperiments = pygsti.construction.manipulate_circuits(listOfExperiments, rules)
ds = pygsti.construction.simulate_data(mdl_datagen, listOfExperiments, nSamples=10000,
                                            sampleError="binomial", seed=1234)

# Revert all the Gi2 labels back to Gi, so that the DataSet doesn't contain any Gi2 labels.
rev_rules = [ (("Gi2",) , ("Gi",)) ] # returns all Gi2's to Gi's 
ds = ds.copy_nonstatic()
ds.process_circuits(lambda opstr: pygsti.construction.manipulate_circuit(opstr,rev_rules))
ds.done_adding_data()<jupyter_output><empty_output><jupyter_text>Running "standard" GST on this `DataSet` resulst in a bad fit:<jupyter_code>target_model.set_all_parameterizations("TP")
results = pygsti.run_long_sequence_gst(ds, target_model, fiducials, fiducials,
                                      germs, maxLengths, verbosity=2)<jupyter_output>--- Circuit Creation ---
   1702 sequences created
   Dataset has 1702 entries: 1702 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244188936768758
  1.1682925525201766
  0.9547999005720996
  0.9492497151099545
  0.021826565768099753
  0.0037403199750632556
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.242640687119284
  1.4142135623730956
  1.4142135623730954
  1.4142135623730951
  2.5394445830714747e-16
  1.118988490269554e-16
  
--- Iterative MLGST: Iter 1 of 6  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 103.835 (92 data params - 31 model params = expected mean of 61; p-value = 0.000518234)
  Completed in 0.4s
  2*Delta(log(L)) = 103.629
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 6  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 334.957 (168 data params - 31 model params = expected mean of 137; p-value = 0)
[...]<jupyter_text>So, since we have a hunch that the reason for the bad fit is that when "Gi" follows "Gx" it looks different, we can fit that data to a model that has two identity gates, call them "Gi" and "Gi2" again, and tell GST to perform the "GxGi => GxGi2" manipulation rule before computing the probability of a gate sequence:<jupyter_code>#Create a target model which includes a duplicate Gi called Gi2
mdl_targetB = target_model.copy()
mdl_targetB['Gi2'] = target_model['Gi'].copy() # Gi2 should just be another Gi

#Run GST with:
# 1) replacement rules giving instructions how to process all of the operation sequences
# 2) data set aliases which replace labels in the *processed* strings before querying the DataSet.
rules = [ (("Gx","Gi") , ("Gx","Gi2")) ] # a single replacement rule: GxGi => GxGi2 
resultsB = pygsti.run_long_sequence_gst(ds, mdl_targetB, fiducials, fiducials,
                                       germs, maxLengths, 
                                       advancedOptions={"opLabelAliases": {'Gi2': pygsti.obj.Circuit(['Gi'])},
                                                        "stringManipRules": rules},
                                       verbosity=2)<jupyter_output>--- Circuit Creation ---
   1738 sequences created
   Dataset has 1702 entries: 1738 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244188936768758
  1.1682925525201766
  0.9547999005720996
  0.9492497151099545
  0.021826565768099753
  0.0037403199750632556
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.242640687119284
  1.4142135623730956
  1.4142135623730954
  1.4142135623730951
  2.5394445830714747e-16
  1.118988490269554e-16
  
--- Iterative MLGST: Iter 1 of 6  128 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 170.639 (128 data params - 43 model params = expected mean of 85; p-value = 1.07327e-07)
  Completed in 0.6s
  2*Delta(log(L)) = 170.262
  Iteration 1 took 0.6s
  
--- Iterative MLGST: Iter 2 of 6  204 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 434.966 (204 data params - 43 model params = expected mean of 161; p-value = 0[...]<jupyter_text>This gives a better fit, but not as good as it should (given that we know the data was generated from exactly the model being used).  This is due to the (default) LGST seed being a bad starting point, which can happen, particularly when looking for context dependence.  (The LGST estimate - which you can print using `print(resultsB.estimates['default'].models['seed'])` - generates the *same* estimate for Gi and Gi2 which is roughly between the true values of Gi and Gi2, which can be a bad estimate for both gates.)  To instead use our own custom guess as the starting point, we do the following:<jupyter_code>#Create a guess, which we'll use instead of LGST - here we just
# take a slightly depolarized target.
mdl_start = mdl_targetB.depolarize(op_noise=0.01, spam_noise=0.01)

#Run GST with the replacement rules as before.
resultsC = pygsti.run_long_sequence_gst(ds, mdl_targetB, fiducials, fiducials,
                                       germs, maxLengths, 
                                       advancedOptions={"opLabelAliases": {'Gi2': pygsti.obj.Circuit(['Gi'])},
                                                        "stringManipRules": rules,
                                                        "starting point": mdl_start},
                                       verbosity=2)<jupyter_output>--- Circuit Creation ---
   1738 sequences created
   Dataset has 1702 entries: 1738 utilized, 0 requested sequences were missing
--- Iterative MLGST: Iter 1 of 6  128 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 170.639 (128 data params - 43 model params = expected mean of 85; p-value = 1.07324e-07)
  Completed in 0.6s
  2*Delta(log(L)) = 170.262
  Iteration 1 took 0.6s
  
--- Iterative MLGST: Iter 2 of 6  204 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 434.966 (204 data params - 43 model params = expected mean of 161; p-value = 0)
  Completed in 0.4s
  2*Delta(log(L)) = 432.7
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 6  486 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1210.84 (486 data params - 43 model params = expected mean of 443; p-value = 0)
  Completed in 0.8s
  2*Delta(log(L)) = 1200.98
  Iteration 3 took 1.0s
  
--- Iterative MLGST: Iter 4 of 6  898 operation sequences ---: 
  --- M[...]<jupyter_text>This results is a much better fit and estimate, as seen from the final `2*Delta(log(L))` number.<jupyter_code>gsA = pygsti.gaugeopt_to_target(results.estimates['default'].models['final iteration estimate'], mdl_datagen)
gsB = pygsti.gaugeopt_to_target(resultsB.estimates['default'].models['final iteration estimate'], mdl_datagen)
gsC = pygsti.gaugeopt_to_target(resultsC.estimates['default'].models['final iteration estimate'], mdl_datagen)
gsA['Gi2'] = gsA['Gi'] #so gsA is comparable with mdl_datagen
print("Diff between truth and standard GST: ", mdl_datagen.frobeniusdist(gsA))
print("Diff between truth and context-dep GST w/LGST starting pt: ", mdl_datagen.frobeniusdist(gsB))
print("Diff between truth and context-dep GST w/custom starting pt: ", mdl_datagen.frobeniusdist(gsC))<jupyter_output>Diff between truth and standard GST:  0.0195517914063444
Diff between truth and context-dep GST w/LGST starting pt:  0.005951917086817148
Diff between truth and context-dep GST w/custom starting pt:  0.005951941892478445
{'fraction_non_alphanumeric': 0.09440049958368027, 'fraction_numerical': 0.08669858451290591, 'mean_word_length': 3.4179310344827587, 'pattern_counts': {'":': 5, '<': 21, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 15, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '165018', 'license': 'Apache-2.0', 'n_tokens_mistral': 3676, 'n_tokens_neox': 3021, 'n_words': 1067, 'path': 'jupyter_notebooks/Examples/ContextDependence.ipynb', 'repo_name': 'colibri-coruscans/pyGSTi', 'size': '9608'}