<jupyter_start><jupyter_text>Imports<jupyter_code>import pysweep
import qcodes as qc
import pysweep.databackends.debug
import pysweep.databackends.list_backend
import pysweep.databackends.qcodes
import pysweep.core.measurementfunctions
from pysweep.core.sweepobject import SweepObject
from pysweep.core.measurementfunctions import MakeMeasurementFunction
import pysweep.convenience as conv
import random
import time
import numpy as np
import pandas as pd

import matplotlib.pyplot as plt
from pysweep.databackends.base import DataParameter
from pytopo.qctools.dataset2 import select_experiment
from qcodes.dataset.measurements import Measurement

from qcodes import load_by_id
import qcodes as qc
import qcodes.dataset.plotting
import importlib
station = qc.Station()
pysweep.STATION = station
qc.config.core.db_location = r'testdata.db'
qc.initialise_database()<jupyter_output><empty_output><jupyter_text>Define a few parameters with physics<jupyter_code>bias = qc.instrument.parameter.ManualParameter("bias", units="V")
T1 = qc.instrument.parameter.ManualParameter("T1", units="V")
T2 = qc.instrument.parameter.ManualParameter("T2", units="V")

bias(0)
T1(0)
T2(0)

def pinchoff(v, rs=10e3, l=2e-6, muc=3*60e-18, vth=-1):
    if v>vth:
        return (rs+l**2/(muc*(v-vth)))
    else:
        return 1e11

def get_current():
    u = bias()
    r = pinchoff(T1())+pinchoff(T2())
    return u/r
current = qc.instrument.parameter.Parameter("current", units="A", get_cmd=get_current)<jupyter_output><empty_output><jupyter_text>Define the functions that define the measurement We need to define what happens at the start of a measurement, what must happen at the end of a measurement and finally what a measurement entails<jupyter_code>def init_measurement(d):
    bias(1e-3)

def end_measurement(d):
    bias(0)<jupyter_output><empty_output><jupyter_text>Any measurement that pysweep does is encapsulated as a MeasurementFucntion class. It contains a function that returns the measurement results as a list, as well as a list containing the metadata for every return value, this list is called the `ParamStruct`<jupyter_code>@MakeMeasurementFunction([pysweep.DataParameter(name='current', unit='A')])
def measure(d):
    return [current()]
type(measure)
measure.paramstruct<jupyter_output><empty_output><jupyter_text>lets also define a function the measures the hypothetical temperature of our fridgeBecause the explicit definition opf the DataParameter is overly verbose, we introduce the shorthand:If an element `param` of the datastruct is not of type `DataParameter`, it will try to construct a dataparameters automatically as`DataParameter(*param)`as such we can define a measurementfunction as<jupyter_code>@MakeMeasurementFunction([['Temperature', 'mK']])
def measure_temperature(d):
    return [random.randint(100, 300)/10]<jupyter_output><empty_output><jupyter_text>Simple measurements - Simple Sweep objects Since we now have the functions that will be execute to measure something, we can now look at the code that will perform the sweeping In pysweep, sweeps are performed with `SweepObjects` which consist of 4 elements* set_function* unit* label* point_function For now, let us ignore the details and see how they can be constructed:`pysweep.sweep_object(T1, np.linspace(0,-1.2,51))`This encodes the idea of sweeping the `T1` gate from 0 to -1.2 in 51 steps (`np.linspace(0,-1.2,51)`) With this extra component, we have enouigh to start performing measurements. This paragraph contains some measurements that corresponds to the easiest cases of measurements. The cases cover 90% of all measurements that will run, but don't show the full power of pysweep<jupyter_code>exp = select_experiment('pinchoff', 'test_sample')
meas = Measurement(exp, station)
r = pysweep.sweep(init_measurement, end_measurement, measure, 
              pysweep.sweep_object(T1, np.linspace(0,-1.2,51)), 
              databackend = pysweep.databackends.qcodes.DataBackend(meas)
            )
qcodes.dataset.plotting.plot_by_id(r.datasaver.run_id)<jupyter_output><empty_output><jupyter_text>Adding another dimension is trivial, where we follow the rule that the inner most loop is at the top, closest to the `MeasurementFunction`<jupyter_code>exp = select_experiment('pinchoff', 'test_sample')
meas = Measurement(exp, station)
r = pysweep.sweep(init_measurement, end_measurement, measure, 
              pysweep.sweep_object(T1, np.linspace(0,-1.2,51)), 
              pysweep.sweep_object(T2, np.linspace(0,-1.2,51)), 
              databackend=pysweep.databackends.qcodes.DataBackend(meas)
            )
qcodes.dataset.plotting.plot_by_id(r.datasaver.run_id)<jupyter_output><empty_output><jupyter_text>Extended examples Here are a few examples of measurements that are still relatively simples, but with a few more bells and whistles Say that we want to measure the fridge temperature in addition to the current. This can be done by creating the combined measurement object `measure+measure_temperature` and hence we simply execute<jupyter_code>exp = select_experiment('pinchoff', 'test_sample')
meas = Measurement(exp, station)
r = pysweep.sweep(init_measurement, end_measurement, measure+measure_temperature, 
              pysweep.sweep_object(T1, np.linspace(0,-1.2,51)), 
              pysweep.sweep_object(T2, np.linspace(0,-1.2,51)), 
              databackend=pysweep.databackends.qcodes.DataBackend(meas)
            )
qcodes.dataset.plotting.plot_by_id(r.datasaver.run_id)<jupyter_output><empty_output><jupyter_text>But often we only want to measure temperature every now and then, so let's assume that we want to measure the fridge tmeperature only after every time we set the T2 parameter<jupyter_code>exp = select_experiment('pinchoff', 'test_sample')
meas = Measurement(exp, station)
r = pysweep.sweep(init_measurement, end_measurement, measure, 
              pysweep.sweep_object(T1, np.linspace(0,-1.2,51)), 
              conv.add_function(pysweep.sweep_object(T2, np.linspace(0,-1.2,51)), measure_temperature), 
              databackend = pysweep.databackends.qcodes.DataBackend(meas)
            )
qcodes.dataset.plotting.plot_dataset(r.datasaver.dataset)<jupyter_output><empty_output><jupyter_text>Hardware sweep Often, hardware is used to sweep one of the parameters instead of software. Pysweep deals with these situations natively. Let's assume that we use a VNA to measure a conductance resonator in the sample, we first define the code that measures the VNA. In this case we write a mock version of this to simulate the result<jupyter_code>@MakeMeasurementFunction([['frequency', 'Herz', 'array', True], ['S11', 'dB', 'array']])
def measure_vna(d):
    freqs = np.linspace(440e6, 460e6, 201)
    omega = 2*np.pi*freqs
    l = 420e-9
    c = 0.3e-12
    r = pinchoff(T1())+pinchoff(T2())
    z = 1j*omega*l+1/(1/r + 1j*omega*c)
    z_0 = 50
    Gamma = (z - z_0)/(z + z_0)
    
    return [freqs, 10 + 20 * np.log10(np.abs(Gamma))-70]<jupyter_output><empty_output><jupyter_text>Where we make explicit in the parameter definition that frequency and S11 are parameters of type array and that frequency is an independent parameter When running the measurement, ensure to reopen T2 to measure the pinchoff along T1<jupyter_code>exp = select_experiment('rf_pinchoff', 'test_sample')
meas = Measurement(exp, station)
T2(0)
r = pysweep.sweep(init_measurement, end_measurement, measure_vna, 
              pysweep.sweep_object(T1, np.linspace(0,-1.2,51)), 
              databackend = pysweep.databackends.qcodes.DataBackend(meas)
            )
qcodes.dataset.plotting.plot_dataset(r.datasaver.dataset)<jupyter_output><empty_output><jupyter_text>A note on parameter dependencies To figure out which parameters depend on which parameters, pysweep uses a simple rule: every parameter depends on ALL independent parameters that occur earlier in the `paramstruct`looking at the paramstruct of the previous measurement:<jupyter_code>r.columns<jupyter_output><empty_output><jupyter_text>This assumption is usually correct but can lead to problems, for example in the next measurement which does not workBecause of the preceding rule, pysweep expects in the following piece of code that `current` will depend on `frequency`, which is of course not the case.<jupyter_code>exp = select_experiment('rf_pinchoff', 'test_sample')
meas = Measurement(exp, station)
T2(0)
try:
    r = pysweep.sweep(init_measurement, end_measurement, measure_vna+measure, 
                  pysweep.sweep_object(T1, np.linspace(0,-1.2,51)), 
                  databackend = pysweep.databackends.qcodes.DataBackend(meas)
                )
except ValueError as e:
    print(e)<jupyter_output><empty_output><jupyter_text>Simple solution Switching around the two measurement functions solves this issue:<jupyter_code>exp = select_experiment('rf_pinchoff', 'test_sample')
meas = Measurement(exp, station)
T2(0)
r = pysweep.sweep(init_measurement, end_measurement, measure+measure_vna, 
                  pysweep.sweep_object(T1, np.linspace(0,-1.2,51)), 
                  databackend = pysweep.databackends.qcodes.DataBackend(meas)
                )
qcodes.dataset.plotting.plot_dataset(r.datasaver.dataset)<jupyter_output><empty_output><jupyter_text>Advanced solution In some cases it might be the case that both orders will not work. The usecase might be two incompatible hardware sweeps bundeled together. In this case the issue can also be resolved bu explicitely telling pysweep that frequency and S11 are together and do not interfere with any other parameter.Because the implicit syntax for the dataparameters gets confusing, I'll use the explicit notation here<jupyter_code>@MakeMeasurementFunction([pysweep.DataParameter(name='frequency_2',
                                              unit='Hz',
                                               paramtype='array',
                                               independent=2  # yes independent, but pysweep will not recognize it as such
                                            ),
                         pysweep.DataParameter(name='S11_2', 
                                              unit='dB',
                                              paramtype='array',
                                              extra_dependencies=['frequency_2']  # explicitely tell that this parameter depends on frequency_2
                                              )])
def measure_vna2(d):
    freqs = np.linspace(440e6, 460e6, 11)
    omega = 2*np.pi*freqs
    l = 420e-9
    c = 0.3e-12
    r = pinchoff(T1())+pinchoff(T2())
    z = 1j*omega*l+1/(1/r + 1j*omega*c)
    z_0 = 50
    Gamma = (z - z_0)/(z + z_0)
    
    return [freqs, 10 + 20 * np.log10(np.abs(Gamma))-70]
exp = select_experiment('rf_pinchoff', 'test_sample')
meas = Measurement(exp, station)
T2(0)
r = pysweep.sweep(init_measurement, end_measurement, measure_vna2+measure_vna, 
                  pysweep.sweep_object(T1, np.linspace(0,-1.2,51)), 
                  databackend = pysweep.databackends.qcodes.DataBackend(meas)
                )
qcodes.dataset.plotting.plot_dataset(r.datasaver.dataset)<jupyter_output><empty_output><jupyter_text>Pysweep (with the qcodes backend) only allows one parameter with the same name per measurement, to avoid overwriting data. This is why we renamed the parameters initially<jupyter_code>@MakeMeasurementFunction([pysweep.DataParameter(name='frequency',
                                              unit='Hz',
                                               paramtype='array',
                                               independent=2  # yes independent, but pysweep will not recognize it as such
                                            ),
                         pysweep.DataParameter(name='S11', 
                                              unit='dB',
                                              paramtype='array',
                                              extra_dependencies=['frequency']  # explicitely tell that this parameter depends on frequency_2
                                              )])
def measure_vna2(d):
    freqs = np.linspace(440e6, 460e6, 11)
    omega = 2*np.pi*freqs
    l = 420e-9
    c = 0.3e-12
    r = pinchoff(T1())+pinchoff(T2())
    z = 1j*omega*l+1/(1/r + 1j*omega*c)
    z_0 = 50
    Gamma = (z - z_0)/(z + z_0)
    
    return [freqs, 10 + 20 * np.log10(np.abs(Gamma))-70]
exp = select_experiment('rf_pinchoff', 'test_sample')
meas = Measurement(exp, station)
T2(0)
r = pysweep.sweep(init_measurement, end_measurement, measure_vna2+measure_vna, 
                  pysweep.sweep_object(T1, np.linspace(0,-1.2,51)), 
                  databackend = pysweep.databackends.qcodes.DataBackend(meas)
                )<jupyter_output>None3 , postponing registration
None2 , postponing registration<jupyter_text>Adaptive sweep - under construction<jupyter_code>@MakeMeasurementFunction([['timestamp', 's']])
def smart_point_function(dict_waterfall):
    timestamp = int(time.time())
    points = [int(c) for c in str(timestamp)]
    return points, [timestamp]
db = pysweep.databackends.debug.DebugDataBackend()
pysweep.sweep(lambda d:{}, lambda d:{}, measure, 
              pysweep.sweep_object(a, [0,1]),
              conv.add_function(pysweep.sweep_object(b, [3,4]), measure_temperature),
              pysweep.sweep_object(c, smart_point_function, dataparameter=np.linspace(0,9,10)), databackend=db
             )<jupyter_output><empty_output><jupyter_text>Real life example - under construction<jupyter_code>%%enqueue single_shot_readout
exp = select_experiment('blobs', SAMPLE)
meas = Measurement(exp, station)
singleshot_dataextractor.setup_acquisition(1024, 1000, 100)
#4.77946e9
parameter_list = [[station.TI.NW_bias, 0e-3], [station.HT.frequency, 4779770000], [station.HT.RF.power, -20.25]]

pysweep.sweep(sf.set_parameters_after(init_rf, parameter_list), end_rf, 
              measure_singleshot_rf,
              pysweep.sweep_object(station.TI.SP4, snr_calibrate_plunger(pysweep.sweep_object(station.TI.SP4, np.linspace(0., 0.4, 101))), dataparameter=[0,1]),
              pysweep.convenience.add_function(pysweep.sweep_object(station.TI.T5, np.linspace(-1, -1.35, 101)), csc),
              databackend = pysweep.databackends.qcodes.DataBackend(meas)
             )
rigol.ch1_output_enabled(False)<jupyter_output><empty_output><jupyter_text>Jagged Data Structures - under construction<jupyter_code>?DataParameter
@MakeMeasurementFunction([DataParameter('time', 's', 'array', 2),
                         DataParameter('first_gate', 'V', 'array', 2),
                         DataParameter('second_gate', 'V', 'array', 2),
                         DataParameter('first_gate_result', 'I', 'array', extra_dependencies=['time', 'first_gate']),
                         DataParameter('time', 's', 'array', 2, duplicate=True),
                         DataParameter('second_gate_result', 'I', 'array', extra_dependencies=['time', 'second_gate'])])
def some_arbitrary_data(d):
    return [np.linspace(0,1, 11), np.linspace(0,0.1, 11), np.linspace(0,-0.1, 21), np.linspace(0,0.2, 11), np.linspace(1,2, 21), np.linspace(0,-0.2, 21)]
importlib.reload(pysweep.databackends.qcodes)
exp = select_experiment('jagged_data', 'test')
meas = Measurement(exp, station)

pysweep.sweep(init_measurement, end_measurement, 
              some_arbitrary_data,
              pysweep.sweep_object(T1, np.linspace(0,1,2)),
              databackend = pysweep.databackends.qcodes.CutDataBackend(meas)
             )
raw_data = load_by_id(12)
data = raw_data.get_parameter_data()
raw_data
data['first_gate_result']['first_gate_result']
plt.plot(data['first_gate_result']['first_gate'][0], data['first_gate_result']['first_gate_result'][0], '.')
plt.show()
plt.plot(data['second_gate_result']['second_gate'][0], data['second_gate_result']['second_gate_result'][0], '.')
plt.show()
plt.plot(data['first_gate_result']['time'][0], data['first_gate_result']['first_gate_result'][0], '.')
plt.plot(data['second_gate_result']['time'][0], data['second_gate_result']['second_gate_result'][0], '.')
plt.xlabel('time')
plt.ylabel('gate_result')
plt.show()
from plottr.data.datadict import datadict_to_meshgrid
from plottr.data.qcodes_dataset import ds_to_datadict
def load(runid):
    return datadict_to_meshgrid(ds_to_datadict(load_by_id(runid)))
ds = ds_to_datadict(load_by_id(12))
from plottr.node.data_selector import DataSelector
dataselector = DataSelector('dataselector')
ds
ds2 = ds.extract(['first_gate_result'])
ds3 = datadict_to_meshgrid(ds2)
ds3.validate()
ds3.shape()
ds3.structure(), ds3.shapes()
ds3.sanitize()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1064611308484374, 'fraction_numerical': 0.023965797555247786, 'mean_word_length': 3.790308624170753, 'pattern_counts': {'":': 0, '<': 76, '<?xml version=': 0, '>': 77, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 32, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '589589', 'license': 'MIT', 'n_tokens_mistral': 5597, 'n_tokens_neox': 5238, 'n_words': 1229, 'path': 'docs/Examples_damaz.ipynb', 'repo_name': 'damazter/pysweep', 'size': '16607'}