<jupyter_start><jupyter_text>Data-reuploading classifier===========================*Author: <NAME> (<EMAIL>)*.. meta::   :property="og:description": Implement a single-qubit universal quantum classifier using PennyLane.   :property="og:image": https://pennylane.ai/qml/_images/universal_dnn1.png.. related::   tutorial_variational_classifier Variational quantum classifier   tutorial_multiclass_classification Multiclass margin classifier   tutorial_expressivity_fourier_series Quantum models as Fourier seriesA single-qubit quantum circuit which can implement arbitrary unitaryoperations can be used as a universal classifier much like a singlehidden-layered Neural Network. As surprising as it sounds,`Pérez-Salinas et al. (2019) `_discuss this with their idea of 'datareuploading'. It is possible to load a single qubit with arbitrarydimensional data and then use it as a universal classifier.In this example, we will implement this idea with Pennylane - apython based tool for quantum machine learning, automaticdifferentiation, and optimization of hybrid quantum-classicalcomputations.Background----------We consider a simple classification problem and will train asingle-qubit variational quantum circuit to achieve this goal. The datais generated as a set of random points in a plane $(x_1, x_2)$ andlabeled as 1 (blue) or 0 (red) depending on whether they lie inside oroutside a circle. The goal is to train a quantum circuit to predict thelabel (red or blue) given an input point’s coordinate... figure:: ../demonstrations/data_reuploading/universal_circles.png   :scale: 65%   :alt: circlesTransforming quantum states using unitary operations~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~A single-qubit quantum state is characterized by a two-dimensional statevector and can be visualized as a point in the so-called Bloch sphere.Instead of just being a 0 (up) or 1 (down), it can exist in asuperposition with say 30% chance of being in the $|0 \rangle$ and70% chance of being in the $|1 \rangle$ state. This is representedby a state vector $|\psi \rangle = 0.3|0 \rangle + 0.7|1 \rangle$ -the probability "amplitude" of the quantum state. In general we can takea vector $(\alpha, \beta)$ to represent the probabilities of a qubitbeing in a particular state and visualize it on the Bloch sphere as anarrow... figure:: ../demonstrations/data_reuploading/universal_bloch.png   :scale: 65%   :alt: blochData loading using unitaries~~~~~~~~~~~~~~~~~~~~~~~~~~~~In order to load data onto a single qubit, we use a unitary operation$U(x_1, x_2, x_3)$ which is just a parameterizedmatrix multiplication representing the rotation of the state vector inthe Bloch sphere. E.g., to load $(x_1, x_2)$ into the qubit, wejust start from some initial state vector, $|0 \rangle$,apply the unitary operation $U(x_1, x_2, 0)$ and end up at a newpoint on the Bloch sphere. Here we have padded 0 since our data is only2D. Pérez-Salinas et al. (2019) discuss how to load a higherdimensional data point ($[x_1, x_2, x_3, x_4, x_5, x_6]$) bybreaking it down in sets of three parameters($U(x_1, x_2, x_3), U(x_4, x_5, x_6)$).Model parameters with data re-uploading~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Once we load the data onto the quantum circuit, we want to have sometrainable nonlinear model similar to a neural network as well as a way oflearning the weights of the model from data. This is again done withunitaries, $U(\theta_1, \theta_2, \theta_3)$, such that we load thedata first and then apply the weights to form a single layer$L(\vec \theta, \vec x) = U(\vec \theta)U(\vec x)$. In principle,this is just application of two matrix multiplications on an inputvector initialized to some value. In order to increase the number oftrainable parameters (similar to increasing neurons in a single layer ofa neural network), we can reapply this layer again and again with newsets of weights,$L(\vec \theta_1, \vec x) L(\vec \theta_2, , \vec x) ... L(\vec \theta_L, \vec x)$for $L$ layers. The quantum circuit would look like the following:.. figure:: ../demonstrations/data_reuploading/universal_layers.png   :scale: 75%   :alt: LayersThe cost function and "nonlinear collapse"~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~So far, we have only performed linear operations (matrixmultiplications) and we know that we need to have some nonlinearsquashing similar to activation functions in neural networks to reallymake a universal classifier (Cybenko 1989). Here is where things gets abit quantum. After the application of the layers, we will end up at somepoint on the Bloch sphere due to the sequence of unitaries implementingrotations of the input. These are still just linear transformations ofthe input state. Now, the output of the model should be a class labelwhich can be encoded as fixed vectors (Blue = $[1, 0]$, Red =$[0, 1]$) on the Bloch sphere. We want to end up at either of themafter transforming our input state through alternate applications ofdata layer and weights.We can use the idea of the “collapse” of our quantum state intoone or other class. This happens when we measure the quantum state whichleads to its projection as either the state 0 or 1. We can compute thefidelity (or closeness) of the output state to the class label makingthe output state jump to either $| 0 \rangle$ or$|1\rangle$. By repeating this process several times, we cancompute the probability or overlap of our output to both labels andassign a class based on the label our output has a higher overlap. Thisis much like having a set of output neurons and selecting the one whichhas the highest value as the label.We can encode the output label as a particular quantum state that we wantto end up in and use Pennylane to find the probability of ending up in thatstate after running the circuit. We construct an observable corresponding tothe output label using the `Hermitian `_operator. The expectation value of the observable gives the overlap or fidelity.We can then define the cost function as the sum of the fidelities for allthe data points after passing through the circuit and optimize the parameters$(\vec \theta)$ to minimize the cost.\begin{align}\texttt{Cost} = \sum_{\texttt{data points}} (1 - \texttt{fidelity}(\psi_{\texttt{output}}(\vec x, \vec \theta), \psi_{\texttt{label}}))\end{align}Now, we can use our favorite optimizer to maximize the sum of thefidelities over all data points (or batches of datapoints) and find theoptimal weights for classification. Gradient-based optimizers such asAdam (Kingma et. al., 2014) can be used if we have a good model ofthe circuit and how noise might affect it. Or, we can use somegradient-free method such as L-BFGS (<NAME>., and <NAME>., 1989)to evaluate the gradient and find the optimal weights where we cantreat the quantum circuit as a black-box and the gradients are computednumerically using a fixed number of function evaluations and iterations.The L-BFGS method can be used with the PyTorch interface for Pennylane.Multiple qubits, entanglement and Deep Neural Networks~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~The Universal Approximation Theorem declares that a neural network withtwo or more hidden layers can serve as a universal function approximator.Recently, we have witnessed remarkable progress of learning algorithms usingDeep Neural Networks.Pérez-Salinas et al. (2019) make a connection to Deep Neural Networks bydescribing that in their approach the“layers” $L_i(\vec \theta_i, \vec x )$ are analogous to the sizeof the intermediate hidden layer of a neural network. And the concept ofdeep (multiple layers of the neural network) relates to the numberof qubits. So, multiple qubits with entanglement between them couldprovide some quantum advantage over classical neural networks. But here,we will only implement a single qubit classifier... figure:: ../demonstrations/data_reuploading/universal_dnn.png   :scale: 35%   :alt: DNN"Talk is cheap. Show me the code." - <NAME>---------------------------------------------------<jupyter_code>import pennylane as qml
from pennylane import numpy as np
from pennylane.optimize import AdamOptimizer, GradientDescentOptimizer
qml.enable_tape()

import matplotlib.pyplot as plt


# Set a random seed
np.random.seed(42)


# Make a dataset of points inside and outside of a circle
def circle(samples, center=[0.0, 0.0], radius=np.sqrt(2 / np.pi)):
    """
    Generates a dataset of points with 1/0 labels inside a given radius.

    Args:
        samples (int): number of samples to generate
        center (tuple): center of the circle
        radius (float: radius of the circle

    Returns:
        Xvals (array[tuple]): coordinates of points
        yvals (array[int]): classification labels
    """
    Xvals, yvals = [], []

    for i in range(samples):
        x = 2 * (np.random.rand(2)) - 1
        y = 0
        if np.linalg.norm(x - center) < radius:
            y = 1
        Xvals.append(x)
        yvals.append(y)
    return np.array(Xvals), np.array(yvals)


def plot_data(x, y, fig=None, ax=None):
    """
    Plot data with red/blue values for a binary classification.

    Args:
        x (array[tuple]): array of data points as tuples
        y (array[int]): array of data points as tuples
    """
    if fig == None:
        fig, ax = plt.subplots(1, 1, figsize=(5, 5))
    reds = y == 0
    blues = y == 1
    ax.scatter(x[reds, 0], x[reds, 1], c="red", s=20, edgecolor="k")
    ax.scatter(x[blues, 0], x[blues, 1], c="blue", s=20, edgecolor="k")
    ax.set_xlabel("$x_1$")
    ax.set_ylabel("$x_2$")


Xdata, ydata = circle(500)
fig, ax = plt.subplots(1, 1, figsize=(4, 4))
plot_data(Xdata, ydata, fig=fig, ax=ax)
plt.show()


# Define output labels as quantum state vectors
def density_matrix(state):
    """Calculates the density matrix representation of a state.

    Args:
        state (array[complex]): array representing a quantum state vector

    Returns:
        dm: (array[complex]): array representing the density matrix
    """
    return state * np.conj(state).T


label_0 = [[1], [0]]
label_1 = [[0], [1]]
state_labels = [label_0, label_1]
Xdata.shape, ydata.shape<jupyter_output><empty_output><jupyter_text>Simple classifier with data reloading and fidelity loss~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<jupyter_code>dev = qml.device("default.qubit", wires=1)
# Install any pennylane-plugin to run on some particular backend

@qml.qnode(dev)
def qcircuit(params, x=None, y=None):
    """A variational quantum circuit representing the Universal classifier.

    Args:
        params (array[float]): array of parameters
        x (array[float]): single input vector
        y (array[float]): single output state density matrix

    Returns:
        float: fidelity between output state and input
    """
    for i in range(len(params[0])):
        qml.Rot(*(params[0][i]*x + params[1][i]), wires=0)
        #qml.Rot(*params[1][i], wires=0)
    return qml.expval(qml.Hermitian(y, wires=[0]))


def cost(params, x, y, state_labels=None):
    """Cost function to be minimized.

    Args:
        params (array[float]): array of parameters
        x (array[float]): 2-d array of input vectors
        y (array[float]): 1-d array of targets
        state_labels (array[float]): array of state representations for labels

    Returns:
        float: loss value to be minimized
    """
    # Compute prediction for each input in data batch
    loss = 0.0
    dm_labels = [density_matrix(s) for s in state_labels]
    for i in range(len(x)):
        f = qcircuit(params, x=x[i], y=dm_labels[y[i]])
        loss = loss + (1 - f) ** 2
    return loss / len(x)<jupyter_output><empty_output><jupyter_text>Utility functions for testing and creating batches~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<jupyter_code>def test(params, x, y, state_labels=None):
    """
    Tests on a given set of data.

    Args:
        params (array[float]): array of parameters
        x (array[float]): 2-d array of input vectors
        y (array[float]): 1-d array of targets
        state_labels (array[float]): 1-d array of state representations for labels

    Returns:
        predicted (array([int]): predicted labels for test data
        output_states (array[float]): output quantum states from the circuit
    """
    fidelity_values = []
    dm_labels = [density_matrix(s) for s in state_labels]
    predicted = []

    for i in range(len(x)):
        fidel_function = lambda y: qcircuit(params, x=x[i], y=y)
        fidelities = [fidel_function(dm) for dm in dm_labels]
        best_fidel = np.argmax(fidelities)

        predicted.append(best_fidel)
        fidelity_values.append(fidelities)

    return np.array(predicted), np.array(fidelity_values)


def accuracy_score(y_true, y_pred):
    """Accuracy score.

    Args:
        y_true (array[float]): 1-d array of targets
        y_predicted (array[float]): 1-d array of predictions
        state_labels (array[float]): 1-d array of state representations for labels

    Returns:
        score (float): the fraction of correctly classified samples
    """
    score = y_true == y_pred
    return score.sum() / len(y_true)


def iterate_minibatches(inputs, targets, batch_size):
    """
    A generator for batches of the input data

    Args:
        inputs (array[float]): input data
        targets (array[float]): targets

    Returns:
        inputs (array[float]): one batch of input data of length `batch_size`
        targets (array[float]): one batch of targets of length `batch_size`
    """
    for start_idx in range(0, inputs.shape[0] - batch_size + 1, batch_size):
        idxs = slice(start_idx, start_idx + batch_size)
        yield inputs[idxs], targets[idxs]<jupyter_output><empty_output><jupyter_text>Train a quantum classifier on the circle dataset~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<jupyter_code># Generate training and test data
num_training = 200
num_test = 2000

Xdata, y_train = circle(num_training)
X_train = np.hstack((Xdata, np.zeros((Xdata.shape[0], 1))))

Xtest, y_test = circle(num_test)
X_test = np.hstack((Xtest, np.zeros((Xtest.shape[0], 1))))


# Train using Adam optimizer and evaluate the classifier
num_layers = 10
learning_rate = 0.6
epochs = 10
batch_size = 32

opt = AdamOptimizer(learning_rate, beta1=0.9, beta2=0.999)

# initialize random weights
theta = np.random.uniform(size=(num_layers, 3))
w = np.random.uniform(size=(num_layers, 3))
params = [w, theta]

predicted_train, fidel_train = test(params, X_train, y_train, state_labels)
accuracy_train = accuracy_score(y_train, predicted_train)

predicted_test, fidel_test = test(params, X_test, y_test, state_labels)
accuracy_test = accuracy_score(y_test, predicted_test)

# save predictions with random weights for comparison
initial_predictions = predicted_test

loss = cost(params, X_test, y_test, state_labels)

print(
    "Epoch: {:2d} | Cost: {:3f} | Train accuracy: {:3f} | Test Accuracy: {:3f}".format(
        0, loss, accuracy_train, accuracy_test
    )
)

for it in range(epochs):
    for Xbatch, ybatch in iterate_minibatches(X_train, y_train, batch_size=batch_size):
        params = opt.step(lambda v: cost(v, Xbatch, ybatch, state_labels), params)

    predicted_train, fidel_train = test(params, X_train, y_train, state_labels)
    accuracy_train = accuracy_score(y_train, predicted_train)
    loss = cost(params, X_train, y_train, state_labels)

    predicted_test, fidel_test = test(params, X_test, y_test, state_labels)
    accuracy_test = accuracy_score(y_test, predicted_test)
    res = [it + 1, loss, accuracy_train, accuracy_test]
    print(
        "Epoch: {:2d} | Loss: {:3f} | Train accuracy: {:3f} | Test accuracy: {:3f}".format(
            *res
        )
    )<jupyter_output>Epoch:  0 | Cost: 0.397512 | Train accuracy: 0.440000 | Test Accuracy: 0.414000<jupyter_text>Results~~~~~~~<jupyter_code>print(
    "Cost: {:3f} | Train accuracy {:3f} | Test Accuracy : {:3f}".format(
        loss, accuracy_train, accuracy_test
    )
)

print("Learned weights")
for i in range(num_layers):
    print("Layer {}: {}".format(i, params[i]))


fig, axes = plt.subplots(1, 3, figsize=(10, 3))
plot_data(X_test, initial_predictions, fig, axes[0])
plot_data(X_test, predicted_test, fig, axes[1])
plot_data(X_test, y_test, fig, axes[2])
axes[0].set_title("Predictions with random weights")
axes[1].set_title("Predictions after training")
axes[2].set_title("True test data")
plt.show()<jupyter_output>Cost: 0.065739 | Train accuracy 0.920000 | Test Accuracy : 0.875500
Learned weights
Layer 0: [1.20707984 1.01940528 2.23925141]
Layer 1: [1.066387   1.16369498 1.10294052]
Layer 2: [ 2.02215849 -1.27460831  1.6283752 ]
Layer 3: [1.41653454 0.01641268 0.23605958]
Layer 4: [-0.06459433  1.32273516  1.15865053]
Layer 5: [-0.38669799 -0.05170591  1.25787642]
Layer 6: [ 0.43707799 -0.96031625  0.94220035]
Layer 7: [-0.71395785 -0.07730338 -0.66112228]
Layer 8: [-0.0554786   1.30794512  0.06781628]
Layer 9: [-1.08467869 -0.63313441  0.71598469]
{'fraction_non_alphanumeric': 0.11926227093276766, 'fraction_numerical': 0.031111896765069823, 'mean_word_length': 4.141472281126931, 'pattern_counts': {'":': 2, '<': 25, '<?xml version=': 0, '>': 24, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '492262', 'license': 'MIT', 'n_tokens_mistral': 5410, 'n_tokens_neox': 4894, 'n_words': 1946, 'path': 'PennyLane/Data Reuploading Classifier/tutorial_data_reuploading_classifier.ipynb', 'repo_name': 'Graciaira/quantum_image_classifier', 'size': '17017'}