<jupyter_start><jupyter_text>Support Vector Machines Classification Using SVMThis tutorial is based on an EPFL machine learning course.Load dataset. We will use the CERN dataset, available from a previous EPFL machine learning challenges. You can download the data here. https://inclass.kaggle.com/c/epfml-project-1/data<jupyter_code>from proj1_helpers import load_csv_data
DATA_TRAIN_PATH = '../data/train.csv' # TODO: download train data and supply path here 
y, x, ids = load_csv_data(DATA_TRAIN_PATH)
# TODO: convert labels to -1,1 ?

## Note: This is the raw dataset, you can also work with your modified features if you prefer
def calculate_cost(y, x, w, lambda_):
    """compute the full cost (the primal objective), that is loss plus regularizer."""
    # Here x is the full dataset matrix, and y are the corresponding +1 or -1 labels
    # ***************************************************
    # INSERT YOUR CODE HERE
    # TODO
    # ***************************************************
    raise NotImplementedError<jupyter_output><empty_output><jupyter_text>Stochastic Gradient Descent for SVM Compute the (stochastic) subgradient for the n-th summand of the SVM optimization objective<jupyter_code>def calculate_gradient(y, x, w, lambda_, n):
    """compute the stochastic gradient of loss plus regularizer."""
    # Here x is one datapoint, and y is the corresponding +1 or -1 label
    # 
    # ***************************************************
    # INSERT YOUR CODE HERE
    # TODO
    # ***************************************************
    # Be careful about the constant N(size) term! The complete objective for SVM is a sum, not an average as in earlier SGD examples!
    raise NotImplementedError<jupyter_output><empty_output><jupyter_text>Implement stochastic gradient descent: Pick a data point uniformly at random and update w based on the gradient for the n-th summand of the objective<jupyter_code>def sgd_for_svm_demo(y, x):
    # ***************************************************
    # INSERT YOUR CODE HERE
    # classify the data by SGD for SVM: TODO
    # ***************************************************
    max_iter = 10000
    gamma = 0.001               # Step-size
    lambda_ = 1.0 / y.shape[0]  # or set to a different value, try cross-validation!
    
    w = np.zeros((x.shape[1], 1))
    
    for iter in range(max_iter):
        # n = sample one data point uniformly at random data from x
        raise NotImplemented
        # loss = TODO 
        # grad = TODO don't forget about the regularizer term
        # w = update w
        raise NotImplemented
        
        if iter % 1000 == 0:
            print("Current iteration={i}, the loss={l}".format(i=iter, l=loss))
    
    print("Objective = {l}".format(l=calculate_cost(y, x, w, lambda_)))

sgd_for_svm_demo(y, x)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.18032212885154061, 'fraction_numerical': 0.009453781512605041, 'mean_word_length': 3.8260135135135136, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '411580', 'license': 'MIT', 'n_tokens_mistral': 837, 'n_tokens_neox': 782, 'n_words': 295, 'path': 'sds/lectures/Exercise/template/Exercise6.ipynb', 'repo_name': 'AlexanderChristian/private_courses', 'size': '2856'}