<jupyter_start><jupyter_text>Monotonic ClassifiersSome classifiers should never "flip-flop" between classes. For example, consider the following classifier that labels system call traces from programs as being benign or malicious programs. No matter how many benign instructions are added to a malicious program, it should never trick the classifier into thinking it is benign.The classifier below takes sequences of system calls obtained from execution traces from malicious and benign programs. Treating each execution trace as a document, we extract a tf-idf[1] vector for feature extraction. Code below is provided that:1. Grabs ground truth traces1. Vectorizes them with tf-idf1. Performs 10-fold cross validation1. Trains a Logistic Regression modelYour task as a malware author yourself, is to find direct and indirect ways to break this model so antivirus software cannot detect your code. Approach these tasks in three chunks:1. Manually manipulate a malicious feature vector such that the classifier mistakenly labels it malicious. If you successfully do this once, create a function that given a benign feature vector returns a new one that will be classified as benign.1. Identify features that, given your knowledge of what monotonic classifiers try to solve, could be used to "flip" a malicious program into benign one.1. Using the aforementioned features, write a function that transforms a malicious syscall trace (appending is fine) to be classified as benign.1. Modify the classifier so these features can no longer be used to "flip" a malicious trace. There's a quick'n'dirty way to do this, but more sophisticated[2] and robust[3] techniques exist if monotonicity is an important feature that your classifier needs. References* [1] https://en.wikipedia.org/wiki/Tf%E2%80%93idf* [2] https://arxiv.org/pdf/1804.03643.pdf* [3] https://www.slideshare.net/MSbluehat/bluehat-v17-detecting-compromise-on-windows-endpoints-with-osquery  Utility Functions<jupyter_code>import os
import sys
import fnmatch
import random
import itertools
from collections import Counter

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression, Lasso
from sklearn.model_selection import cross_val_score, cross_val_predict
from sklearn.externals import joblib
import numpy as np
import pandas as pd
import seaborn
import matplotlib.pyplot as plt

## utils
def rwalk(directory, pattern):
    """Recursively search "directory" for files that match the Unix shell-style
    wildcard given by "pattern" (like '*.mp3'). Returns matches as a generator."""
    for root, dirnames, filenames in os.walk(directory):
        for filename in fnmatch.filter(filenames, pattern):
            yield os.path.join(root, filename)

def gettraces(benignpath='../data/01-monotonic-classifiers/benign-traces',
              malpath='../data/01-monotonic-classifiers/malicious-traces'):
    return list(rwalk(malpath, '*.trace')), list(rwalk(benignpath, '*.trace'))

def get_random_malicious_trace(malpath='../data/01-monotonic-classifiers/malicious-traces'):
    """Grab the text of a random malicious system call trace."""
    mal, _ = gettraces(malpath=malpath)
    with open(random.choice(mal)) as f:
        return f.read()<jupyter_output><empty_output><jupyter_text>Use the TfidfVectorizer to vectorize ground truthThe following extracts vectors from each benign and malicious execution trace and returns four values:1. `X`: the feature vectors1. `y`: the class labels1. `terms`: the list of labels (0 is benign, 1 is malicious)1. `vectorizer`: a TfidfVectorizer which is fit to the terms in the ground truth and can be used to fit new syscall traces with `vectorizer.transform([trace1, trace2, ..., traceN])`<jupyter_code>def vectorize(featuredir='../data/01-monotonic-classifiers/feature-vectors'):
    pos_traces, neg_traces = gettraces()
    pos_y = [1 for _ in pos_traces]
    neg_y = [0 for _ in neg_traces]
    docs = [open(x).read() for x in pos_traces + neg_traces]
    y = np.array(pos_y + neg_y)
    
    vectorizer = TfidfVectorizer()
    X = vectorizer.fit_transform(docs)
    terms = np.asarray(vectorizer.get_feature_names())
    return X, y, terms, vectorizer

X, y, terms, vectorizer = vectorize()
X
terms
def _heatmap(crosstab):
    plt.clf()
    p = seaborn.heatmap(crosstab, square=True)
    plt.tight_layout()
    plt.show()

def _cv(X, y, k, name, clf, csvname, modeldir=None, terms=None, resultdir=None):
    print('## %s' % name)
    print('### Cross Validation')
    print('`%s`' % str(cross_val_score(clf, X, y, cv=k)))
    print('### CV Confusion Matrix')
    y_pred = cross_val_predict(clf, X, y, cv=k)
    print('```')
    print(pd.crosstab(y, y_pred, rownames=['True'], colnames=['Predicted']))
    print('```')
    _heatmap(pd.crosstab(y, y_pred, rownames=['True'], colnames=['Predicted'],
                         normalize='index'))
    clf.fit(X, y)
    return clf
clf = _cv(X, y, 10, 'name', LogisticRegression(solver='lbfgs'), 'foo.csv', modeldir='../work', terms=terms, resultdir='../work')
clf.classes_
clf.coef_[0]
terms<jupyter_output><empty_output><jupyter_text>Direct AttackYou (somehow) have direct access to the feature vectors. Don't ask how, celebrate! Traces are stored in `../data/01-monotonic-classifiers/{malicious,benign}-traces/`. I recommend writing a functions that returns a feature vector given the path of a trace, comparing benign and malicious vectors, and manually transforming a malicious vector to a benign vector. If you have time, write a function to perform this manipulation.  Retrieve and Examine Vectors<jupyter_code># grab trace, vectorize, classify, manipulate, re-classify
def get_feature_vector(path, vectorizer):
    with open(path) as f:
        return vectorizer.transform([f.read()])
mal_fv = get_feature_vector('../data/01-monotonic-classifiers/malicious-traces/0999.trace', vectorizer)
ben_fv = get_feature_vector('../data/01-monotonic-classifiers/benign-traces/0999.trace', vectorizer)
print('malicious\t\t\tbenign')
for mal, ben in zip(str(mal_fv).split('\n'), str(ben_fv).split('\n')):
    print('%s\t%s' % (mal, ben))
print('\nMalicious field larger than benign')
print(mal_fv > ben_fv)
print('\nMalicious field smaller than benign')
print(mal_fv < ben_fv)<jupyter_output>malicious			benign
  (0, 15)	0.334132498889	  (0, 15)	0.0417904620039
  (0, 14)	0.341169421524	  (0, 14)	0.0459695082042
  (0, 13)	0.398678065123	  (0, 13)	0.357830830908
  (0, 12)	0.325882313731	  (0, 12)	0.0397009389037
  (0, 11)	0.359125706867	  (0, 11)	0.0485814120795
  (0, 10)	0.0361552231914	  (0, 10)	0.32387608053
  (0, 9)	0.036397875696	  (0, 9)	0.344771311532
  (0, 8)	0.0327580881264	  (0, 8)	0.32909988828
  (0, 7)	0.0344566556589	  (0, 7)	0.32596560363
  (0, 6)	0.0356699181821	  (0, 6)	0.338502742231
  (0, 5)	0.341169421524	  (0, 5)	0.0438799851041
  (0, 4)	0.0334860456403	  (0, 4)	0.32701036518
  (0, 3)	0.0334860456403	  (0, 3)	0.316040368904
  (0, 2)	0.0334860456403	  (0, 2)	0.315517988129
  (0, 1)	0.3503902167	  (0, 1)	0.0407457004538
  (0, 0)	0.359368359372	  (0, 0)	0.0360442734783

Malicious field larger than benign
  (0, 0)	True
  (0, 1)	True
  (0, 5)	True
  (0, 11)	True
  (0, 12)	True
  (0, 13)	True
  (0, 14)	True
  (0, 15)	True

Malicious field smaller than benign
  ([...]<jupyter_text>Interesting! Note the indices where the malicious field is smaller than benign. Let's increase these indices in a malicious vector and see if that causes the class labels to flip.<jupyter_code>def feature_vector_malicious_to_benign(fv):
    """This adds 1 to all indices in a malicious vector that are smaller than the benign vector to 
    transform it into a vector that will classify as benign."""
    delta = np.array([0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0])
    return fv + delta
print(clf.predict(mal_fv))
print(clf.predict(feature_vector_malicious_to_benign(mal_fv)))<jupyter_output>[1]
[0]<jupyter_text>Great! We have successfully switched the label from malicious (1) to benign (0). Let's see if our function works on all the malicious samples now.<jupyter_code>maltraces, _ = gettraces()
orig = []
mutated = []
for maltrace in maltraces:
    fv = get_feature_vector(maltrace, vectorizer)
    orig.append(clf.predict(fv))
    mutated.append(clf.predict(feature_vector_malicious_to_benign(fv)))
print(Counter([x[0] for x in orig]))
print(Counter([x[0] for x in mutated]))<jupyter_output>Counter({1: 1000})
Counter({0: 1000})<jupyter_text>Huzzah! Our malware is now undetectable!  Indirect Attack Part 1 (Find terms)So we've made our malware undetectable (above), but we did so by directly manipulating the feature vector. This assumes a powerful and/or dedicated attacker and we're a bit lazy. How can we alter our malware's _behavior_ such that our malware is classified as benign software? Let's identify `terms` that are more likely to be associated with benign software than malicious software by finding _negative_ coefficients in the model.<jupyter_code>sorted(list(zip(clf.coef_[0], terms)))<jupyter_output><empty_output><jupyter_text>A-ha! These negatively weighted features will do nicely. This means we can simply append a lot of `ntreadfile` calls to our malware and it'll push the classification towards benign. Let's try it for one, random malicious trace. Consider that we can read as many files as we want in code and _not_ disrupt the malicious behavior that occurred earlier. Don't be shy about adding a lot of them!<jupyter_code>foo = get_random_malicious_trace()
v1 = vectorizer.transform([foo])
v2 = vectorizer.transform([foo + '\n' + '\n'.join(itertools.repeat('ntreadfile', 100000))])
clf.predict(v1)
clf.predict(v2)<jupyter_output><empty_output><jupyter_text>OK! Looks like we can force the classification to flip by only indirectly manipulating the feature vector. Do you think this case is more or less reasonable than directly manipulating feature vectors?  Indirect Attack Part 2 (Adversarial Sample Generation)<jupyter_code>def gen_adversarial_sample(path):
    with open(path) as f:
        s = f.read()
        numsyscalls = len(s.split())
        s_benign = s + '\n' + '\n'.join(itertools.repeat('ntreadfile', numsyscalls))
        return s, s_benign
s, s_benign = gen_adversarial_sample('../data/01-monotonic-classifiers/malicious-traces/0999.trace')
print(len(s))
print(len(s_benign))
# It works!
clf.predict(vectorizer.transform([s, s_benign]))
orig = []
mutated = []
for maltrace in maltraces:
    mal, adv = gen_adversarial_sample(maltrace)
    malvec = vectorizer.transform([mal])
    advvec = vectorizer.transform([adv])
    orig.append(clf.predict(malvec))
    mutated.append(clf.predict(advvec))
print(Counter([x[0] for x in orig]))
print(Counter([x[0] for x in mutated]))<jupyter_output>Counter({1: 1000})
Counter({0: 1000})<jupyter_text>Not only does it work on a single example, but it breaks everything we saw in the ground truth!  Make Classifier MonotonicA simple way to make a classifier monotonic for our purposes is to not allow attackers to abuse negative coefficient features. Add a vector to `clf.coef` such that there are no longer negative coefficients, and demonstrates this defeats your adversarial generation function from the previous exercise. We can accomplish this by:* Identify the index in `clf.coef_` for the feature you abused* Set its weight to `0.0`* Rerunning our classification examples from above<jupyter_code>clf.coef_[0][clf.coef_.argmin()] = 0.0
maltraces, bentraces = gettraces()
orig = []
mutated = []
benign = []
for maltrace in maltraces:
    mal, adv = gen_adversarial_sample(maltrace)
    malvec = vectorizer.transform([mal])
    advvec = vectorizer.transform([adv])
    orig.append(clf.predict(malvec))
    mutated.append(clf.predict(advvec))
print('Original malicious traces')
print(Counter([x[0] for x in orig]))
print('Adversarial malicious traces')
print(Counter([x[0] for x in mutated]))

# What about the benign examples?
for bentrace in bentraces:
    benvec = get_feature_vector(bentrace, vectorizer)
    benign.append(clf.predict(benvec))
print('Original benign traces')
print(Counter([x[0] for x in benign]))<jupyter_output>Original malicious traces
Counter({1: 1000})
Adversarial malicious traces
Counter({1: 1000})
Original benign traces
Counter({0: 1000})
{'fraction_non_alphanumeric': 0.10684596577017115, 'fraction_numerical': 0.05509372453137734, 'mean_word_length': 4.821157495256167, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '553886', 'license': 'MIT', 'n_tokens_mistral': 4502, 'n_tokens_neox': 3951, 'n_words': 1334, 'path': 'solutions/Monotonic Classifiers.ipynb', 'repo_name': 'suryapa1/hands-on-adversarial-machine-learning', 'size': '12270'}