<jupyter_start><jupyter_text>Using the Jupyter notebookThis is an interactive workbook, where you can run python code. You must have a working python environment with the required libraries installed to use the workbook.The workbook contains both text cells with instructions (like this cell) and code cells, where you can write code and execute it. Some of the code is already filled in. In that case you can execute the code by clicking on the cell and hit &lt;shift&gt; + &lt;enter&gt;. In the assignments, you will need to fill in some code first yourself.    * Use ` + ` to execute a cell and advance tot the next cell.* Use ` + ` to execute a cell and NOT advance to the next celll.  ACT APIMost of the functionality in the ACT platform can be accessed through a REST API. Here are some examples using curl: Get a list of all object types`curl -XGET --header 'ACT-User-ID: 1' http://act-eu1.mnemonic.no/v1/objectType`Example output, using jq and sort to filter/sort the result```$ curl -XGET \  --silent \  --header 'ACT-User-ID: 1' \  http://act-eu1.mnemonic.no/v1/objectType \  | jq '.data[] | .name' \  | sort"accountNumber""asn""campaign""certificate""credential""cryptocurrencyAddress""goal"(...)``` Search for facts with object value "127.0.0.1"`curl -X POST \  --header 'Content-Type: application/json' \  --header 'ACT-User-ID: 1' \  -d '{"objectValue":["127.0.0.1"] }' \  http://act-eu1.mnemonic.no/v1/fact/search`   SwaggerSwagger is used both for API documentation and for experimenting with the API. If you have access to a private instance (not read only), swagger can be accessed at http://[host]/swagger/.  Python APIA high level Python API is created to ease the use of the API. You can find the stable version, including documentation on [pypi](https://pypi.org/project/act-api/) and the development version on [github](https://github.com/mnemonic-no/act-api-python).  Connecting to the APITo connect to the API import the ACT library, and instantiate `act.api.Act`<jupyter_code>import act.api

c = act.api.Act("https://act-eu1.mnemonic.no", user_id = 1, log_level = "warning")

help(c)<jupyter_output><empty_output><jupyter_text>Fact Types / Object TypesWith the client you now have access to most of the API, and we can for instance get a list of all Object Types:<jupyter_code>for obj_type in c.get_object_types():
    print(obj_type.name)<jupyter_output><empty_output><jupyter_text>Assignment 1Write code to get list of all fact types.Hint: look at the code to list object types above and use the function `get_fact_types()` instead.<jupyter_code># Write code to print all fact types here<jupyter_output><empty_output><jupyter_text>Facts / ObjectsFacts/Objects can be retrived by searching type/value or directly through its id (its unique uuid).<jupyter_code>facts = c.fact_search(object_type="ipv4", object_value="172.16.58.3")
print(facts.size)
one_fact = facts[0]<jupyter_output><empty_output><jupyter_text>Above we retrived the first fact in the variable `one_fact`Values can be obtained using dot notation. Objects related to a fact are stored in the objects array and there can be 1 or two values in this array.<jupyter_code># Print first fact

print("""
Fact type: {}
Fact value: {}
Fact id: {}
Source Object type: {}
Source Object value: {}
Destination Object type: {}
Destination Object value: {}
""".format(
  one_fact.type.name,
  one_fact.value,
  one_fact.id,
  one_fact.source_object.type.name,  
  one_fact.source_object.value,  
  one_fact.destination_object.type.name,  
  one_fact.destination_object.value
))<jupyter_output><empty_output><jupyter_text>Assignment 2Below a fact is retrieved by its id.. All facts with the same type/value will have the same id.Q: what is the fact type of this fact?Q: what are the objects connected to this fact?<jupyter_code>fact = c.fact(id="2ad63681-e4fd-46fb-9cf6-dc65cb1943c7").get()

# Enter code to get get fact type and objects for this fact<jupyter_output><empty_output><jupyter_text>Assignment 3In this scenario we have received logs for a period of suspicious activity. You can retrieve the logs with the code below.<jupyter_code>import json
logs = json.loads(open("logs.json").read())<jupyter_output><empty_output><jupyter_text>The log is an array of dictionaries. With the following keys:* timestamp* vendor* product* signature* user* src_ip [optional]* dest_ip [optional]* dest_port [optional]* file_hash [optional] Execute the code below to see what type of logs (vendor/product) we have.<jupyter_code>set(["{}/{}".format(log["vendor"], log["product"]) for log in logs])
# Get list of unique IP addresses
dest_ip = list(set([log.get("dest_ip") for log in logs if log.get("dest_ip")]))
print("Number of IPs: {}".format(len(dest_ip)))

facts = c.fact_search(object_type="ipv4", object_value=dest_ip)
for fact in facts:
    print(fact)<jupyter_output><empty_output><jupyter_text>In the result above there are different type of facts, and not necessarily all of them are signs of suspicious activity. * What type of facts are found?* Which source IPs / users are involved?* Which facts are most likely related to suspicious activity?<jupyter_code># Get source IPs / username of users that are involved<jupyter_output><empty_output><jupyter_text>Other indicatorsIn the above search, we looked at IP addresses. What other indicators do we have in our logs that can be used to search for in the platform? Hint* Look at available object_types in the act platform and compare with what type of logs we have<jupyter_code>## Extract other indicators from the logs and search the platform for these indicators<jupyter_output><empty_output><jupyter_text>Assignment 4 - IncidentWe will no explore how to create facts in the platform. To do this, we you will need to connect to another instance, since act-eu1 is readonly. Either use a private instance, or an instanc that is provided for you during a workshop. act-eu4 which is used below is not publically available.<jupyter_code>cw = act.api.Act(
    "http://act-eu4.mnemonic.no", 
    user_id = 1,
    log_level = "warning",
    requests_common_kwargs = {"auth": ("act", "<PASSWORD>")})<jupyter_output><empty_output><jupyter_text>We do not create objects in the platform, they are created indirectly whenver we add a fact that refer to a object.So what we need to to is to add facts, and they can be added like this:    ```    fact = cw.fact(         fact_type, fact_value) \           .source(source_object_type, source_object_value) \           .destination(destination_object_type, destination_object_value)```The code above only creats a local representation of the fact. To add the fact to the platform, it must be added using `add()`:```    fact.add()```For connecting objects to reports, incidents, etc, we will use the `mentions` fact. Some facts have validators that only accepts specific values, and we can get these parameters from the API:<jupyter_code># Show validator parameters for mentionis Fact
[ft.validator_parameter for ft in cw.get_fact_types() if ft.name == "mentions"][0]<jupyter_output><empty_output><jupyter_text>No we want to add the facts from the analysis above.* Which fact(s) do we want to add?* Hint: you can decide for yourself the value of an incident, as this is normally an internal reference. A god idea would be to put your name as part of the value in this excersise, for instance 'incident-&lt;yourname&gt;'. * Add fact from both uri and content to incident* For URI you also want two additional facts that represents scheme and ipv4* Add the fact(s) to the platform<jupyter_code>### Write code to add fact(s)<jupyter_output><empty_output><jupyter_text>Open a browser with the link to look at your incident in the GUI:        http://act-eu4.mnemonic.no/object-fact-query/incident/incident-[your-name];        (replace incident-[your-name] with the value you specified when creating the fact)  Assignment 5 - Graph queriesGraph queries are implemented in the platform using the [Apache TinkerPop stack](http://tinkerpop.apache.org/). The TinkerPop stack uses a query language, gremlin to support graph traversals. In this assignment we will construct some simple graph queries, using the Python API.The documentation for Apache TinkerPop can be found here:* [http://tinkerpop.apache.org/docs/current/](http://tinkerpop.apache.org/docs/current/)* [http://tinkerpop.apache.org/docs/current/reference/](http://tinkerpop.apache.org/docs/current/reference/)A more approachable book can be found here:* [http://kelvinlawrence.net/book/Gremlin-Graph-Guide.html](http://kelvinlawrence.net/book/Gremlin-Graph-Guide.html)In addtion it is handy to know how object and facts hare mapped to the graph:* object -> nodes* facts -> egdesGraph queries can be executed with the following function in the Python API:`c.object(type, value).traverse(gremin_query)`The (useless and) simple query below will return the starting node:<jupyter_code>obj = c.object("ipv4", "192.168.127.12").traverse('g')
print(obj)<jupyter_output><empty_output><jupyter_text>Let us explore more closely the facts associated with this object by stepping in/out from the initial object.To achive this, we use these gremlin functions:* `in()` - traverse to other objects pointing to us* `out()` - traverse to other objects which we point to* `both()` - travers to other objects either pointing to us or we point toTry using g.in(), g.out() and g.both() from our starting node.<jupyter_code>for res in c.object("ipv4", "192.168.127.12").traverse('g.in()'):
    print(res)<jupyter_output><empty_output><jupyter_text>Q: What do we get here?Compare with what you see here:[https://act-eu1.mnemonic.no/object-fact-query/ipv4/192.168.127.12](https://act-eu1.mnemonic.no/object-fact-query/ipv4/192.168.127.12)Right click on the edges in the graph. Q: What are we missing form the result above?Hint: We have used `in()` to step to the next node, but there also exists a function `inE()`. What is the difference? Try using inE() instead below.<jupyter_code># Write a query that adds the information missing compared to what you see in the GUI<jupyter_output><empty_output><jupyter_text>Assignment 6 - more advanced graph queriesScenario: Forensics has been conducted on a machine, and a suspicious sample with sha256 _95b7c2155d7dc60d097bdabbd5c42934a7828d1ae4a5e4b4119c9a07ebc15a96_ was found. Is this sample related to the same case, and if so, how?Hints:* graph via tool or campaign* bothE().otherV()<jupyter_code># Write a graph query finds any link through tool to the the other finding<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09642213153649046, 'fraction_numerical': 0.014899876625225397, 'mean_word_length': 5.001138952164009, 'pattern_counts': {'":': 2, '<': 70, '<?xml version=': 0, '>': 72, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 7, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '409341', 'license': '0BSD', 'n_tokens_mistral': 3231, 'n_tokens_neox': 3103, 'n_words': 1249, 'path': 'api-notebook.ipynb', 'repo_name': 'mnemonic-no/act-workshop-api', 'size': '10536'}