<jupyter_start><jupyter_text>SD212: Graph mining Review of Lab 7: Soft Clustering This is the review of the open answers of Lab 7 (questions 3, 5 and 10).   Getting started<jupyter_code>from connector import Connector
base_url = 'http://172.16.31.10:8234'
lab_id = 'lab7'<jupyter_output><empty_output><jupyter_text>RatingYou must give a score between 0 and 3, referring to the correct answer and the following indicative rating:* **0:** no answer* **1:** incomplete answer (several points missing)* **2:** almost complete answer (1 point missing or several incomplete points)* **3:** complete answer The score granularity is 0.5 (e.g., you can enter 1.5)In case of multiple scores for the same review, only the last one will be considered.  Question 3Comment the results on the toy graph.  Correct anwser Points to check:1. The soft modularity $Q(\mathbf{p})$ of $\mathbf{p}$ is higher than the modularity $Q(C)$ of $C$. Hence, the **quality** of the soft-clustering induced by $\mathbf{p}$ is **better** than the one of the crisp clustering $C$ according to the modularity quality function.2. The soft modularity arises from **a relaxation** of the classic modularity optimization problem. Thus, the soft modularity function $\mathbf{p} \mapsto Q(\mathbf{p})$ can take higher values than the classic modularity function $C \mapsto Q(C)$. This toy graph example illustrates this phenomenon.  To do* Enter your login* Enter review 0 (your answer) * Enter reviews 1 2 3 (other students' answers)<jupyter_code>login = 'zhuf'
question_id = 3
connector = Connector(base_url, lab_id, login)
get_answer = lambda review_id: connector.get_answer(question_id, review_id)
post_rating = lambda review_id, rating: connector.post_rating(question_id, review_id, rating)
get_answer(0)
# To be modified
rating = 3
post_rating(0, rating)
get_answer(1)
# To be modified
rating = 3
post_rating(1, rating)
get_answer(2)
# To be modified
rating = 3
post_rating(2, rating)
get_answer(3)
# To be modified
rating = 3
post_rating(3, rating)<jupyter_output>Rating posted<jupyter_text>Question 5Interpret the results of multiple runs of Louvain on the Miserables graph.  Correct answerPoints to check:1. The fact that **Magnon often changes clusters between two runs of Louvain** illustrates the fact that a crisp clustering (i.e a partition) does not model well the underlying cluster structure of the Miserables graph.2. This example **motivates the need of soft-clustering techniques** instead crisp clustering methods as Louvain. Indeed, a soft clustering method would be able to model the *mixed membership* of Magnon to multiple clusters.  To do* Enter review 0 (your answer) * Enter reviews 1 2 3 (other students' answers)<jupyter_code>question_id = 5
connector = Connector(base_url, lab_id, login)
get_answer = lambda review_id: connector.get_answer(question_id, review_id)
post_rating = lambda review_id, rating: connector.post_rating(question_id, review_id, rating)
get_answer(0)
# To be modified
rating = 3
post_rating(0, rating)
get_answer(1)
# To be modified
rating = 3
post_rating(1, rating)
get_answer(2)
# To be modified
rating = 3
post_rating(2, rating)
get_answer(3)
# To be modified
rating = 3
post_rating(3, rating)<jupyter_output>Rating posted<jupyter_text>Question 10Interpret the results of the soft clustering algorithm on the OpenFlights graph.  Correct answerPoints to check:1. As expected, the **soft modularity** of the soft clustering obtained after 3 epochs of our algorithm is **higher** than the modularity of the initial clustering obtained with Louvain.2. We observe that the nodes that have a positive probability to belong to more than one cluster often corresponds to **bridges between different regions of the world**. For instance, the **Lisbon Portela Airport**, that has a mixed membership, is a bridge between Brazil (and the rest of South America) and South Europe.3. We remark that the **proportion of nodes with mixed membership is very small**. Therefore, the membership matrix $\mathbf{p}$ is **sparse**.  To do* Enter review 0 (your answer) * Enter reviews 1 2 3 (other students' answers)<jupyter_code>question_id = 10
connector = Connector(base_url, lab_id, login)
get_answer = lambda review_id: connector.get_answer(question_id, review_id)
post_rating = lambda review_id, rating: connector.post_rating(question_id, review_id, rating)
get_answer(0)
# To be modified
rating = 3
post_rating(0, rating)
get_answer(1)
# To be modified
rating = 0
post_rating(1, rating)
get_answer(2)
# To be modified
rating = 3
post_rating(2, rating)
get_answer(3)
# To be modified
rating = 0
post_rating(3, rating)<jupyter_output>Rating posted
{'fraction_non_alphanumeric': 0.08354866494401378, 'fraction_numerical': 0.021102497846683894, 'mean_word_length': 5.71242774566474, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '589200', 'license': 'Apache-2.0', 'n_tokens_mistral': 1438, 'n_tokens_neox': 1318, 'n_words': 590, 'path': 'SD212/lab_session/review7.ipynb', 'repo_name': 'zhufangda/Telecom-Paristech', 'size': '4646'}