<jupyter_start><jupyter_text>http://tau-data.id/umi/   Modul 10: Pendahuluan Social Network Analysis(C) <NAME> - 2019tau-data Indonesia ~ https://tau-data.id ~ <EMAIL>  Pendahuluan Social Network Analysis  Workshop Schedule Hari ke-5 (Jumat, 30 Jan 2020)**Social Media & network Analysis*** 09:00 – 11:00 	Topic Modelling* 11:00 – 12:00	Latihan Topic Modelling* 13:00 – 14:00	Pendahuluan Social Network Analytics* 14:00 – 15.00	Centrality Analysis dan Community Detection* 15:00 – 16.00	Latihan Centrality Analysis dan Community Detection Studi Kasus: **Sentiment Analysis dan Topic Modelling Kasus Banjir**  Outline SNA :* Review Graph* Centrality Analysis* Graph Partitioning* Community Detection* Discussion on Final Project Sejarah	1736: <NAME> - Basel, 1707-St. Petersburg, 1786	Mampu mengungkap misteri J<NAME> di Prussia (saat ini Kaliningrad, Russia)  Graph (Definisi)			Suatu Graph G adalah koleksi atau pasangan dari dua himpunan V dan E dengan		V = V(G) = himpunan verteks atau simpul atau node.		E = E(G) = himpunan edge atau ruas atau sisi.	     Graph Types Shortest Path and Cycles Degree/Derajat Vertex Bipartite Graph Graph Representation  Graph Isomorphism Pemodelan Graph untuk Penyelesaian Masalah Graph Applications Graph in Social Media Analytic  Mari mulai dengan Membuat Graph Sederhana (Toy Data) 1. Mendefinisikan Graph 2. Menambahkan Vertex dan Edge 3. Graph Properties 4. Visualisasi Graph<jupyter_code># 1. Mendefinisikan Graph (kosong)
import networkx as nx

G = nx.Graph()
G
V = [1, 2, 7, 9, 12, 19] # Bisa juga string, misal "A" atau nama "Budi"
E = [(1,2), (2,19), (9,2), (9,1), (2,8), (8,10), (12,7),(7,2), (7,9)] # Perhatikan "8" dan "10" TIDAK ADA di V

G.add_nodes_from(V)
G.add_edges_from(E)

print('Banyak vertex = ', G.number_of_nodes())
print('Banyak Edges = ', G.number_of_edges())
# Draw Graph # https://networkx.github.io/documentation/networkx-1.10/reference/drawing.html#layout
import matplotlib.pyplot as plt

pos = nx.spring_layout(G) # Spring LayOut

nx.draw_networkx_nodes(G,pos, alpha=0.1,node_color='blue',node_size=800) # Gambar Vertex
nx.draw_networkx_edges(G,pos,width=1) # Gambar edges
nx.draw_networkx_labels(G,pos) #Gambar Label Nodes
plt.show() # Show the graph
# Ingat Graph Isomorphism?
pos=nx.circular_layout(G) # Circular Layout

nx.draw_networkx_nodes(G,pos) # Gambar Vertex
nx.draw_networkx_edges(G,pos,width=2) # Gambar edges
nx.draw_networkx_labels(G,pos) #Gambar Label Nodes
plt.show() # Show the graph
pos = nx.random_layout(G) # Random

nx.draw_networkx_nodes(G,pos, alpha=0.2,node_color='blue',node_size=600) # Gambar Vertex
nx.draw_networkx_edges(G,pos,width=2) # Gambar edges
nx.draw_networkx_labels(G,pos) #Gambar Label Nodes
plt.show() # Show the graph
shell = [[1,2,7,9],[12,19,8,10]]
pos = nx.shell_layout(G, shell) # Shells

nx.draw_networkx_nodes(G,pos, alpha=0.2,node_color='blue',node_size=600) # Gambar Vertex
nx.draw_networkx_edges(G,pos,width=2) # Gambar edges
nx.draw_networkx_labels(G,pos) #Gambar Label Nodes
plt.show() # Show the graph
pos=nx.spectral_layout(G) # Spectral

nx.draw_networkx_nodes(G,pos, alpha=0.2,node_color='blue',node_size=600) # Gambar Vertex
nx.draw_networkx_edges(G,pos,width=2) # Gambar edges
nx.draw_networkx_labels(G,pos) #Gambar Label Nodes
plt.show() # Show the graph<jupyter_output><empty_output><jupyter_text>PyGraphVis (Demonstration)<jupyter_code>from graphviz import Digraph

f = Digraph('finite_state_machine', filename='fsm.gv')
f.attr(rankdir='LR', size='8,5')
f.attr('node', shape='doublecircle')
f.node('LR_0'); f.node('LR_3')
f.node('LR_4');f.node('LR_8')
f.attr('node', shape='circle')
f.edge('LR_0', 'LR_2', label='SS(B)'); f.edge('LR_0', 'LR_1', label='SS(S)')
f.edge('LR_1', 'LR_3', label='S($end)'); f.edge('LR_2', 'LR_6', label='SS(b)')
f.edge('LR_2', 'LR_5', label='SS(a)'); f.edge('LR_2', 'LR_4', label='S(A)')
f.edge('LR_5', 'LR_7', label='S(b)'); f.edge('LR_5', 'LR_5', label='S(a)')
f.edge('LR_6', 'LR_6', label='S(b)'); f.edge('LR_6', 'LR_5', label='S(a)')
f.edge('LR_7', 'LR_8', label='S(b)'); f.edge('LR_7', 'LR_5', label='S(a)')
f.edge('LR_8', 'LR_6', label='S(b)'); f.edge('LR_8', 'LR_5', label='S(a)')
f
# Hapus Vertex/Edges
G.remove_node(19)
G.remove_edge(1, 2)

pos=nx.spectral_layout(G) # Spectral

nx.draw_networkx_nodes(G,pos, alpha=0.1,node_color='blue',node_size=800) # Gambar Vertex
nx.draw_networkx_edges(G,pos,width=2, alpha=0.1) # Gambar edges
nx.draw_networkx_labels(G,pos, size=1000) #Gambar Label Nodes
plt.show() # Show the graph
# Degree
G.degree[10]<jupyter_output><empty_output><jupyter_text>(Shortest) Path<jupyter_code>print(nx.shortest_path(G, source=9, target=10))


pos=nx.spectral_layout(G)
nx.draw_networkx_nodes(G,pos, alpha=0.1,node_color='blue',node_size=800) # Gambar Vertex
nx.draw_networkx_edges(G,pos,width=2, alpha=0.1) # Gambar edges
nx.draw_networkx_labels(G,pos, size=1000) #Gambar Label Nodes
plt.show() # Show the graph<jupyter_output><empty_output><jupyter_text>Graph From Social mediaMentions, Followers, Friends<jupyter_code># "Contoh" API Keys (Sesuaikan dengan API keys masing-masing)
# consumer_key, consumer_secret, access_token, access_secret
import warnings; warnings.simplefilter('ignore')
import TSutanto_lib as TS

Ck = 'koEdSiixWb5i0dL0eucivnzL'
Cs = 'BlkKig18Qjywwx8LwcbXFZ9FNWASf4rwsKedJh8autMJ5QpNQc'
At = '2214118411-O56nMT4vYEWbdcIkr9bRFxVrBk6Sj15aUOdXuxx'
As = 'yZmK4hxlskXDWxtMabrHi2r3rM5QpNKq8fvEB2k1Ikslt'

# Koneksi ke twitter
twitter = TS.twitter_connect(Ck, Cs, At, As)
topic = '@susipudjiastuti tanya'
n = 999 # Contoh jumlah tweet yang ingin diambil
Tweets = TS.getTweets(twitter, topic, N = n) # lan = 'id' atau 'en' ...
[t['full_text'] for t in Tweets[:7]]
# Draw the Tweet Graph
G = TS.Graph(Tweets,Label = False, layOut='shell')
G = TS.Graph(Tweets,Label = False, layOut='circular')
G = TS.Graph(Tweets,Label = False, layOut='random')
%matplotlib qt
G = TS.Graph(Tweets,Label = False, layOut='shells')
%matplotlib inline 

G = TS.Graph(Tweets,Label = False, layOut='spectral')
G = TS.Graph(Tweets,Label = True, layOut='spring')<jupyter_output><empty_output><jupyter_text>I. Centrality&nbsp;AnalysisBertujuan untuk menemukan pengguna yang paling berpengaruh dalam suatu topik pembicaraan di media sosial. Analisanya biasanya dilakukan melalui data graph dari hubungan jaringan pertemanan (follower/friend) antar pengguna atau komunikasi antar pengguna (mentions). Centrality by Degree Apakah interpretasinya?<jupyter_code># Let's now examine, who are the most "important" users in this Graph?
Gt = TS.Centrality(G, N=10, method='degree', outliers=False, Label = True, layOut='spring')<jupyter_output><empty_output><jupyter_text>Closeness Centrality<jupyter_code>Gt = TS.Centrality(G, N=10, method='closeness', outliers=False, Label = True, layOut='spring')<jupyter_output><empty_output><jupyter_text>Betweenness Centrality<jupyter_code>Gt = TS.Centrality(G, N=10, method='closeness', outliers=False, Label = True, layOut='spring')<jupyter_output><empty_output><jupyter_text>Eigenvector Centrality Digunakan juga oleh Google dalam Algoritma PageRank-nya untuk menentukan halaman web terpenting.<jupyter_code>Gt = TS.Centrality(G, N=10, method='eigen', outliers=False, Label = True, layOut='spring')
# Menggunakan centrality measure untuk merubah tampilan Graph
## Contoh dengan graph generators https://networkx.github.io/documentation/networkx-1.10/reference/generators.html

#g = nx.dorogovtsev_goltsev_mendes_graph(3)
g = nx.karate_club_graph()

pos = nx.spring_layout(g) # Spring LayOut
nx.draw_networkx_nodes(g,pos, alpha=0.2,node_color='blue',node_size=600) # Gambar Vertex
nx.draw_networkx_edges(g,pos,width=2,alpha=0.1) # Gambar edges
nx.draw_networkx_labels(g,pos) #Gambar Label Nodes
plt.show() # Show the graph
# Menggunakan Centrality measure (misal degree) untuk merubah ukuran node

K = 100 # Scale factor
d = nx.degree(g) 
d = [d[node]*K for node in g.nodes()]

pos = nx.spring_layout(g) # Spring LayOut
nx.draw_networkx_nodes(g,pos,node_size=d) # Gambar Vertex
nx.draw_networkx_edges(g,pos,width=2,alpha=0.15) # Gambar edges
nx.draw_networkx_labels(g,pos) #Gambar Label Nodes
plt.show() # Show the graph
# Menggunakan tingkat "kepentingan" sebagai warna
ranking = nx.degree_centrality(g)
warna = list(ranking.values())
print(warna)
pos = nx.spring_layout(g) # Spring LayOut
nx.draw_networkx_nodes(g,pos, node_color=warna,node_size=d) # Gambar Vertex
nx.draw_networkx_edges(g,pos,width=2,alpha=0.1) # Gambar edges
nx.draw_networkx_labels(g,pos) #Gambar Label Nodes
plt.show() # Show the graph<jupyter_output><empty_output><jupyter_text>Summary Coba diskusikan optimal use case dari masing-masing metric ini. II. Community Detection (CD)CD dilakukan pada data jaringan media sosial untuk menemukan komunitas-komunitas dalam pertemanan atau pembicaraan di media sosial. Secara sederhana CD dapat dimengerti sebagai proses "semacam clustering" (pengelompokan) , namun atas suatu graph.  Bipartition (Bisection) Partitioning* * This algorithm paritions a network into two sets by iteratively swapping pairs of nodes to reduce the edge cut between the two sets.* https://www.youtube.com/watch?v=MMlf66PQdN8* Paper: <NAME>.; <NAME> (1970). “An efficient heuristic procedure for partitioning graphs.” Bell Systems Technical Journal 49: 291–307. Oxford University Press 2011.<jupyter_code>B = nx.algorithms.community.kernighan_lin_bisection(g)
B
warna = []
for v in B[0]:
    warna.append(1)
for v in B[1]:
    warna.append(2)
pos = nx.shell_layout(g, B)
nx.draw_networkx_nodes(g,pos, node_color=warna,node_size=d) # Gambar Vertex
nx.draw_networkx_edges(g,pos,width=2,alpha=0.1) # Gambar edges
nx.draw_networkx_labels(g,pos) #Gambar Label Nodes
plt.show() # Show the graph<jupyter_output><empty_output><jupyter_text>Graph Clustering via "Modularity"* Modules biasa disebut juga groups, clusters atau communities* Terdapat berbagai cara dalam menghitung "Modularity" (contoh dibawah)* Graph with high modularity have dense connections between the nodes within "modules" but sparse connections between nodes in different modules.* Salah satu metodenya : Greedy Modularity Maximization (GMM)* GMM begins with each node in its own community and joins the pair of communities that most increases modularity until no such pair exists.* <NAME>., <NAME>., & <NAME>. “Finding community structure in very large networks.” Physical Review E 70(6), 2004.* Other resources for study: https://slideplayer.com/slide/7050174/<jupyter_code># WARNING!!!... Hanya bisa jika networkX versi 2.2 ke atas 
M = nx.algorithms.community.greedy_modularity_communities(g)
print(M)
W = []
warna = 1
for module in M:
    for node in module:
        W.append(warna)
    warna = warna +1
print(W)
pos = nx.shell_layout(g, M)
nx.draw_networkx_nodes(g,pos, node_color=W,node_size=d) # Gambar Vertex
nx.draw_networkx_edges(g,pos,width=2,alpha=0.1) # Gambar edges
nx.draw_networkx_labels(g,pos) #Gambar Label Nodes
plt.show() # Show the graph<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13029583637691747, 'fraction_numerical': 0.029492330168005843, 'mean_word_length': 6.680925666199158, 'pattern_counts': {'":': 0, '<': 50, '<?xml version=': 0, '>': 50, 'https://': 5, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '199374', 'license': 'Apache-2.0', 'n_tokens_mistral': 4532, 'n_tokens_neox': 4210, 'n_words': 992, 'path': 'UMI_10_Pendahuluan Social Network Analytics.ipynb', 'repo_name': 'marhe/UMI', 'size': '11015'}