<jupyter_start><jupyter_text>특정 위치를 클릭하는 동작이 있어야 넘어감 새로운 글이 올라올때 주의할 부분 필요<jupyter_code>from selenium import webdriver
browser = webdriver.Chrome('/Users/klee30810/Downloads/chromedriver')
url = 'https://www.instagram.com'
browser.get(url)<jupyter_output><empty_output><jupyter_text>검색 페이지 접속하기검색어 입력칸 선택 -> 검색어 임렵하기 -> 태그리스트 중에서 태그 선택하게 -> 태그 클릭하기 -> 페이지<jupyter_code>search = 'adsk'
url = f'https://www.instagram.com/explore/tags/{search}/'
browser.get(url)

first_article = browser.find_elements_by_css_selector('div._9AhH0')[0]   # 
first_article.click()

# 다음 게시물 이동 클릭
next_url = 'a.coreSpriteRightPaginationArrow'
next_button = browser.find_elements_by_css_selector(next_url)[0].click()

next_button.click()
time.sleep(2)

# 본문 내용 수집
from bs4 import BeautifulSoup

html = browser.page_source
soup = BeautifulSoup(html, 'html.parser')

content = soup.select('div.C4VMK > span')[0].text
print(content)


import time

for i in range(10):
    html = browser.page_source
    soup = BeautifulSoup(html, 'html.parser')

    content = soup.select('div.C4VMK > span')[0].text
    print(content)

    next_url = 'a.coreSpriteRightPaginationArrow'
    next_button = browser.find_elements_by_css_selector(next_url)[0]

    next_button.click()
    time.sleep(2)<jupyter_output>Here we go #AU2020. Super excited to connect virtually with everyone @autodesk @autodeskuniversity @adskfusion360 this week. Thank you #expertelite for the cool swag and never-ending support. #autodeskuniversity #adsk #autodesklife #futureofmaking #design #process #cad #fusion360
Found some old screenshots of  constructing a prototype for the thesis project 👉🏼.....#design #footwear #designsketch #ckinspiration #kicks #id #footweardesign #shoedesign #designprocess #kickstagram #lastcomesfirst #prototype #3dprint #3dprinting #footwear #conceptkicks #3dmodeling #digitalmodeling #fusion360 #autodesk #autodeskfusion360 #adsk #adskfusion360 #kicksoftheday #callingallcreators #lacelessdesign #academyofartuniversity #academyuid #thesisproject #creativeprocess #shoemodeling
AUTODESK - КРУПНЕЙШИЙ РАЗРАБОТЧИК В СЕКТОРЕ 3D-МОДЕЛИРОВАНИЯ 🏆ПО для управления производством, а также программы для моделирования объектов, рисования и визуализации - стали одними из самых востребованных в сфере IT-технолог[...]<jupyter_text>좋아요 수 가져오기<jupyter_code>from bs4 import BeautifulSoup

html = browser.page_source
soup = BeautifulSoup(html, 'html.parser')

content = soup.select('div.C4VMK > span')[0].text
 
# likes = soup.select('a.zV_Nj')[0].text.replace('좋아요 ','').replace('개', '')
try:
    likes = soup.select('a.zV_Nj')[0].text[4:-1]
    views = 0
except : # 동영상
    try:
        views = soup.select('span.vCOH2')[0].text[3:-1]
        likes = 0
    except:
        likes, views = 0,0
likes
try:
    likes = soup.select('a.zV_Nj')[0].text[4:-1]
except : # 동영상
    likes = 0

try:
    views = soup.select('span.vCOH2')[0].text[3:-1]
except:
    views
for i in range(10):
    html = browser.page_source
    soup = BeautifulSoup(html, 'html.parser')

    try:
        content = soup.select('div.C4VMK > span')[0].text
    except : # 본문이 없다면
        content = ''
    
    print(content)
    
    try:
        likes = soup.select('a.zV_Nj')[0].text[4:-1]
    except : # 동영상
        likes = 0
    try:
        views = soup.select('span.vCOH2')[0].text[3:-1]
    except:
        views

    print(likes, views)

    next_url = 'a.coreSpriteRightPaginationArrow'
    next_button = browser.find_elements_by_css_selector(next_url)[0]

    next_button.click()
    time.sleep(2)<jupyter_output>По итогам недели с 22 по 26 ноября среди акций, входящих в листинг 1-го уровня "Московской биржи",🟢лидеров роста на этой неделе НЕТ🔴просели бумаги "HEADHUNTER" (#HHRU) (-12.84%), "YANDEX" (#YNDX) (-11.51%), "MAIL.RU" (#MAIL) (-9.41%), "ПОЛЮС" (#PLZL) (-9.36%), "OZON" (#OZON) (-8.17%);🟢из иностранных акций лучше рынка оказались"MODERNA" (#MRNA) (+35.7%), "XPENG DR A" (#XPEV) (+14.7%), "HP" (#HPQ) (+13.97%), "TAL" (#TAL) (+11.4%), "INCYTE" (#INCY) (+10.42%);🔴из иностранных бумаг хуже рынка выглядели "GAP" (#GPS) (-26.71%), "AUTODESK" (#ADSK) (-17.6%), "MACY'S" (#M) (-13.53%), "BESTBUY" (#BBY) (-12.99%), "CINEMARK" (#CNK) (-11.92%)#мосбиржа #ценныебумаги #фондовыйрынок #инвестиции #интересное #trading #финансы
0 0
По итогам недели с 22 по 26 ноября среди акций, входящих в листинг 1-го уровня "Московской биржи",🟢лидеров роста на этой неделе НЕТ🔴просели бумаги "HEADHUNTER" (#HHRU) (-12.84%), "YANDEX" (#YNDX) (-11.51%), "MAIL.RU" (#MAIL) (-9.41%), "ПОЛЮС" (#PLZL) (-9.36%), "OZON" (#OZON) (-8[...]<jupyter_text>작성 일시 가져오기<jupyter_code>date = soup.select('time._1o9PC.Nzb55')[0]['datetime'][:10]
date
for i in range(10):
    html = browser.page_source
    soup = BeautifulSoup(html, 'html.parser')

    try:
        content = soup.select('div.C4VMK > span')[0].text
    except : # 본문이 없다면
        content = ''
    
    print(content)
    
    try:
        likes = soup.select('a.zV_Nj')[0].text[4:-1]
    except : # 동영상
        likes = 0
    try:
        views = soup.select('span.vCOH2')[0].text[3:-1]
    except:
        views

    date = soup.select('time._1o9PC.Nzb55')[0]['datetime'][:10]

    print(likes, views, date)

    next_url = 'a.coreSpriteRightPaginationArrow'
    next_button = browser.find_elements_by_css_selector(next_url)[0]

    next_button.click()
    time.sleep(2)<jupyter_output>Stock deals is the way...#lotr #lotrmemes #lordoftherings #middleearth #urukhai #orc #ugluk #merrybrandybuck #peregrintook #merryandpippin #pippin #meatsbackonthemenuboys #blackfriday #blackfridaydeals #adsk #autodesk #stocks #stockmarket #investing #invest #buythedip #buylowsellhigh
12 0 2021-11-25
#shie#mct#abriss#lostplaces #verlasseneorte#lostplacesgraffiti #graff#coulor#graffitiart #magdeburg#lane#adsk#writing #stylewriting
42 0 2021-11-25
Last-close results: #SPY is Bullish, bolded, confidence 65.4%; Stock pick: #CSX, #HRL, #PIPP, #VMW, #ADSK, #APH, #ROK, #TEAM, Daily #market prediction and #stock pick @xhtliang120 or TaoCapMktModel on #Twitter.
0 0 2021-11-25
Last-close results: #SPY is Bullish, bolded, confidence 65.4%; Stock pick: #CSX, #HRL, #PIPP, #VMW, #ADSK, #APH, #ROK, #TEAM, Daily #market prediction and #stock pick @xhtliang120 or TaoCapMktModel on #Twitter.
0 0 2021-11-25
Real-time for next day (assuming that #SPY carrys the current price to the next day): #SPY is Bulli[...]<jupyter_text>여러 글 반복 작업- 셀레늄 작업중에는 건드리지 말고 전체창으로 놔둬라- time.sleep(2) : 진행해보면서 사이트마다 다를 수 있으므로 숫자를 설정할 것<jupyter_code>search = 'adsk'
url = f'https://www.instagram.com/explore/tags/{search}/'
browser.get(url)
time.sleep(2)

first_article = browser.find_elements_by_css_selector('div._9AhH0')[0]   # 
first_article.click()
time.sleep(2)

howmany = 10
results = []

for i in range(howmany):
    html = browser.page_source
    soup = BeautifulSoup(html, 'html.parser')

    try:
        content = soup.select('div.C4VMK > span')[0].text
    except : # 본문이 없다면
        content = ''
    
    try:
        likes = soup.select('a.zV_Nj')[0].text[4:-1]
    except : # 동영상
        likes = 0
    try:
        views = soup.select('span.vCOH2')[0].text[3:-1]
    except:
        views

    date = soup.select('time._1o9PC.Nzb55')[0]['datetime'][:10]

    print(content, likes, views, date)
    data = [content, likes, views, date]
    results.append(data)

    next_url = 'a.coreSpriteRightPaginationArrow'
    next_button = browser.find_elements_by_css_selector(next_url)[0]

    next_button.click()
    time.sleep(2)  # 새로운 데이터가 들어오기 위해서는 일정 시간이 필요

# close_button = ##
# close_button.click()<jupyter_output>#Hi-tech#Revit_2021.1#ADSK#3D'sMax_2021.1#Corona_7.1#no_photoshop 22 0 2021-08-19
Autodesk Inventor - Is it just me?.#dollypartonchallenge#thedollypartonchallenge#dollypartonmemechallenge#dollypartonchallenge😂#linkedinfacebookinstagramtinder#Autodesk#ADSK#ADSKInventor#AutodeskInventor#AutodeskInventorProffesional#AutodeskInventor2020#AutodeskLife#CAD#CADgeek 107 0 2020-01-29
We have a proud kid showing off his #tinkercad model from the #adsk-parent class by colleague <NAME>. #autodesklife 16 0 2020-03-25
It's easy to make aesthetic, functional and great looking parts with 3d printing. With a bit of finishing this piece looks like it was molded! 45 0 2020-09-30
Here we go #AU2020. Super excited to connect virtually with everyone @autodesk @autodeskuniversity @adskfusion360 this week. Thank you #expertelite for the cool swag and never-ending support. #autodeskuniversity #adsk #autodesklife #futureofmaking #design #process #cad #fusion360 119 0 2020-11-16
Der Zeit voraus aber[...]<jupyter_text>엑셀 저장<jupyter_code>import pandas as pd

df = pd.DataFrame(results)
df.columns = [content, likes, views, date]
df.to_excel('./adsk_crawling.xlsx', index=False)

seach_list = ['adsk', 'fomc', 'crm', 'docu']
for search in search_list:
    url = f'https://www.instagram.com/explore/tags/{search}/'
    browser.get(url)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13854271920961042, 'fraction_numerical': 0.03963175030874593, 'mean_word_length': 4.418491484184915, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 32, 'https://': 4, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '643231', 'license': 'MIT', 'n_tokens_mistral': 3900, 'n_tokens_neox': 3898, 'n_words': 773, 'path': 'crawling/instagram_crawling.ipynb', 'repo_name': 'klee30810/TIL_CS', 'size': '8920'}