<jupyter_start><jupyter_text>瀏覽器開發者工具介紹* 知道瀏覽器的開發者工具* 能夠使用瀏覽器的開發者工具觀察資料  作業目標利用開發者工具去尋找「 https://www.zhihu.com/explore 」和「 http://aicoin.cn 」網站的 HTTP 網址，以及他所帶的 Headers 分別是哪些，試著用 request 去發送有沒有帶 Headers 的差異。  1. https://www.zhihu.com/explore<jupyter_code>import requests
url = 'https://www.zhihu.com/explore'
r = requests.get(url)
r.encoding = 'utf-8'
print(r.text)
### 利用查看開發者工具觀察 Headers

```
'''

'''
```
import requests
url = 'http://www.zhihu.com'

headers = {
    'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
    'accept-encoding': 'gzip, deflate, br',
    'accept-language': 'zh-TW,zh;q=0.9,en-US;q=0.8,en;q=0.7',
    'cache-control': 'max-age=0',
    'cookie': '_zap=4be322d1-81a2-42f8-a147-1f01e3a79c74; d_c0="AMBUimIzexKPTjf-SJGc-460hQm6D9go_YE=|1610285505"; _xsrf=060db494-b5e5-403b-a66c-1912de630b6e; SESSIONID=dlt9UNDRcGutphFgnnu4H4RGepNX7eITmeABG0Vbx9x; JOID=VVsVA08YM_Br8RLpQB_k60KTJpVVeFyEVqdYrhJ4cIYfhXmrJMFI8gz2FepEhMvU3QJA9uuJUov-Gc2Cciinasw=; osd=Wl4UBUoXNvFt9B3sQRnh5EeSIJBafV2CU6hdrxR9f4Meg3ykIcBO9wPzFOxBi87V2wdP8-qPV4T7GMuHfS2mbMk=; Hm_lvt_98beee57fd2ef70ccdd5ca52b9740c49=1610285509,1610689979,1611036896; Hm_lpvt_98beee57fd2ef70ccdd5ca52b9740c49=1611036896; l_n_c=1; q_c1=5e5201b4bca243549e28ad47225b6fb5|1611037072000|1611037072000; r_cap_id="OGRiYWI2YWVkMmQ4NGU1NmFiYzVkNGRmYzRiZmQ2YWU=|1611037072|2614f9810e3ef6bb506d85dff0185b7fcaed30a2"; cap_id="MjM3MDZmZjA4NjQzNDViM2JmMjliZWY4MTNlMGM0OTk=|1611037072|fa0aa96ee3bb255be7633cb847535dd5c8feff7c"; l_cap_id="N2QwZGQ2NmJhMTI3NGQzNzg2NDQyOTc5YTRiYWExMjA=|1611037072|0ae7387b57b0e20f309d2a0bc4c304a8683863f2"; n_c=1; KLBRSID=b33d76655747159914ef8c32323d16fd|1611037079|1611036882; __utma=51854390.1163285783.1611037080.1611037080.1611037080.1; __utmb=51854390.0.10.1611037080; __utmc=51854390; __utmz=51854390.1611037080.1.1.utmcsr=localhost:8888|utmccn=(referral)|utmcmd=referral|utmcct=/; __utmv=51854390.010--|3=entry_date=20210119=1',
    'referer': 'http://localhost:8888/',
    'sec-ch-ua': '"Google Chrome";v="87", "\"Not;A\\Brand";v="99", "Chromium";v="87"',
    'sec-ch-ua-mobile': '?1',
    'sec-fetch-dest': 'document',
    'sec-fetch-mode': 'navigate',
    'sec-fetch-site': 'cross-site',
    'sec-fetch-user': '?1',
    'upgrade-insecure-requests': '1',
    'user-agent': 'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Mobile Safari/537.36'
}

r = requests.get(url, headers=headers)

r.encoding = 'utf-8'
print(r.text[0:600])<jupyter_output><!doctype html>
<html lang="zh" data-hairline="true" data-android="true" data-theme="light"><head><meta charSet="utf-8"/><title data-react-helmet="true">知乎 - 有问题，上知乎</title><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=0,viewport-fit=cover"/><meta name="renderer" content="webkit"/><meta name="force-rendering" content="webkit"/><meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"/><meta name="google-site-verification" content="FTeR0c8arOPKh8c5DYh_9uu98_zJbaWw53J-Sch9MTg"/><meta name="description" property="og:description" content="知乎，中文<jupyter_text>2. http://aicoin.cn<jupyter_code>import requests

url = 'http://aicoin.cn/'
r = requests.get(url)
r.encoding = 'utf-8'
print(r.text)<jupyter_output><!DOCTYPE html><html style="height:100%;width:100%"><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8" /><meta http-equiv="Server" content="CloudWAF" /><title id="title">访问被拦截！</title><script>function bindall() {var requestid = "30-31-24-20210119142722-a0a30d43";if(requestid==null || requestid == ""){return;}document.getElementById("d").innerText = requestid;}</script></head><body onload="bindall()" style="height:100%;width:100%;margin:0px;font-family:Microsoft yahei"><div><style type="text/css">	.button{float:right;margin-right:2rem;text-decoration:none;background:white;color:#e94d4c;padding: 5px 15px 5px 15px;font-size:1rem;font-family: 微软雅黑,宋体,Arial,Helvetica,Verdana,sans-serif;font-weight:bold;border-radius:0rem;border: none;-webkit-transition:all linear 0.30s;-moz-transition:all linear 0.30s;}.button:hover{text-decoration:none;background:#e94d4c;color:#f2f2f2;}</style><input type="button" class="button" onclick="changeLanguageZh()" value="中文"/><input type="bu[...]<jupyter_text>利用查看開發者工具觀察 Headers```'''放上截圖'''```<jupyter_code>import requests
url = 'http://aicoin.cn/'

headers = {
#     'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3',
#     'Accept-Encoding': 'gzip, deflate, br',
#     'Accept-Language': 'zh-TW,zh;q=0.9,en-US;q=0.8,en;q=0.7,zh-CN;q=0.6,ja;q=0.5',
#     'Cache-Control': 'max-age=0',
#     'Connection': 'keep-alive',
#     'Cookie': 'HWWAFSESID=9ac609dc8860802e9e; HWWAFSESTIME=1555660103975; _ga=GA1.3.94203977.1555660106; _gid=GA1.3.1203042209.1555660106; _gat_gtag_UA_108140256_2=1; Hm_lvt_3c606e4c5bc6e9ff490f59ae4106beb4=1555660106; Hm_lpvt_3c606e4c5bc6e9ff490f59ae4106beb4=1555660106; _pk_id.2.cac6=8ef0dd206a8c2b75.1555660106.1.1555660106.1555660106.; _pk_ses.2.cac6=1; aicoin_session=<KEY>',
#     'Host': 'www.aicoin.net.cn',
#     'If-None-Match': 'W/"9020a-lD5br353ZpSb7dlj59oCbvjQXFE"',
#     'Upgrade-Insecure-Requests': '1',
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36'
}

r = requests.get(url, headers=headers)
print(r.text[0:600])<jupyter_output><!DOCTYPE html><html><head><meta charSet="utf-8" class="next-head"/><meta name="viewport" content="width=device-width, initial-scale=1" class="jsx-1021118270 next-head"/><meta name="description" content="AICoin拥有全面而优质的区块链资源，提供实时行情、专业K线、数据分析、资产管理等专业服务，更高效地提供有价值的信息，提升资产管理的效率，成为行情数据分析的领跑者。" class="jsx-1021118270 next-head"/><link rel="canonical" href="https://www.aicoin.cn?lang=zh" class="jsx-1021118270 next-head"/><title class="jsx-1021118270 next-head">AICoin - 为价值 · 更高效</title><link rel="preload" href="https://assets-www.aicoin.net.cn/beta-v20/_next/static/06lug4_Dcc4j7ieGnWNPB/pages/index.js"
{'fraction_non_alphanumeric': 0.1799902707961732, 'fraction_numerical': 0.14982973893303064, 'mean_word_length': 8.043988269794722, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 46, 'https://': 5, 'lorem ipsum': 0, 'www.': 7, 'xml': 4}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '233019', 'license': 'MIT', 'n_tokens_mistral': 3455, 'n_tokens_neox': 3034, 'n_words': 247, 'path': 'Day018_HW.ipynb', 'repo_name': 'Ruila/PythonCrwalerMarathon_Day18', 'size': '6440'}