<commit_before>---
layout:post
title:
---

###HTTP方法之HEAD
	
* linux curl 模拟head方法

		curl -i -X HEAD "http://www.163.com"
		HTTP/1.1 200 OK
		Expires: Sat, 06 Sep 2014 04:43:50 GMT
		Date: Sat, 06 Sep 2014 04:42:30 GMT
		Server: nginx
		Content-Type: text/html; charset=GBK
		Transfer-Encoding: chunked
		Vary: Accept-Encoding,User-Agent,Accept
		Cache-Control: max-age=80
		Age: 25
		X-Via: 1.1 czdx89:8107 (Cdn Cache Server V2.0), 1.1 jinh61:9 (Cdn Cache Server V2.0)
		Connection: keep-alive

* head方法多用于爬虫机器人判断网页是否有用或是否有效
<commit_msg>change<commit_after>---
layout:post
title:http head
---

###HTTP方法之HEAD
	
* linux curl 模拟head方法

		curl -i -X HEAD "http://www.163.com"
		HTTP/1.1 200 OK
		Expires: Sat, 06 Sep 2014 04:43:50 GMT
		Date: Sat, 06 Sep 2014 04:42:30 GMT
		Server: nginx
		Content-Type: text/html; charset=GBK
		Transfer-Encoding: chunked
		Vary: Accept-Encoding,User-Agent,Accept
		Cache-Control: max-age=80
		Age: 25
		X-Via: 1.1 czdx89:8107 (Cdn Cache Server V2.0), 1.1 jinh61:9 (Cdn Cache Server V2.0)
		Connection: keep-alive

* head方法多用于爬虫机器人判断网页是否有用或是否有效

{'fraction_non_alphanumeric': 0.1261595547309833, 'fraction_numerical': 0.09833024118738404, 'mean_word_length': 3.5720338983050848, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5806168', 'n_tokens_mistral': 589, 'n_tokens_neox': 484, 'n_words': 117}