<commit_before>Scholar crawler
===============

Python scripts that crawl [Google Scholar](https://scholar.google.com/)
to harvest citation information.<commit_msg>Included usage and dependencies<commit_after>Scholar crawler
===============

Python scripts that crawl [Google Scholar](https://scholar.google.com/)
to harvest citation information.

1. PYTHON DEPENDENCIES
	- requests 2.6.0
	- selenium 2.45.0
	- lxml 3.4.2
	- pandas 0.16.0 (for future use)



2. CALL FROM COMMAND LINE
   python authorCrawler <enter authorID here>

   For example, to check <NAME>' profile on Google
Scholar, with url:
   https://scholar.google.com/citations?user=PA9La6oAAAAJ&hl=en&oi=ao

call:
   python authorCrawler PA9La6oAAAAJ



3. FUNCTIONALITY
   to be included...
{'fraction_non_alphanumeric': 0.13095238095238096, 'fraction_numerical': 0.027777777777777776, 'mean_word_length': 4.330985915492958, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 1}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3901089', 'n_tokens_mistral': 269, 'n_tokens_neox': 240, 'n_words': 71}