<commit_before>WEBX Webcrawler Application

> Performs depth first search crawling for a specific term from a base link mentioned by the user. Crawler jumps from one page to other navigational links situated on the page. The result also contains various other details of the search term such as the frequency and positions of the word on the webpage.

How to Run:
	> Establish an Internet Connection.
	> Open Webx>src>webx folder.
	> Run start file on IDE that supports Java.
	> Put JSoup jar files into the project external library. 
	> On the User Interface enter no of pages to Crawl.
	> Write a base valid link and enter word to be searched.
	> Crawler displays web graph, green indicates links on which word found & red indicates links where not found.<commit_msg>New Readme file<commit_after>WEBX Webcrawler Application

> Performs depth first search crawling for a specific term from a base link mentioned by the user. Crawler jumps from one page to other navigational links situated on the page. The result also contains various other details of the search term such as the frequency and positions of the word on the webpage.

How to Run:

	> Establish an Internet Connection.
	> Open Webx>src>webx folder.
	> Run start file on IDE that supports Java.
	> Put JSoup jar files into the project external library. 
	> On the User Interface enter no of pages to Crawl.
	> Write a base valid link and enter word to be searched.
	> Crawler displays web graph, green indicates links on which word found & red indicates links where not found.
{'fraction_non_alphanumeric': 0.03374432186891629, 'fraction_numerical': 0.0, 'mean_word_length': 4.105960264900662, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 23, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5117830', 'n_tokens_mistral': 388, 'n_tokens_neox': 374, 'n_words': 243}