I started building a web crawler and read somewhere that it's a very hard problem to find a good seed page for the crawler. Can anyone explain me if there is any pre-defined procedure/ guidlines of finding a good seed page? or how you say that a particular page is a good seed page?
How to find a 'good' seed page for a web crawler?
web;internet;web crawler
A good seed page needs to have As many links as possibleFor as many different topics as possibleThat's all, really. The first things that come to mind would be Wikipedia and the Open Directory Project.