How to do targeted web crawling?
wehriam: You could try latent semantic analysis of new pages to determine similarity to other documents in your domain.I've been doing a lot of web crawling recently, feel free to get in touch if you'd like to discuss further.