1. A method for automating identification and download of one or more web assets residing in a cloud based infrastructure, the method comprising: seeding, by a web asset management computing device, one or more predetermined URLs to a cyber scraper to download corresponding one or more web pages; validating, by the web asset management computing device, one or more first URLs from the downloaded one or more web pages based at least in part on one or more classification rules, the one or more classification rules satisfying one or more criteria associated with retrieving the one or more web assets; parsing, by the web asset management computing device, each of the validated one or more first URLs into one or more predictor values derived from one or more predictor variables, the one or more predictor values comprise features extracted from the one or more web pages and web links associated with the one or more web pages; generating, by the web asset management computing device, one or more navigation rules by performing logistic regression on the predictor values, wherein the performing logistic regression on the predictor values comprises using a polynomial function comprising of the one or more predictor values and coefficients pertaining to the features extracted from the one or more web pages and web links associated with the one or more web pages; constructing a web asset pipeline comprising one or more second URLs using generated one or more navigation rules; training, by the web asset management computing device, the cyber scraper to identify and download the one or more web assets or an intermediate web artifact page for each of the one or more second URLs; and identifying and downloading, by the web asset management computing device, the one or more web assets or the intermediate web artifact page during run time for each of the one or more second URLs generated using the one more navigation rules.