In my blogs Webmaster Tools, there is a notification in Crawl Errors section, that is Google couldn't crawl your site because we were unable to access the robots.txt file.My blogs robots.txt file is: User-agent: Mediapartners-GoogleDisallow: User-agent: *Disallow: /searchAllow: /Sitemap: http://example.blogspot.com/feeds/posts/default?orderby=UPDATEDI dont think the above file details are wrong but I dont understand why I received such dangerous notification.How can I fix this issue?
How to fix robots.txt file error in my GWT?
google search console;robots.txt;crawl errors
If you have a message saying that Google couldn't crawl your site because we were unable to access the robots.txt file. Then it is not the contents of the robots.txt file that is in question, it is that Google simply couldn't access the file. And when Google can't access a robots.txt file then it won't crawl the site. Using fetch as Googlebot in Webmaster Tools is a good idea. If your robots.txt file fetches successfully then it could be a past issue. If not, then you obviously need to look further to ensure Googlebot access. 