I have a proxy at http://rahul2001.com/proxy/I site can be 'proxied' by like this:http://rahul2001.com/proxy/proxy.php/http://site-to-be-proxied.example.com/The problem is that Google seems to be crawling pages through my proxy :/ I do NOT link a proxied version of Yahoo food or Google Careers or Google Search Hindi Help, but these still turn up in the search results...PROBLEMS:-I DO NOT want to block my website/proxy from search engines entirely, I still want the proxy service itself to be able to be located in search.-I DO NOT want Google to use up my bandwidth by crawling useless sites.-I DO NOT want to use captcha since a few of my apps use this proxy.-I DO NOT want Google to spoil the search results of my website in this manner.What do I do??ALSO, why is Google entering random URLs into the form??EDITAfter adding the meta tag, I get an error :(proxy.php (first few lines):<head><meta name=robots content=noindex, nofollow /></head><?php/*miniProxy - A simple PHP web proxy. <https://github.com/joshdick/miniProxy>Written and maintained by Joshua Dick <http://joshdick.net>.miniProxy is licensed under the GNU GPL v3 <http://www.gnu.org/licenses/gpl.html>.*/Error:Warning: Cannot modify header information - headers already sent by (output started at /home/rahulcom/public_html/proxy/proxy.php:3) in /home/rahulcom/public_html/proxy/proxy.phpHELL BREAKS LOOSE - GOOGLE IS CRAWLING THE INTERNET USING MY PROXY!!https://www.google.co.in/search?q=site:rahul2001.com+proxy
Stop Google from crawling proxied pages but still allow the proxy itself to be found via search engines
seo;google;google search;web crawlers;proxy