My colleague claims that he has seen cases where websites were discovered by Google without having a link to them anywhere on the web.For him, this means that the only way to not being indexed (for example when a website being developed is already online, but not ready to be used by customers yet) is to restrict the access to the files on the server with a password.How can Google find a website without following links? If for some reason the site is discovered, isn't robots.txt and a global <meta name=robots content=noindex> (on every page) enough?
Is it possible for Google to discover a website that is not linked to anywhere?
google;indexing;web crawlers
You never know who might be aware of your hidden pages. Anybody could link to these pages from their site without you even knowing about it. Matt Cutts from Google has addressed this in one of his videos on YouTube.It's also worth bearing in mind that any 3rd-party scripts you run on your site could report back the URL where they're running, as well as any internal links within the page. This could include Google Analytics, Affiliate Marketing, random jQuery plugins, or other UI scripts.