is that there are just so many websites out there already.


English Google SEO office-hours from January 22, 2021

So really finding that niche where


you can provide something unique and valuable


can sometimes be a bit challenging.


And also for the more commercial topics,


there are a lot of very well-established, well-monetized


websites out there already where it will be hard to get in.


So if you want to put together a bookstore


and you say, oh, I will make it online because, I don't know,


I have the best science fiction book collection of anyone else,


then opening a bookstore online, that's


going to be really challenging because there


are some really strong competitors out there already.


Let's see.


I want to start a new website.


Oh, I think it's the same person.


And I want to move some of the best content


from my previous site that was hit


by a core update in the past.


How do I do that in a safe way?


I don't want to transfer the bad website historic signals


to the content from the old website.


So I think there are generally two approaches here


that you can take.


For the most part, if the content is good content,


then I would just take that content and move it


to a new website if you think that that's really


what you want to do.


And moving it to a new website would


be, real one, redirecting from the old URLs to the new ones


so that we can pass some of those signals over.


Whereas if you don't want to be associated


with the old website, then you essentially


need to create new content.


So if you just copy and paste the whole content


and put it on a new website, then we


will see that and recognize, oh, this


is the same content as the other website here has.


So we will try to pick a canonical between these two,


or we'll say these pages are kind of equivalent


and pass those signals anyway.


So if you want to be associated with the old website,


you might as well just redirect.


If you don't want to be associated


with the old website, then you really


need to create something new.


I was wondering if Google is going to penalize us


for using a pop-up for a limited time, say for a week or so.


The brand team wants to run a survey on site.


I don't think we would penalize a website for anything


like this.


So in particular, the web spam team


has other things to do than to penalize


a website for having a pop-up.


There are two aspects that could come into play.


On the one hand, we have the on mobile,


the policy of the intrusive interstitials.


So that might be something to watch out


for that you don't keep it too long


or show it to everyone all the time.


And with that policy, it's more of a subtle ranking factor


that we can use that to adjust the ranking slightly


if we see that there is actually no useful content on this page


when we load it.


So that's something that could come into play,


but that's more something which would be a temporary thing.


If you have this survey on your site for a week or so,


then during that time, we might pick up on that signal.


We might respond to that signal.


And then if you have removed it, we


can essentially move on as well.


So it's not that there's going to be a lasting effect there.


The other thing I think, I don't know, I forgot.


I think another aspect that you probably want to watch out for


is if you're showing the pop up instead of your normal content,


then we will index the contents of the pop up.


If you're showing the pop up in addition


to the existing content, which sounds like the case,


if it's really a pop up, then we would still


have the existing content to index,


and that would kind of be OK.


So yeah.


Can I ask a follow up on that?


Because there is actually one American paper that,


literally every day, a pop up comes


to sign on to its newsletter.


You can see a little of the text below and under,


but you have to click it every time.


And I've also actually wondered about, first of all,


I wonder, are they really getting that many people


on a daily basis to sign up to this newsletter


compared to what this might be doing negatively to them?


Is that something that would be a negative?


Because it is actually extremely annoying.


Yeah, yeah.


That sounds like something that we would pick up on.


One of the things there is that we


focus on the mobile version of the site,


and we only use that in the mobile search results.


So that's something where, if they're not


showing it on mobile, then maybe we


wouldn't be picking up on that.


But if it's on mobile as well, then that


would definitely fall into the category


of intrusive interstitials.


And we'd say this is kind of like something


where we would slightly demote the website in search.


Sometimes the tricky part there is,


with the slight demotions, it's not the case


that we'll remove the site from search,


or we'll kind of move it to page 100 or something like that.


But if it's really relevant content,


then maybe we'll still show it on the first page of the search


results, just not as highly as it could be.


Yeah, Alberto?


AUDIENCE 1 OK, can you hear me?


Yes.


Excuse me for my English, always, because I have a question.


With the website, my website is first in the Google search,


OK, in the first page.


But I discovered that it's made in WordPress,


that if I erase the final S of a HTTP,


I can reach two versions of the same page,


with SS secure and without.


I found that there is a solution changing the HTTP access


with a permanent 301 redirection, OK?


But I am a little afraid to lose ranking


with a permanent redirection.


So it's better let them think like it is with the two


versions of URL that Google decides,


which is the canonical and which is not, OK,


without making any redirection, OK?


Or for better authority of canonical URL,


it's better to make the permanent redirection of 301


to ack the THS, OK?


I know if my, excuse me, my English, but the code is OK.


Thank you.


Yeah, yeah, it's clear.


I would definitely do the redirect.


So you will not lose any value if you do the redirect there,


because it's within the same website.


So I would do the redirect, because it's


the clean approach to setting up the HTTPS version of a website.


So I would just set that up.


I think with a redirect, the one place where I would watch out


for is if you're doing a site move, moving from one domain


to another domain, then that's something


where we have to process that, and it can get complicated,


and things like that.


But with HTTP and HTTPS, it's perfectly fine.


There's nothing fancy or tricky for us about that.


So I would just do the redirect.


OK, thank you.


Thank you, John.


John, may I add something?


Sure.


Alberto, also, keep in mind and be careful not


to generate any redirect chains.


Choose a specific version with or without www,


so you can use this version as an official URL of your website.


And make sure that all of the other versions with www,


without www, or with www, or with HTTP, the HTTP version,


including the HTTPS, with or without,


depends on what you decide to go with.


Are pointing to the final URL you have decided to go with.


Sorry for this a little bit difficult explanation,


but it's usually hard to explain this without any examples.


I will provide you some examples, actually,


over the chat.


Sorry about this, but Alberto, I find


to another information from the official page of the plugin,


and they give you a recommendation


to change this option of the plugin, to stop it


and change with another one.


This is from the official page of the plugin,


so you can follow my last comment.


Oh, Renato, was there some bugs with this plugin?


Yeah.


Because of the sitemap, you're talking about sitemap, right?


Yeah, OK.


Sorry about that, John.


Yeah, sorry for that, John.


So it's already parallel discussion.


OK, cool.


Fantastic.


OK, let me jump to the next question.


I have a YouTube channel with 9,000 subscribers,


and I also have a blog.


Sometimes I write a blog post and use the same text


I create YouTube videos for.


Is this content duplication because Google Search


can understand videos?


I'm saying that because my two blog posts are crawled,


but they're not ranking on Google.


Even when you put that link of the blog post in Google Search,


it's not there.


It's crawled two months ago.


My other blog post, where I didn't create a video,


they're ranking without a problem.


Could you recommend what to do in this case?


Should I use a canonical tag or delete


the blog post or the video?


So first of all, we don't do kind


of text analysis of the videos and then map them to web pages.


So if your video has the same content as your blog post,


it's still something different.


People sometimes go to Google with the intent


to read something, and sometimes they


go to Google with the intent to watch something


or to listen to something.


And those are very different things.


We would not say the text in this video


is exactly the same as a blog post.


Therefore, we don't show either of them,


or we only show one of them.


So if you have a video that matches your blog post,


I think that's perfectly fine.


That's a great way, I think, to kind of spread


your information in different channels.


So I would definitely not stop doing that.


I would not take the video down or take the blog post down.


If the blog post is not ranking in Google,


then that would be very specific to the blog post


and not specific to the video-blog post combination.


Also, with regards to duplicate content,


if you had the same content in textual form on your website


where it's clearly duplicate content,


then what would happen there is we


would just pick one of those versions


to show in Google Search.


It's not the case that we would say, oh, this website


has some duplicate content.


We will not show it at all in Google.


But rather, we'll say, there are two versions here.


We will pick one of these to show,


and we will just not show the other one.


So that's something where, even when we do recognize duplicate


content, it's not the end of the world.


It's really just a matter of us saying,


we don't want to show the same thing to users multiple times


in the search results.


So we will pick one, and we will show that one.


I found these lines in a Google Webmaster Central blog related


to AMP.


AMP will no longer be necessary for stories


to be featured in the top stories on mobile.


It will be open to any page.


Is this already live in the Google Search results


for the news section, or when will it


be implemented from Google?


I am pretty sure, at the moment, this is not live.


As far as I understand, this is a change


that is going to happen when we launch the page experience


as a ranking factor, which is planned, I believe, or made.


So that's something where it's not live at the moment.


It's still coming.


Back here.


Let me just mute you for a moment.


I'm a newbie.


I don't have any idea how to index my website in Google,


how to rank my site, what is a sitemap,


how do I submit a sitemap.


Yeah, lots of questions here.


My recommendation here would be to go off and read


a lot of information first.


So especially if you're new to the web


and you have different ideas that you want to publish


and you don't really know about the technical details,


I would recommend going through some of the SEO starter guides.


So we have an SEO starter guide that


is fairly long and comprehensive,


lots of information in there.


It can take a while to go through.


It also covers sitemap files, how to get indexed,


how ranking works a little bit.


There are also other SEO beginner guides


out there which have a lot of really useful information,


which give sometimes a little bit of a less Google-focused


view, which I think is also great to have


in the beginning.


And I would go through these different guides


to try to figure out what is all involved with going


online and being available in search.


For the most part, I think the nice aspect is,


for the most part, you can use any system that


is out there that makes websites,


and it will just work well in search.


So all of the platforms that are commonly used,


be it something like Blogger from Google


or if you're using Wix or Squarespace or whatever you


have, then essentially those are fairly well tuned already


for Google so that you don't have to do anything


special to make them work.


There are lots of things you can do additionally


if you want to focus a little bit more on search.


But for the most part, all of these platforms


will just work for you.


So that's kind of my approach there is, on the one hand,


dive in and try things out.


On the other hand, there are lots of these starter guides


that help to give you the basics and some of the starting


information there.


And just as a word of warning, there


is an endless amount of information


that you can go through afterwards.


So even if you go through the starter guides,


it's not the case that you will have seen everything.


There are just so many details that you


can focus on if you want to.


My mdot website shows primary caller is desktop,


but I want to switch to smartphone.


What should I do in this case?


It was smartphone before, but it shows desktop recently.


Why did it happen?


I don't know.


So I mean, why it happened.


I don't know why you would see that.


So in general, our systems automatically


switch to mobile first indexing when


we find that a website is ready for that.


And if a website currently is still with the desktop crawler,


then there would be two reasons for that.


On the one hand, it could be a part


of an experimental control group, which we always


keep for these bigger changes.


Usually, that's a fairly small route, though.


And the other part could be that our systems still


think that something on your website


is not quite ready for mobile first indexing.


And that could be something like you


have more images on your desktop version


than you have in your mobile version, something


along those lines or structured data.


We have documented various of these things already.


And that's something where kind of towards the date


that we have specified for when we would switch everything


over, which I think is March or April.


I forgot.


Then towards that date, we will start probably


shifting websites over, even if there are still issues.


So my recommendation there would still


be to kind of like double check to make sure that you


have everything covered.


And if you're sure that everything is covered,


then just let it be, let it shift over on its own.


Sometimes our systems are confused by things


that you might not care about.


So that's an aspect that sometimes comes into play when


I look at some of these websites.


For example, if you go to a desktop page,


you will often have related articles on the bottom.


And sometimes these related articles


have a thumbnail image associated with them.


And on a desktop, you might have, I don't know,


maybe five or six of these related articles on the bottom.


Whereas on mobile, if you go to the mobile version of the page,


you might just have one related article on the bottom


with one thumbnail image.


And then our systems would look at that and say, well,


on desktop, you have six of these images.


On mobile, you just have one.


Clearly, you're not providing the full content on mobile.


And as a site owner, it's like, technically, that's correct.


But you probably don't really care about that.


Maybe those thumbnail images are not


the critical part of your website.


So it's not something where you'd say,


this is really would be a problem for me.


And a lot of the cases I looked at kind of fall


into that category where people make the website work well


on mobile, but it just doesn't have exactly the same setup


as on desktop.


And then our systems say, oh, it's not ready.


And actually, you as a site owner


might say, well, it's good enough.


It's what I want.


And as we move towards the deadline there,


then it will shift over anyway.


And also, there is no ranking effect


associated with the mobile-first indexing change.


So it's not that you will rank better or higher


if a website is in mobile-first indexing.


But will it be the case if it's the opposite,


that if it's not mobile index ready, is that a negative?


So we would not see that as a ranking effect.


But we would perhaps just not have all of the content.


So if the primary images are actually missing on mobile


and those are images that you used


to get a lot of traffic for from image search,


then suddenly we would not have those images anymore


and we wouldn't be able to show the website.


So it's not a change in ranking.


It's more like we don't have the full content,


so we can't index it or rank it for that content.


How would it be when the website is only partly on the mdot site


and on the www?


There's a part is responsive, a part is not mobile friendly,


but it's not covered by the mdot.


And the other part is not mobile friendly on the www,


but has mdot coverage.


So the system that we have to switch between or to switch


a website to mobile first indexing


would look at the domain overall.


And we would try to aggregate across the domain


to figure is it ready or not.


And we would switch it over based on that.


But for indexing, we do that on a per page basis.


So if some pages are on the mdot and they have the full content


there, we will pick that up.


If some are responsive on the www version,


then that's the mobile version that we


would use for indexing there.


So for the switch over, we might be kind of confused.


If some things are ready and some things don't look ready,


but for the actual indexing afterwards,


probably that would be OK.


And how do you know or how would we


know if we are in this experimental control group?


I don't think you would know.


OK.


Yeah.


I mean, we don't specify that externally.


And it's not something that is really large.


So usually it's something like 1 10th of a percent


or something like that where we just hold things back


a little bit so that we can make sure that when we compare,


it looks correct.


And how would the effect be on parts


from Google like Discover or Google News


in such a mix-up constellation?


I don't think there would be any difference there.


So we would also do this on a per page basis.


And as far as I know, Google News and Discover,


they both are based on the same indexing


that we use for web search.


So that should just be fine.


The one aspect I don't know about Google News


is how when you're within the Google News corpus,


I think you have to specify your website or something like that


to get registered for Google News.


I don't know how they would handle a mix of www and m dot.


My guess is they can deal with that properly


because it's been like that for a long time now.


But that might be something to check with the Google News


folks.


Can we some parts check how the mobile first index


is dealing with the page when we are not ready yet?


How do you mean?


Well, if I'm checking several tools


like the live URL testing or also the Core Web Vitals


or mobile testing of Google, they always


have a reference like the alternate tech for the m dot


sites.


And it's a bit confusing if they are now


dealing with the m dot sites for the check


or with the www as the desktop version.


Yeah, I don't know.


I think in a case like that, I would still check both versions


if you can check that manually.


But make sure that when you're testing things for indexing


to focus on the mobile version.


So I think it's always a bit tricky with www and m dot


versions because you do have two separate templates that you


need to check.


But for indexing, we will focus on the m dot version.


For users, you still have to focus on both of those versions.


So especially things like speed, that's


something where you would test the desktop version


and the mobile version separately.


I think in Search Console, we also


show that information separately with regards to speed.


So that's something where doing both is a good idea.


But for indexing, I would really focus on the mobile version.


OK, thank you.


Sure.


Hi, Alberto.


Hello, I have a question that is very nice for my podium.


I disabled, disabled, on my website,


analytics, Google Analytics.


Why?


And I don't know if it's correct or not, OK?


Because I am a fanatic of speed website, OK?


I love the speed website without making only an active ML


website with WordPress, OK?


Speed with WordPress.


I found that with analytics, ranking


is not better than without because there


is a time to transmit data between the Google and website.


This with the Google Tag Manager, too.


It is with the Facebook Pixel Manager, too.


OK, so I prefer to don't catch data with analytics,


but has a more speed website because Google in his metrics


doesn't recognize that we have analytics, OK?


And so your website is one second and a half lowest


because there is analytics.


So you are worse than another site that is more speed,


do you know?


This is the question, OK?


Thank you.


Yeah.


So I think, first of all, we don't do anything special


for analytics.


It's not the case that we would say, oh, you


use Google Analytics.


That's fine.


We don't care if a website is slow because Google Analytics.


If it's slow, it's slow.


And that's something where sometimes it


makes sense to look at the different embedded things


that you have, like analytics, like ads, like Tag Manager,


or whatever, and to figure out which parts you really


need for your websites and which parts you can say,


well, I don't need them.


I prefer to make my website a little bit faster


and to say, well, maybe I don't have this data.


So that's something which you have to decide.


But sometimes there are also ways


that you can implement analytics in a way that is very fast.


So that might also be worthwhile to check out maybe


in the Analytics Help Forum to see what is the fastest


way to implement analytics.


Another approach that I've seen is


that people take a sample of the traffic


and they just measure that, where they can say,


I will measure every 100th visitor with analytics,


and everyone else, I will not measure like that,


which means that on average, you still


have a little bit of data with regards


to how people go through your website, which


parts of your website are useful.


But for the most part, your users


are able to get through very quickly.


So those might be some approaches there.


I think there is a lot of value in analytics


and in understanding how users interact with your website.


So I would try to find a way to keep this data


and to be able to use that data to help your website get


better, rather than to just say, oh, I will not track anything


so that I can be one millisecond faster than everyone else.


Because in the end, you need to find things


that you can improve on your website.


And you need to constantly be improving your website.


And you might be a little bit faster today.


But if you keep your website the way


that it is because you don't have data on which parts people


find useful, then other people will


make websites that maybe are a little bit slower


but are a lot more useful.


And then maybe we will show those websites.


So that's a balance you have to try to find.


AUDIENCE 1 Thank you.


Thank you.


All right.


Let me just maybe grab one or two more questions


from the list.


And then we can do some more live as well.


I'm having a hard time with Google marking


my website URL as noindex.


It takes forever to Google to recheck my website.


And most of the time, I'll get back the same report.


How can I change the URL mark noindex so that I can finally


start selling stuff?


Google is making it really hard for my website to be recognized.


So I'm not 100% sure how you mean this with regards


to Google marking my website noindex.


So usually, noindex is something that you


have within your website, maybe a setting in the website system


that you have where you're actively telling us


not to index specific pages.


And removing that is something that you


have to remove on your website.


And when we recognize that it's gone for individual pages,


we will start showing those pages in search.


So that's something where you, I suspect,


you still need to find that setting in your content


management system to change that out.


I think you're here in the Hangout as well, right?


Yes, yes, I am right here.


Can you give me a little bit more detail


on what you're seeing?


Well, I just got started.


I'm from Switzerland as well.


And I just created a second website.


I had a lot of problems with the duplicates


as well because I needed parts of my first website


to be on the second one as well.


I was able to change those all up.


But at the moment, I'm having a hard time with the no index.


I'm crawling the websites.


And I'm still waiting for it.


My boss said it usually takes a while.


I'm just very impatient.


And I really want to sell a lot of stuff on my website.


But I can't really get up there in the ratings at the moment.


OK.


I would suggest maybe going to the Search Central Help Forum


and specifying your site there and explaining


a little bit what issues you're seeing there.


Because my suspicion is that there is just something small


that you need to change in your setup.


And then it will work.


With regards to suddenly being indexed and showing up


in rankings, that does take a bit of time.


And especially with regards to rankings,


if it's a competitive area where other people are selling


similar things, then it takes time.


It takes a lot of work.


You have to find different ways to get the word out as well.


That could be maybe doing some advertising in the beginning


to drive that initial amount of traffic there


and to find maybe there are other creative ways


that you can get people to find your website


and discover the cool things that you have there.


But the folks in the Help Forum are


very good at spotting a lot of the things that


hold the website back.


And that's probably a good way to start.


I think Dito and Nikolai are also active in the Help Forum.


So I'm sure if you post a link here in the chat,


then they'll also be jumping on it as well.


But I think, especially in the beginning,


it's useful to get feedback from a lot of different people.


And the Help Forums are a great place for that.


It's not the case that everyone will have perfect feedback,


and you will just rank number one


if you follow all of that advice.


But it gives you some things to think about.


SPEAKER 2 OK, perfect.


Thank you very much.


JOHN MUELLER Sure.


All right.


I think we're kind of running low on time.


So maybe I'll just switch to more questions from you all.


And I'll also stick around a little bit longer


after the recording if any of you


want to hang around a bit more.


SPEAKER 3 I have one question of 2dm.sites


and the mobile first crawling.


How important is it for the mobile first crawler


that all sites are mobile friendly?


Is it a problem still having problems there or issues there?


JOHN MUELLER For mobile first indexing,


we don't care if it's mobile friendly or not.


So that's something if the text is on the page,


if you have to zoom in and kind of find it, it's on the page.


It's for indexing, that's perfectly fine.


It's only for ranking afterwards that we


would use the mobile friendliness as a ranking


factor.


But for indexing, for mobile first indexing,


it doesn't matter at all.


So in particular, those very old websites


that are built with tables, you can look at them on mobile.


You have to zoom in, all of that.


But they work perfectly fine on mobile first indexing


because they have exactly the same content on mobile


as they have on desktop.


You have to zoom in.


It's hard to find.


But it's all there.


Great, thank you.


Can I add one thing here?


Sure.


So mobile first indexing is technical thing


and mobile friendliness is user behavior thing.


This is why you are saying it will not


have impact of whether it is zooming out or scrolling down


kind of thing.


Pretty much, yeah.


No, no.


It's not a good idea.


It's scrolling down kind of thing, pretty much, yeah.


Mobile first indexing is the HTML page, the text on the page,


and mobile friendliness is more about usability.


Thanks.


Hi, John.


Hi.


We have a website.


And on our website, there is a particular section


that we would like to use, the news section


related to our industry.


So I just wanted to know that apart


from the normal activities that we do, like putting content,


do we need to do something extra so that Google


show these news items in Google News


also apart from normal web results?


Like, we need to register our website somewhere


for Google News.


My understanding is you still need


to set up the website in the News Publisher Center.


But I don't have any details on the Google News side.


So that's something where for normal web search,


for things also like the top stories, those kind of things,


we don't need anything special.


But particularly for Google News,


they do need a little bit more information.


And I believe that's all in the Google News Publisher Center.


So you need to check that out.


Sure.


No problem.


Thank you.


Hi, John.


Can I come in?


OK.


So I have two questions related to news publishers.


So basically, number one is the best practices


to syndicate content for a news publisher,


alerting Google, which one is original content.


Does actually means there are some tagging rules there,


right, with the health support and everywhere, right?


So does actually this means whenever actual tagging


will help original content to rank over the syndicated


content, that actually works?


Or does it have a different setup or rule?


I mean, for syndicated content for news publisher.


That's one question.


And other part, Google discovered visibility


for the news publishers, right, against the Google Broadcore


series of updates, right?


That's two questions.


OK.


So for syndicated content, we do have the rel canonical


that we recommend, where if you are syndicating content


from another website, we recommend using the rel canonical


to tell us what the original source is.


So that helps us to understand the connection


between these two.


However, it's not always the case


that we will only show the original source.


A lot of times, it makes sense to show different versions


of the same content.


So in particular, this is something


that you might care about if you have the original source


and you syndicate your content out to other websites.


Then you have to assume that we will index


all of those other versions as well.


And it's possible that we will show one of these other versions


above your website.


I don't know how that is handled in Google News,


but at least in web search, it's certainly possible


that if you syndicate the content out,


then those syndicated content copies


could show in the search results.


And in some cases, they may show above your content.


So that's something where if you're saying


this is a critical problem, it should never


happen for our website, then you might


need to reconsider how you syndicate your content.


In many cases, that's less of an issue.


And you would say, well, the important part


is that the information gets out,


that maybe we get mentioned as a source,


that that's kind of included there as well.


And then the syndicated content might be visible in search,


maybe even above your results, but it still


attracts attention to your website


because you're kind of the primary source


of that information.


So balancing those two sides is something


that is almost on your side.


You need to make that choice.


Do I want to syndicate or do I not want to syndicate?


All right, fair enough.


And the second part was about the discover thing, right?


The news publisher and the broadcast series update.


Is there any relation?


Because we are seeing a lot of fluctuations


in discover traffic, right?


Whenever there's a bid or fluctuation or maybe


some vulnerabilities around the rankings or others.


So is there any relation of it?


Yeah, so we do use a number of the same quality algorithms


in discover as we use in web search.


So when a broad core update happens in web search,


it's very common that you would also see changes in discover


as well.


So that's certainly not totally unrelated.


I mean, we see discover as almost a part of search.


So it's not something that we would


say is completely independent and uses completely separate


algorithms and everything.


It does include a lot of similar things.


Also, the content, of course, is based


on what we call an index for web search.


It's not a separate index or anything like that.


All right, fair enough.


Thank you.


OK, let me pause here.


I'll pause the recording.


And if any of you want to hang around a little bit longer,


you're welcome to do so.


Thank you all for joining.


Thanks for submitting so many questions


and asking so many questions along the way.


I hope you all found this useful.


It's always one of the highlights of the weeks


to kind of get some direct feedback from people


and, I don't know, almost see people in person.


It's like such a weird thing.


So thanks for jumping in, and hope


to see you all in one of the future Hangouts again.


Thanks a lot.


Bye.


How long to keep 301 redirects? #AskGooglebot

One of the questions we get often is how long a site should keep a redirect after a site move.


Taking a step back, our recommendation for site moves is to use 301 permanent redirects.


It seems that nothing's permanent in this world, so how long is permanent when it comes to redirects?


At Google, we try to reprocess all pages at least every few months.


Most pages are checked much more often.


However, the amount of crawling is limited, and there are many, many pages that we'd like to crawl, so we have to prioritize.


When a URL changes, our systems need to see the change in the form of a redirect for at least a few times in order to record that change.


To be certain that a redirect has been seen a few times, we recommend keeping the redirect in place for at least one year.


It's also important to update all references to the old URL.


That includes links the page may have within your website and critical links from outside of the website.


This is all somewhat simplified, and to be safe, it's always fine to keep a redirect longer, but you should aim for at least one year.


I hope you found this short answer useful.


If there are other questions on your mind that we can answer like this, then drop us a note with the hashtag AskGooglebot.


See you next time.


English Google SEO office-hours from July 1, 2022

Oh, hi, and welcome to today's Google Search Central SEO


Office Hours session.


It's nice that you could join us here today,


at least virtually on YouTube.


My name is John Mueller.


I'm a search advocate on the Search Relations team


here at Google in Switzerland.


And part of what we do is answer questions


around web search and websites, maybe questions


that you submitted as well.


Maybe starting off, I have one topic that has come up


repeatedly recently.


And I thought I would just try to answer it


in the form of a question why we're at it here.


So first of all, when I check my PageSpeed Insight


score on my website, I see a simple number.


Why doesn't this match what I see in Search Console


and the Core Web Vitals report?


Which one of these numbers is correct?


Whew, yeah.


I think maybe first of all, to get the obvious answer


out the door, there is no correct number when


it comes to speed, when it comes to understanding


how your website is performing for your users.


In PageSpeed Insights, by default,


I believe we show a single number that


kind of is a score from 0 to 100, something like that,


which is based on a number of assumptions, where we assume


that different things are a little bit faster or slower


for users.


And based on that, we calculate a score.


In Search Console, we have the Core Web Vitals information,


which is based on three numbers for speed, responsiveness,


and interactivity.


And these numbers are slightly different, of course,


because it's three numbers, not just one number.


But also, there's a big difference in the way


that these numbers are determined.


Namely, there's a difference between so-called field data


and the lab data.


Field data is what users have actually


seen when they go to your website.


And this is what we use in Search Console.


That's what we use for search as well,


whereas lab data is kind of a theoretical view


of your website, where our systems have


certain assumptions where they think, well,


the average user is probably like this using this kind


of a device and with this kind of a connection, perhaps.


And based on those assumptions, we


will estimate what those numbers might be for an average user.


And obviously, you can imagine those estimations will never


be 100% correct.


And similarly, the data that users have seen,


that will change over time as well,


where some users might have a really fast connection


or a fast device.


And everything goes really fast on their website


or when they visit your website.


And others might not have that.


And because of that, this variation


can always result in different numbers.


Our recommendation is generally to use the field data,


the data you would see in Search Console,


as a way of understanding what is kind of the current situation


for our website, and then to use the lab data,


namely the individual tests that you can run directly yourself,


to optimize your website and try to improve things.


And when you are pretty happy with the lab data


that you're getting with your new version of your website,


then over time, you can collect the field data,


which happens automatically, and double-check


that users actually see it as being faster or more responsive


as well.


So in short, again, there is no absolutely correct number


when it comes to any of these metrics.


There is no kind of like absolutely correct answer


where you'd say, this is what it should actually be.


But rather, there are different assumptions and different ways


of collecting data.


And each of these are subtly different.


All right.


So let's see.


First question otherwise that I have on my list here,


which all of these questions were submitted on our YouTube


community page.


And we'll try to announce the next ones there as well.


So you can add more questions there.


You can also, of course, post in our community help forum,


where other experts are able to jump in and give you


some advice as well.


So first up, we have a few customer pages


using Next.js without a robots.txt or a sitemap file.


Simplified, theoretically, Googlebot


can reach all of these pages.


But why is only the home page getting indexed?


There are no errors or warnings in Search Console.


Why doesn't Googlebot find the other pages?


So maybe taking a step back, Next.js


is a JavaScript framework, which means that the whole page is


generated with JavaScript.


But as in a general answer as well


for all of these kinds of questions,


like why is Google not indexing everything,


it's important to first say that Googlebot will never index


everything across a website.


I don't think it happens to any non-trivial sized website


that Google would go off and index completely everything.


It's just from a practical point of view,


it's not possible to index everything across the whole web.


So that kind of assumption that the ideal situation


is everything is indexed, I would leave that aside


and say, you really want Googlebot to focus


on the important pages.


The other thing, though, which became a little bit clear


when I think the person contacted me on Twitter


and gave me a little bit more information about their website


was that the way that the website was generating links


to the other pages was in a way that Google was not


able to pick up.


So in particular, with JavaScript,


you can take any element on an HTML page


and say, if someone clicks on this,


then execute this piece of JavaScript.


And that piece of JavaScript can be


to navigate to a different page, for example.


And Googlebot does not click on all elements


to see what happens.


But rather, we go off and look for normal HTML links, which


is the kind of traditional normal way


that you would link to individual pages on a website.


And with this framework, it didn't generate


these normal HTML links.


So we could not recognize that there's actually more to crawl,


more pages to actually look at.


And this is something that you can


fix in the way that you implement your JavaScript site.


We have a ton of information on the Search Developer


documentation site around JavaScript and SEO,


in particular on the topic of links,


because that comes up every now and then.


There are lots of creative ways to create links.


And Googlebot really needs to find those HTML links


to make it work.


Additionally, we have a bunch of videos on our YouTube channel.


And if you're watching this, since nobody's here,


you must be on the YouTube channel.


If you're watching us on the YouTube channel, go out


and check out those JavaScript SEO videos on our channel


to kind of get a bit of a sense of what else


you could watch out for when it comes


to JavaScript-based websites.


We are able to process most kinds


of JavaScript-based websites normally,


but some things you still have to watch out


for, like these links.


OK.


Let's see.


Next up, does it affect my SEO score negatively


if my page is linking to an external, insecure website?


So on HTTP, not HTTPS.


So first off, we don't have a notion of an SEO score.


So you don't really have to worry about SEO score.


But regardless, I kind of understand the question is,


is it bad if I link to an HTTP page instead of an HTTPS page?


And from our point of view, it's perfectly fine.


If these pages are on HTTP, then that's what you would link to.


That's kind of like what users would expect to find there.


There's nothing against linking to sites like that.


There's no kind of downside for your website


to kind of avoid linking to HTTP pages


because they're kind of old or crusty and not as cool


as on HTTPS.


I would not worry about that.


OK.


Next up, with semantic and voice search,


is it better to use proper grammar


or write how people actually speak?


For example, it's grammatically correct to write more than x


years, but people actually say over x years.


Or write a list beginning with such as x, y, and z,


but people actually say like x, y, and z.


Good question.


So the simple answer is you can write however you want.


There's nothing holding you back from just writing naturally.


And essentially, our systems try to work


with the natural content that we have found on your pages.


So if we can crawl and index those pages with your content,


we'll try to work with that.


And there's nothing special that you really need to do there.


The one thing I would watch out for with regards


to how you write your content is just


to make sure that you're writing for your audience.


So for example, if you have some very technical content,


but you want to reach people who are like on technical,


then write in the non-technical language


and not in a way that only is understandable to people


who are really deep into that kind of technical information.


So kind of like the, I would guess,


the traditional marketing approach of write


for your audience.


So our systems usually are able to deal


with that perfectly fine.


OK.


Next up, a question about links and disavows.


Over the last 15 years, I've disavowed over 11,000 links


in total.


I never bought a link or did anything


unallowed like sharing.


The links that I disavowed may have been from hacked sites


or from nonsense auto-generated content.


Since Google now claims that they have better tools


to not factor these types of hacked or spammy links


into their algorithms, should I just delete my disavow file?


Is there any risk or upside or downside to just deleting it?


So this is a good question.


It comes up every now and then.


And disavowing links is always kind


of one of those tricky topics because it


feels like Google is probably not telling you


the full information.


But from our point of view, it's actually


like we do work really hard to avoid taking these kind


of links into account.


And we do that because we know that the disavow links tool is


somewhat a niche tool.


And SEOs know about it.


But the average person who runs a website has no idea about it.


And all of those links that you mentioned


there are the kind of links that any website gets over the years.


And our systems understand that these are not


things that you're trying to do to kind of a game or algorithm.


So from that point of view, if you're really


sure that there's nothing around like a manual action


that you had to resolve with regards to these links,


I would just delete the disavow file and move on with life


and kind of leave all of that aside.


One thing I would personally do is just


download it and make a copy so that you have kind


of a record of what you deleted.


But otherwise, if you're sure these are just


kind of the normal crufty things from the internet,


I would just delete it and move on.


There's much more to spend your time on when it comes


to websites than just disavowing these random things that


happen to any website on the web.


OK, so let's see.


Adding a schema markup with Google Tag Manager


is good or bad for SEO?


Does it affect ranking?


So first of all, you can add structured data


with Google Tag Manager.


That's definitely an option.


Google Tag Manager is a simple piece of JavaScript


you add to your pages.


And then it does something on the server side,


and it can modify your pages slightly using JavaScript.


For the most part, we're able to process this normally.


And the structured data that you generate


like this, that can be counted just like any other kind


of structured data on your web pages.


And from our point of view, structured data,


at least the types that we have documented,


is primarily used to help generate rich results,


we call them, which are kind of like these fancy search


results with a little bit more information, a little bit more


color or detail around your pages.


And if you add your structured data with the Tag Manager,


that's perfectly fine.


From a practical point of view, I


prefer to have the structured data directly on the page


or directly on your server so that you


know exactly what is happening.


It makes it a little bit easier to debug things.


It makes it easier to test things.


So trying it out with Tag Manager from my point of view,


I think, is perfectly legitimate.


It's an easy way to try things out.


But for the long run, I would try


to make sure that your structured data is on your site


directly just to make sure that it's easier


to process for anyone who comes by to process your structured


data.


And it's easier for you to track and debug and maintain over


time as well so that you don't have


to check all of these different separate sources.


OK, let's see.


Simplifying a question a little bit, which is better,


blocking with robots.txt or using the robots.meta tag


on a page?


How do we best prevent crawling?


So this also comes up from time to time.


We actually did a podcast episode recently


about this as well.


So I will check that out.


The podcasts are also on the YouTube channel.


So you can click around a little bit.


You'll probably find that quickly.


In practice, there is a subtle difference here


where if you're an SEO and you've


worked with search engines, then probably you


understand that already.


But for people who are kind of new to the area,


it's sometimes unclear exactly where all of these lines are.


And with robots.txt, which is the first one that you


mentioned in the question, essentially you


can block crawling.


So you can prevent Googlebot from even looking


at your pages.


And with the robots.meta tag, when


Googlebot looks at your pages and sees that robots.meta tag,


you can do things like blocking indexing.


In practice, both of these kind of result in your pages


not appearing in the search results,


but they're subtly different.


So if we can't crawl, then we don't know what we're missing.


And it might be that we say, well, actually, there's


a lot of references to this page.


Maybe it is useful for something.


We just don't know.


And then that URL could appear in the search results


without any of its content because we can't look at it.


Whereas with the robots.meta tag,


if we can look at the page, then we can look at the meta tag


and see if there's a no index there, for example.


Then we stop indexing that page.


And then we drop it completely from the search results.


So if you're trying to block crawling,


then definitely robots.txt is the way to go.


If you just don't want the page to appear in the search


results, then I would pick whichever one is easier


for you to implement.


On some sites, it's easier to set a checkbox saying that I


don't want this page found in search.


And then it adds a no index meta tag.


On others, maybe editing the robots.txt file is easier.


Kind of depends on what you have there.


All right, let's see.


Are there any negative implications


to having duplicate URLs with different attributes


in your XML sitemaps?


For example, one URL in one sitemap


with an hreflang annotation and the same URL in another sitemap


without that annotation.


So maybe first of all, from our point of view,


this is perfectly fine.


This happens every now and then.


Some people have hreflang annotations in sitemap files


specifically kind of separated away.


And then they have a normal sitemap file for everything


as well.


And there's some overlap there.


From our point of view, we process these sitemap files


as we can.


And we take all of that information into account.


There is no downside to having the same URL


in multiple sitemap files.


The only thing I would watch out for


is that you don't have conflicting information


in these sitemap files.


So for example, if with the hreflang annotations,


you're saying, oh, this page is for Germany.


And then in the other sitemap file,


you're saying, well, actually, this page is also for France,


or in French.


Then our systems might be like, well, what is happening here?


We don't know what to do with this kind of mix of annotations.


And then it can happen that we pick one or the other.


Similarly, if you say this page has been last changed


20 years ago, which doesn't really make much sense.


But say you say 20 years.


And in the other sitemap file, you say, well, actually,


it was five minutes ago.


Then our systems might look at that and say, well,


one of you is wrong.


We don't know which one.


Maybe we'll follow one or the other.


Maybe we'll just ignore that last modification


date completely.


So that's kind of the thing to watch out for.


But otherwise, if it's just mentioned multiple sitemap


files, and the information is either consistent


or kind of works together, and that maybe one


has the last modification date, the other


has the hreflang annotations, that's perfectly fine.


OK, a little bit more technical question, I guess.


I'm in charge of a video replay platform.


And simplified, our embeds are sometimes indexed individually.


How can we prevent that?


So by embeds, I looked at the website.


And basically, these are iframes that


include kind of a simplified HTML page with a video player


embedded in that.


And from a technical point of view,


if a page has iframed content, then we


see those two HTML pages.


And it is possible that our systems index


both of those HTML pages, because they


are separate HTML pages.


One is included in the other, usually.


But they could theoretically kind of stand on their own


as well.


And there is one way to prevent that,


which is a fairly new combination with robots meta


tags that you can do, which is with the index if embedded


robots meta tag together with a no index robots meta tag.


And basically, on the embedded version,


so the HTML file with the video directly in it,


you would add the combination of no index


plus index if embedded robots meta tags.


And that would mean that if we find that page individually,


we would see, oh, there's a no index.


We don't have to index this.


But with the index if embedded, it essentially


tells us that, well, actually, if we


find this page with the video embedded


within the general website, then we


can index that video content, which


means that the individual HTML page would not be indexed,


but the HTML page with the embed,


with the video information, that would be indexed normally.


So that's kind of the setup that I would use there.


And this is a fairly new robots meta tag.


So it's something that not everyone really


needs because this combination of iframed content


or embedded content is kind of rare.


But for some sites, it just makes sense


to do it like that.


OK.


Let me see what else we can squeeze in here.


Let's see.


Another question about HTTPS, maybe.


I have a question around preloading SSL via HSTS.


We're running into an issue where implementing HSTS


into the Google Chrome preload list,


and the question kind of goes on with a lot of details.


But what should we do for search?


So maybe just taking a step back, when you have HTTPS pages


and you have an HTTP version, usually you


would redirect from the HTTP version to HTTPS.


And the HTTPS version would then be the secure version


because that has all of the properties of the secure URLs.


And the HTTP version, of course, would


be the one that is kind of open or a little bit vulnerable.


And if you have this redirect, then theoretically,


an attacker could take that into account


and kind of mess with that redirect.


And with HSTS, basically, you're telling the browser


that once they've seen this redirect,


it should always expect that redirect.


And it shouldn't even try the HTTP version of that URL.


And for users, that has the advantage


that nobody even goes to the HTTP version of that page


anymore, which makes it a little bit more secure.


And the preload list for Google Chrome


is basically a kind of a static list that is included,


I believe, in Chrome, probably in all of the updates.


Or I don't know if it's downloaded separately.


Not completely sure.


But essentially, this is a list of all of these sites


where we have confirmed that HSTS is set up properly


and that redirect to the secure page exists there


so that no user ever needs to go to the HTTP


version of the page, which makes it a little bit more secure.


From a practical point of view, this difference


is very minimal.


And I would expect that most sites on the internet


just use HTTPS without kind of worrying


about the preload list.


Setting up HSTS is always a good practice.


But it's something that you can do on your server.


And as soon as a user has seen that,


then their Chrome version keeps that in mind automatically


anyway.


So from kind of a general point of view,


I think using the preload list is definitely a good idea


if you can do that.


But if there are practical reasons


why that isn't feasible or not possible,


then from my point of view, at least


like only looking at the SEO side of things,


I would not worry about that.


When it comes to SEO, for Google, what really matters


is essentially the URL that is picked as the canonical.


And for that, it doesn't need HSTS.


It doesn't need the preload list that has no effect at all


on how we pick the canonical.


But rather, for the canonical, the part that is important


is we see that redirect from HTTP to HTTPS.


And we can kind of get a confirmation


within your website through the site not file,


the internal linking, all of that,


that the HTTPS version is really the one that


should be used in search.


And if we use the HTTPS version in search,


then that automatically gets all of those kind


of subtle ranking bonuses from search.


And the preload list and HSTS is not really necessary there.


So that's kind of the part that I would focus on there.


Let's see.


Maybe one more question here.


I don't really have a great answer,


but I think it's important to at least mention as well.


What are the possible steps for investigation


if a website owner finds their website is not ranking


for their brand term anymore and they checked all of the things


and it doesn't seem to be related


to any of the usual things?


So from my point of view, I would primarily


focus on the Search Central Help community


and post all of your details there.


Because this is kind of the place


where all of those escalations go


and where the products and the help forum,


they can take a look at that.


And they can give you a little bit more information.


They can also give you kind of their personal opinion


on some of these topics, which might not match 100%


what Google would say.


But maybe they're a little bit more practical,


where, for example, probably not relevant to this site,


but you might post something and say, well,


my site is technically correct and post all of your details.


And one of the product experts looks at it and says,


well, it might be technically correct,


but it's still a terrible website.


You need to kind of get your act together and create


some better content.


And from our point of view, we would


focus on kind of the technical correctness.


And what you really need is someone


to give you that, I don't know, personal feedback almost.


But anyway, in the help forums, if you


post the details of your website with everything


that you've been seeing, oftentimes the product experts


are able to take a look and give you


some advice on specifically your website and the situation


that it's in.


And if they're not able to kind of figure out


what is happening there, they also


have the ability to escalate these kind of topics


to the community manager of the help forums.


And the community manager can also bring things back


to the Google search team.


So if there are things that are really weird,


and every now and then something really weird


does happen with regards to search,


it's a complex computer system.


Anything can break.


But the community managers and the product experts


can bring that back to the search team,


and they can take a look to see, is there something


that we need to fix, or is there something


that we need to tell the site owner,


or is this kind of just the way that it is,


which sometimes it is.


But that's generally the direction


that I would go for these kind of questions.


The other thing that is kind of like subtly mentioned here


is I think the site not ranking for its brand name.


One of the things to watch out for, especially


with regards to brand names, is that it


can happen that you say something is your brand name,


but it's not actually a recognized term from users.


For example, you might say, I don't know,


you might call your website bestcomputermouse.com.


And for you, that might be what you call your business


or what you call your website, bestcomputermouse.


But when a user goes to Google and enters bestcomputermouse,


that doesn't necessarily mean that they want to go directly


to your website.


It might be that they're looking for computer mouse.


And in cases like that, there might


be a mismatch of what we show in the search results


with what you think you would like


to have shown for the search results for those queries,


if it's something more of a generic term, actually.


And these kind of things also play into search results


overall.


The product experts see these all the time as well.


And they can recognize that and say, well, actually,


just because you call your website bestcomputermouse.com,


I hope that site doesn't exist.


But anyway, just because you call your website this


doesn't necessarily mean it will always


show on top of the search results


when someone enters that query.


But that's kind of something to watch out for.


But in general, I would definitely


go to the help forums here.


And I would include all of the information


that you know that might play a role here.


So if there was a manual action involved,


and you're kind of, I don't know, ashamed of that, which,


it's kind of normal.


But all of these informations, they


help the product experts to better understand


your situation and to give you something actionable


that you can do to kind of take as a next step


or to understand the situation a little bit better.


So the more information you can give them from the beginning,


the more likely they'll be able to help you with your problem.


Cool.


All right.


And with that, I think we're pretty much out of time.


At least my meeting room time here is running out.


So thank you for watching so far.


I'll see if we can add the next Office Hours


sessions to the YouTube community page


as well to kind of make sure that we


stay on top of those things that people are asking.


If you like this, feel free to subscribe to the channel.


We have lots more SEO content on the channel


as well, including the JavaScript SEO things


that I mentioned.


And otherwise, thank you for watching,


and wishing you a great time.


English Google Webmaster Central office-hours from April 28, 2020

All right, welcome, everyone, to today's Webmaster Central


Office Hour Hangouts.


My name is John Mueller.


I'm a webmaster trends analyst here at Google in Switzerland.


And part of what we do are these Office Hour Hangouts


where webmasters, SEOs, publishers


can join in and ask any kind of website search-related


questions that might have come up.


Looks like a handful of people are here already,


but I'm sure we'll see more over time.


A bunch of questions were submitted already on YouTube,


so we can go through some of those.


But like always, if any of you want


to get started with the first question,


you're welcome to jump in now.


Or if not.


Hi, John.


Go for it.


Thanks.


Yeah, hi, Martin.


Great to see you as well.


John, I had a quick question about cloaking, basically.


And the context is that we want to add tracking parameters


to our internal link structure.


And like it's always said that adding too much


might not be good because it delays crawling,


and it might be not good for a million URL


website, which we have.


So the data team has suggested that when someone clicks


the URL, they add it on the go, and they block the bots


from accessing those parameters by identifying user agents.


So is that a permissible technique?


Because the content remains the same,


just that the tracking parameters


are added for the users and not the bots.


In general, I mean, it's permissible.


You can do this.


I think it's probably not a great practice.


But in general, it's doable.


It's essentially similar to the old session IDs


that used to be used in the old days, where per session, you


would add parameters to the links on a page.


And then when a user clicks on those links,


those session parameters would be passed on


so that you can track that a little bit better.


In general, for crawling, that's something


that makes sense to suppress.


So that search engines don't run across those.


What I would not do is block them by robots.txt,


but instead maybe use a rel canonical on those pages


to point at the canonical version,


or maybe even redirect to the canonical version


if you can do that.


In general, I try to avoid doing this kind of tracking


through the URL itself, because it makes everything


a little bit messier.


Because then when you look at your log files,


it's a lot harder to tell which are the pages that actually


get the most traffic, how do users in general


navigate around my website, because you always


have these kind of tracking parameters attached


to the URLs, depending on the user,


depending on how they came in.


So I think it's probably suboptimal to do it like that.


It's not illegal in any sense or against the Webmaster


guidelines or anything like that.


It's just, I'd say, not a great practice.


Thank you so much, and hope everyone is safe at business.


Is it counterproductive to target the same keyword


with two pages, or is it actually


an agoromistic penalty for a two-page target


the same keyword?


So for example, if I rank for top four for a page,


if I publish another page targeting the same keyword,


will it drag down the top four ranking,


or is it just counterproductive to publish another page


to target that?


Yeah, so there is no penalty for doing that.


It's not that there is any kind of manual action


or any algorithmic action that would say,


oh, you have two pages, therefore,


they can't be as good.


Most of the time, the issue just comes up


in that you tend to have two pages that are somewhat


kind of midterms with regards to how good they are,


how strong they are.


And if the alternative is to have


one page that is much stronger that could potentially


rank better than any of those pages individually,


then you're kind of trading off having two pages shown


in search, but they're a little bit lower


in the rankings versus one page that's shown a little bit more


visibly in search.


So in that kind of a trade-off situation,


often you will prefer to rank a little bit higher


just because you're a little bit more visible in that case.


There are many cases where essentially there


would be no change in ranking, where maybe the top results are


so strong that there is no chance for you to kind of jump


in there by combining your pages.


Or maybe you're already ranking number one and number two


or number one and number three.


Then it's not that you would be ranking better than number one


if you just had one page.


So from that point of view, it's always


something where I think it's a good idea to look at this,


but also to keep in mind the context


and to think about what the alternative would be.


So don't just blindly see it as something that is bad


and you need to fix, but rather think


about what would the alternative be.


Would it be better for my site if I had one page,


or does it not change anything at all perhaps?


SPEAKER 2


Sure.


All right.


Any other questions before we jump on in with the rest?


SPEAKER 3


I have another question, if you don't mind.


Go for it.


So is it a bad idea that we always use non-canonical URL


in the internal link?


So for example, especially if we implement internal link,


it always have a parameter in that,


and that parameter canonicalized to the one without.


And I see a lot of websites doing this kind of link


throughout the site.


Would that hurt their page rank flow?


It doesn't hurt the page rank flow or the page


rank of the pages in that sense, because we


see those multiple URLs, and we see they're the same content.


They have a rel canonical, so we treat them as one page.


What can happen, however, is that we pick one of these URLs


to use as a canonical instead of the one


that you have specified as a rel canonical.


So that's something which sometimes you'll


see these URLs in search if you do a site query of research


for them specifically.


Sometimes you'll see them in Search Console in the reports.


And it essentially just makes it a little bit harder for you


to kind of keep track of things, because you're looking


essentially at both of these URLs and saying, well,


in my mind, I have to keep in mind


that these are the same URLs, when actually there


are multiple URLs, where you could make it easier just


by having one single URL.


But that's essentially something where,


from a ranking point of view, it wouldn't change anything.


It's really just making it easier


for you to understand your site.


SPEAKER 1 OK.


Also, make Google easier to identify


which one is canonical if I link them directly?


Yeah, when we pick a canonical, we use the rel canonical.


We use internal and external links.


We use redirects, a bunch of other factors


as well to pick a canonical.


So if the rel canonical points at one URL


and the internal links point at another one,


then we're kind of in a conflicting situation.


SPEAKER 2 I see.


So follow up on that.


So in Google Merchant Center, they have a feed, right?


Is that a bad idea to use canonicalized link


in the Google Merchant Center's link?


Or should we use the actual product


page that is canonical in the Google Merchant Center's feed?


I don't know for sure.


I need to kind of check out all of these details


with the Merchant Center team, because they've kind of opened


things up and gone towards the direction of kind of more


organic search results, at least in the US.


But I don't know what their recommendations are in detail.


So I need to double check that everything is the same.


So if I have a question specifically for Google


Merchant Center, what is a canonical resource


that I can go to?


Is that a person or a group that I can get


or take as answer for those things?


I don't know.


I don't know at the moment.


I need to look that up.


It might be that as a first step that maybe the Google Ads


team would be a good guess, because that's


kind of where the Merchant Center so far has been located.


SPEAKER 1 OK.


Thank you so much.


SPEAKER 2 Sure.


SPEAKER 3 John, can I follow up a bit on that last question


regarding internal links and canonicals?


SPEAKER 3 So let's say that you have this internal linking


where all links point to those pages that also


canonicalize to other pages.


But you figure out canonicals and respect them,


and everything is OK from that point of view.


Do they still use crawl budget?


Is this fixing that and putting the actual canonical version


would help with the crawl budget,


or that doesn't make any difference?


Do you still need to crawl those non-canonical version


every now and then because they're


within the internal linking architecture,


or do you kind of figure out, oh, I know what this is about.


I won't really bother crawling that page?


SPEAKER 1 We do still look at them from time to time,


but it's not as often as we would crawl normal URLs.


So usually what happens when we have a set of URLs


and we pick one URL as canonical for that set,


then we will mostly focus on that single URL


and crawl that one primarily.


We'll still occasionally look at the other ones,


but it's not nearly as much as kind


of the normal crawling would.


So when it comes to crawl budget,


it's something where initially we would look


at all of those different URLs.


So if you went to a big site and you


started adding session parameters again to all URLs,


then initially we would get lost with crawling.


But fairly quickly, we would figure out, OK,


these are the canonical URLs.


These are non-canonical URLs.


We will focus on the canonical ones.


And that should more or less still work out.


SPEAKER 2 But do internal links play any role here?


So if you have internal links or not


to those non-canonical URLs, does that


affect how often you'll recrawl, even if rarely,


those versions?


SPEAKER 1 The difference between canonical URLs


is we'll focus on the canonical URLs.


But if it's kind of like 50-50, like we choose these


or we choose those, then it's possible


that we will recrawl those non-canonical URLs a little bit


more often than URLs that we think are completely irrelevant.


SPEAKER 2 OK, so is that the case with redirects as well?


So if you're internally linking to a page that always redirects


to something else, does the internal link,


should you replace that with the final version?


Or you'll kind of manage it on your own,


and it doesn't matter if you internally


link to the redirect page or to the final destination?


I imagine you would see something similar.


I think in practice, this difference


is more theoretical than it is practical.


If you have access to the server logs of a bigger site,


you can probably find pages like that


where you're linking to a redirecting page,


and you can double check to see what the amount of crawling


actually is there.


I imagine for the most part, we just figure this out.


We pick the canonical, and we focus the crawling really


primarily on that.


So I mean, I don't have any numbers to throw out,


but I wouldn't be surprised if it's like, I don't know,


30 to 1 or some really strong ratio like that, where


we'd say we really focus on the canonical URLs.


And every now and then, we'll still


look at the non-canonical ones, regardless


of where they came from.


OK, I'm just asking whether if you have a very big site,


and it's linking to all of these pages that either redirect


or canonicalize to something else,


whether it's worth the effort going URL by URL


or maybe some automatic way to try to replace everything


so it leads directly to the final version,


whether that's worth the effort.


I think if you have a really large site,


and you're doing this on a large scale, I would clean that up.


Like, let's say, I don't know, if YouTube


had redirects from all of the old pages to new ones,


and the internal links still went to the old pages,


that's something I would say is worth cleaning up,


because there are just so many of those URLs.


And if we keep running into the old ones and keep trying them,


then it's just adding such a mass of unnecessary crawling.


But if you have a normal-sized website, even kind


of like a mid-sized e-commerce site with,


I don't know, a couple million URLs,


I don't see this playing a practical role.


AUDIENCE 1 OK, thanks.


JOHN MUELLER Sure.


All right, let me look at some of the questions that


were submitted.


And feel free to jump in in between as well.


Or if you have more questions or comments on the questions


or answer, we can take a look at that.


I heard Googlebot sometimes submit forms.


If so, could it do it if the form is in an iframe as well?


And what if the iframe is hosted on another domain?


How would that impact crawling, indexing, and so on?


So it's extremely rare that Googlebot would submit a form.


It's something where we primarily


did this way in the beginning when websites


were structured in a way that we could not crawl them properly.


In particular, we saw this issue on a lot of government


websites where there was a lot of content on the site.


But to find it, you had to go to a search form


to actually find links to that content.


And for sites like that, pretty much the only way


to get to the detailed content was


to go through the search form.


However, for pretty much every modern site,


we can crawl normally.


And people are used to creating a structure where


we can crawl with categories and subcategories


where essentially we never need to go


through any of the forms.


So I would imagine most of the people who have sites,


who have logs that they can look at,


if you look at the server logs and you look at Googlebot,


you would probably never see Googlebot


submitting any of the forms that you have on the site.


So that's something that's really extremely, extremely


rare and something usually where when it does happen


with a website, it's kind of a sign


that we can't crawl normally, where we realize


there's a lot of content, but we can't actually


find that content at all.


So that's something where if you're seeing this happening,


I would kind of go down the direction of like,


what am I doing wrong?


What could I be doing differently


with regards to my site's navigational structure?


That's, I think, the primary aspect there.


And with that in mind, adding more complexity,


like iframes or other domains, I suspect a lot of that


would just not happen, just for practical reasons.


Because we want to avoid running into a situation where we


accidentally enter things like credit card numbers


and accidentally Googlebot goes off and buys things


or fills out some contact forms with random information.


All of that doesn't really make sense.


And it causes almost more problems than it helps anyone.


So that's something where I imagine


if you have a configuration with iframes and other domains,


you would probably never see Googlebot go through that.


The one thing where sometimes you


will see something like this happening


is if you have an AJAX-based website or JavaScript-based


website in general, where you're using some kind of a POST


request to get data to load on a page,


then that's something where Googlebot


might be executing that.


So if that's a part of your page's rendering,


that it does a POST request to an API


and then it gets some answers and displays those answers,


then that's something where when we render the page,


we might be doing that.


It's not that Googlebot is crawling those POST requests


for the form data, but just that in the process of rendering


your page, if there's a POST request,


we'll try to do what a normal browser would do.


And we might show that.


And in a case like that, we follow,


I imagine, the normal browser security guidelines.


So doing things like cross-domain, I believe,


is just a lot harder.


I don't know offhand what the defaults are there


with modern browsers.


But that's something where it's just like you're


adding more complexity again.


We had two goals in our mind.


The first is showing ratings as rich results for our seller


pages when someone searches for a seller name plus reviews


or ratings.


And the second is showing a search box when


someone searches for our brand.


In February, for case one, we implemented organization schema


on our seller page, added organization attributes


like brand name, URL, logo, address, reviews, ratings.


For case two, we implemented site link search box schema


on our home page.


The result one, in Search Console enhancement reports,


we started seeing logo section, where


we have around 250 valid items.


But in performance, we haven't gotten any rich results


for our seller pages yet.


And the second, in Search Console section,


we started seeing one valid item.


But we're not seeing any search box when we type


a brand name in Google Search.


My boss is upset with me now.


My thoughts for the next step for the seller pages


are remove all organization schema


and implement ratings only.


One of our competitors is doing the same.


Is there any hidden mistake that I should take care of?


Our main focus is the search box.


So for reviews, you definitely need


to watch out for the guidelines that we


have with regards to review markup in our developer


documentation.


In particular, reviews are only available for a certain set


of items that you mark up on a page.


So it's not the case that you can take anything and just


add reviews, and Google will show those reviews in search.


But rather, we only show them for a certain set


of structured data elements.


So I'd double check to make sure that you're following


the guidelines there and that the information that you


are marking up matches the policies


that we have for reviews.


So that's probably the most important part.


With regards to the sitelink search box,


this is kind of a tricky one and something


I see people struggle with from time to time.


The hard part here is that adding the markup


does not make it more likely that a sitelink search


box will be shown.


But rather, if we were to show one,


we would use one that's based on your markup.


So it's very rare, or I don't know,


it feels very rare that we would show a sitelink search


box in general for queries for sites.


And only for those cases where we would show it,


if you have the markup, we'll try to use the markup.


If you don't have it, we'll just use a default setup.


So that's something where if you're currently not seeing


a sitelink search box at all, then adding the markup for that


will have no effect.


A question on the FAQ schema.


I've seen in documentation what qualifies as an FAQ


and what not.


But for my website, the FAQ was showing for some pages,


now it has disappeared.


But in Search Console, I can still see I'm


applicable to the FAQ schema.


It's really hard to say what exactly was happening here.


So on the one hand, the policies are definitely important.


So I'd watch out to make sure that you're


following the policies.


On the other hand, with structured data in general,


just because you have marked something up


does not guarantee that it's shown in search.


So there are various things that come into play here.


On the one hand, it has to be valid markup.


It has to follow our policies.


It has to be a site that's kind of a reasonably high quality


so that we can kind of trust it.


And all of these things have to allow.


And it's possible that we can do things


like some of the other more visible types of markup.


It doesn't make sense for us to know this,


but every URL that's shown in the search results,


because otherwise everything would just look really messy.


So that's something where also kind of worth


looking at the queries that you're actually targeting


and seeing what the results look like there.


So that's kind of the direction I would go here.


Question about maintaining multiple TLDs that all contain


the same English content.


If a company has a.com and expanded to multiple TLDs


10-plus years ago using the same content written in English,


and the content is generic, does it


make sense to remove the ccTLDs and fold them together


with.com via site migration?


Maybe redirect them back to.com.


Again, the English content definitely


applies to all of those countries,


and Google is already choosing the.com as canonical


for many of those URLs.


Some of the ccTLDs do rank in their country


now based on hreflang magic, where Google is choosing


the.com as canonical and displays the ccTLD


in the search results.


Maintaining those additional ccTLDs


is tough for the company, and it doesn't make sense


to keep a complete duplicate of the.com sitting


on multiple TLDs.


So yeah, I mean, it sounds like you've already figured out


that the ccTLDs aren't as critical


for that particular case.


So that's something where maybe it does make sense


to fold things together and to kind of pick one version


as a canonical version.


In general, I would use country code versions or just generally


local versions when you really have something unique


that you want to do on a per country basis.


So if there are maybe policy reasons or legal reasons


why you need to have individual country versions,


or if you have different content depending on the country


versions, have separate URLs.


But if essentially it's all the same content


and you just have the different country


versions for historical reasons, then it probably


makes sense to fold all of those together into one version


so that you have one clear, one strong version.


Makes it easier for tracking as well.


Makes it a lot easier for maintenance, definitely.


So that's something where I think you've already


kind of analyzed that all of this


is duplication that you don't necessarily need.


So I would tend to fold those together.


AUDIENCE 2 Can I ask a follow up to that?


JOHN MUELLER Sure.


AUDIENCE 2 Just so basically, so if a current brand doesn't


want to expand globally and just stay locally


and sitting on a dot com for 15 years now,


is it worth it to go ahead and redirect the dot com


to the dot CA from a TLD purpose?


I know you guys count that as immediate geotargeting.


There's no need to do that in Search Console, right?


So is it worth it to go ahead and if they


don't want to expand?


I think that's fine too, yeah, so a dot CA


can be active globally as well.


It's not that it's suppressed in other countries.


So from that point of view, if you just


want to have one version and the dot CA


is your kind of primary version, then that's perfectly fine,


yeah.


But the ranking will tank for a little bit


and then it come back?


I think if you're folding things together,


then I wouldn't worry so much about the ranking


because you're kind of taking one existing site


and you're just adding to that.


It's not that you're taking two sites and changing them,


but you're kind of building up on one existing site.


So probably from a ranking point of view,


that should be fairly stable.


OK, thank you.


I have another follow up question on that.


So I have a question about x-default in a multi-lingual


site.


So for example, there is a lot of websites,


they actually redirect their root domain to ENUS version,


but they put x-default to the root domain.


So they are basically putting x-default to a page,


pointing to a page that being redirect.


So does that make sense?


I mean, that page is not 200.


And I think a lot of time, the root domain actually


has more back link than the ENUS URL.


So in this case, is that actually a good idea


to just put a root domain as the ENUS alternative


instead of redirecting to ENUS and put ENUS as ENUS


alternative?


I think in a case like that, where you have a root URL


that's redirecting depending on the country version


or the country of the user, that's


something where we've said in the past


that using the x-default is fine.


Essentially, what you're saying with the x-default


there is that on the root URL, you


have your own logic for handling users of different countries.


And if you tell us these are the known country versions,


then we will send people there when


we recognize they're from there.


But if people from other countries come to your website,


you're telling us, please send them to my root URL,


and I will decide where they should go.


And from our point of view, that's


a perfectly valid decision for you to make.


So from what I hear is that if they're using browser detection,


then it's OK to put x-default to a root domain


that being redirect.


Yeah, yeah, definitely.


The only thing I would watch out for


is the individual language and country versions don't


do the browser detection there.


So on the root, do the browser detection.


But if you send them to the EN US version,


then keep them on the EN US version


even if their browser happens to be in French suddenly.


Don't do.


Oh, sorry.


Another follow-up question is that I


heard that it's actually not a good idea


to do browser detection from Google's guidelines.


So should we prevent just not doing it on root domain also?


So for example, root domain is just the content of English,


right?


So should we just redirect it, the root domain,


to EN US no matter what?


Or maybe just put root domain as EN US instead of redirecting?


If we want, we don't want to do browser detection anymore.


I think that's up to you.


From our point of view, if you have one page where


you're doing this kind of browser or country detection,


that's fine as long as we can really


access the other versions.


So that's kind of the tricky part, where sometimes we've


seen people on the EN US version,


they also check for browser and location,


and then they redirect, which sometimes results in Googlebot


finding, for example, the French version.


And Googlebot crawls from the US,


so it gets redirected to the English version.


And then we would never be able to index the French version.


But if you only do this kind of detection on the root URL,


where there's actually no content,


then that's not a problem for us.


I see.


And that's not.


So in this case, we detect in a browser


when people go to the root domain.


And root domain has a very high backlink, right?


But it redirects different places according


to their browser or IP location.


How would the page rank work in this case then?


So it's tricky.


Usually what would happen is we would


see the redirect from the root URL to the English version,


and that would be the one that we would use for kind


of page rank calculation.


But with hreflang, we can swap out the different URLs.


So it's something where usually that just works out.


It's not that you have to do anything special with regards


to page rank in a case like that.


OK, thank you so much.


Sure.


Hi, Mr. Mahler.


Hi.


I'm Bahan from Search Engine Journal.


So we were experimenting with Google Newsstand app


and built their sections of our website


by submitting either an RSS or web location.


So on certain categories, we don't have an AMP,


and we have to submit their web location.


But when I open a Google News app on the phone,


open the section on the app, I have


found that articles are not sorted by the freshness.


They are sorted by relevancy.


And the first article I see, I see articles from March.


And the most fresh articles in that case are pushed bottom.


But sometimes I see fresh ones first.


So is this something that was done intentionally


or is it kind of bad?


I have no idea.


I don't know how the Google News app deals


with those kind of things.


Intuitively, I think it's something I mean,


I only see this in Discover when I look at Discover.


But in there, I feel that's something


where we try to understand what the relevance is.


And sometimes it's just not the newest stuff


is the most relevant.


But I have no idea how that's handled on the Google News


side.


So that's something where it's probably worth


checking with the Google News team.


I believe in the News Publisher Help Center,


there is a contact form that you can use where you should


be able to reach out to someone from the Google News team.


And maybe they can help you with this.


It's a bit of a tricky question because it's almost


like a ranking question.


But especially if you find something


that looks to you more like a bug


rather than kind of like a misunderstanding,


then that's something I would let them know.


Thank you.


Thank you, Mr. Maro.


I've seen that as well and have given the News team


feedback about that.


It used to also be on desktop.


You only had articles on Google News from the last 30 days.


And now sometimes they'll be months old.


And it's old news sometimes on Google News.


So when we did use a feed as a section source,


it does pull order correctly by the freshness.


But when we did switch to a web location,


and I want to add to it that we have a list item schema


implemented in our category pages.


So it states the order of the articles by that schema


as well, which can help Google understand the order.


So web location kind of goes differently.


Yeah, on the Publisher Center, the web crawl, the web location


does not seem to work as reliably as crawling it


from a feed.


Yeah.


If you have generating the articles by the feed,


it's much more reliable.


Yes, but the question is, the problem there


is if you generate non-AMP articles from feed,


it is going to add inside the News app.


But we want traffic to come to our website.


Right.


And then for that, you have to have the AMP.


Yeah, we have AMP, but on certain categories,


we don't have AMP.


So that is a problem.


Oh, man, Chris, I think we need to do a special session just


with you.


Sorry, I didn't mean to thank you.


I can't hear.


I can't hear a forum here.


No, no, this is perfect.


It's always like people coming in with these kind


of news publisher questions, and I'm like, I have no idea.


And you're one of the people that


has the most experience in this.


So cool to have you here.


What's the status of Google's decision


on the mobile-first indexing after September?


Any delay due to COVID-19?


So, so far, we're still seeing lots of sites shifting over


and getting ready for mobile-first indexing.


So we haven't completely made up our mind


that we will stick with this date,


but we want to give it a little bit more time to settle down


and see how it works out.


My guess is within the next month or so,


we'll make that call and go one way or the other.


It's also really useful to have feedback from any of you


who are running into issues around this, where you're


seeing that your development team just isn't available


to make these kind of changes or anything around that.


The more feedback we have, the easier


it is to just make a decision on this,


because otherwise, we just see sites are being improved,


and we think, oh, well, people are still doing the good work,


kind of getting things ready.


So maybe we don't need to change the date.


But send us feedback.


Question about cloaking.


The case that we have with a site with millions of URLs


is growing, and we want to manage crawl by it efficiently.


I think this is the parameter question


that we looked at in the beginning.


Yes, John.


That's right.


OK.


Is there anything more that you have that we


should cover in that regard?


Just to make a statement and say that, this would technically


be called as cloaking in terms of statements, right?


Yeah, I think it's tricky with just that name,


because there are so many different ways that a website


can be dynamic and kind of subtly be different.


For us, what is really problematic


when it comes to cloaking is when the content is


significantly different.


So when, for example, you have a web page about cars,


and when Googlebot crawls it, it's


a web page about pharmaceuticals,


then that's really hard for us.


And that's something where the web spam team might get involved.


On the other hand, if you're just


adding parameters to individual URLs,


that's something where we tend not


to worry too much about that.


But generally, you would probably


advise to not do it and have a better way of solving it.


Yeah, that's generally what we advise anyway,


just because it makes all of the maintenance


and debugging so much easier.


So it's very common that we run across situations


where maybe someone from the indexing team


will contact us and say, please contact this website


and tell them to stop cloaking.


And then they give us details, which basically say,


they're showing Googlebot something broken,


and they're showing users something that works,


and they're shooting themselves in the foot.


And they don't realize it, because when


they look at the website themselves, it looks fine.


But when Googlebot looks at it, it looks broken.


So that's something where it's very easy to break things


in subtle ways.


So if you can stick to one version,


it just makes it so much easier.


SPEAKER 2


Got it.


Thank you so much, John.


Hi.


I have a question that's related to the unknown traffic


that we are receiving.


I don't know if it's coming from a spam bot,


but while checking analytics, the traffic


is coming from California.


We are based out of India.


And also, I've checked the source.


It's coming from Facebook, and the average session duration


is less than 0.2 seconds.


Even I checked the IP log data, but I'm not


able to arrive at a solution.


So is there any way that I have missed, or can you


just suggest me any solutions?


I don't know.


So the IP address comes from Facebook,


or the referrer comes from Facebook?


Or what are you seeing?


The IP log data that I've taken from our website.


So we have not checked anything from Facebook, sir.


OK.


So in general, if you're seeing something like this,


where it's not coming from Google crawling,


and maybe it looks like bot traffic,


or it looks like traffic that is irrelevant for your website


in general, then from our point of view,


just purely from a search side, you're


welcome to block that traffic.


Because at least from our point of view,


if we feel that normal users, when we send them


to your website, they're seeing a normal experience.


And if you're blocking traffic that you don't want to have


access to your website, that's essentially


up to you between you and those users that you're blocking.


And if you feel that these requests are not actually made


from users at all, and they don't provide you


any value for your website, then maybe that's something


that you could just block.


Is there any way that I can just block them?


Because I'm not able to get the IP location.


So how is that I block them?


That depends on your server and on the setup that you have.


So yeah, sometimes you can check the server logs.


Sometimes that's something where you can just block an IP range


directly on the server and say, don't respond to these IP


addresses.


It might also be the case that your website is set up


in a way that you don't easily have access


to that information.


You can't easily block things.


And sometimes a workaround might be


to use something like a content delivery network in between,


which does give you a little bit more functionality


in that regard.


So that's something where you need


to check it out with the technical people working


on your website and work with them to find ways


to block that kind of traffic.


OK.


SPEAKER 2


Hi, John.


Hi.


I have two questions for you today.


So the first is actually about Ajax calls.


So I was happy to see that Martin is here.


So not actually a dev, so I might


have an infinite scroll implementation that


uses an Ajax call to fetch the, it's


for a category page of articles.


So the Ajax calls to fetch more articles.