But maybe we'll get started with the first ones here.


English Google SEO office-hours from October 8, 2021

Let's see, I'm not quite sure how the order was originally,


because it swapped around a little bit.


But I'll just go from the top.


I'm sure we'll make it.


Kamal.


Hi.


We have an exam website.


People added some exams and give the label for the exams.


And Google indexes all the pages, all the label pages,


like tag pages.


And their role is filtering the exams based on the text,


based on the topic.


And Google indexes all the topics.


But most of them include the same exams,


like a duplicate page.


And we canonicalized these URLs, firstly.


But it caused a lot of crawling time for Google.


For example, we used the third-party tool


to crawl all the pages.


It was about 100,000 pages.


Then we give a noindex on robot text,


and the crawling time degrees.


And it is very fine.


But we want to remove some of these pages which


indexed by Google.


For example, we find them on search results.


What we don't find, don't want to do that.


How we can do that?


We gave noindex, and Google cannot access.


Should we remove them one by one?


I would just leave them as noindex.


And what will happen is, over time,


when we recrawl those pages, they will drop out.


But that can happen maybe, I don't know,


take a couple of months.


But it's not that we would crawl those pages more just because


of the noindex.


Essentially, when we crawl them, we will see the noindex,


and then we will drop them.


And usually, if we see a noindex,


then we would not crawl them that often.


And I think with something in the order of 100,000 pages,


it just takes a while.


But it's not super, super gigantic and impossible


to recrawl.


Does it affect negatively to our SEO performance


in terms of we will see that these are duplicate pages


but canonicalized at the same time now using noindex?


No, no.


It's perfectly fine.


It's purely a matter of the crawling side of things.


If we see that they're duplicate,


then we will try to treat them as duplicates,


and we will just focus on one canonical URL for them.


And yeah, so from that point of view,


I would just let it be reprocessed and drop out


on its own.


SPEAKER 2 OK, thank you very much.


Sure, Lagree, can't hear anything.


I think you're still muted.


Otherwise, we'll come back to you.


OK, yeah, so feel free to reconnect.


And I'll just go to the next one and come back to you


when you're ready.


OK, Michael.


SPEAKER 3 Hi, John.


So you've recommended several times in the past


that large sites, that they focus on smaller set of pages,


I guess.


So the site I'm working on right now,


we have a lot of pages that, a lot of pages, like 1,000 pages


that don't get any traffic, that are old.


So I've been recommending to remove those.


But there's a question that our dev team


has that they were under the impression


that the more pages that Google has indexed of your site,


the higher authority it ascribes to the site


and are reticent to remove any pages.


Could you shed some light on that?


Yeah, so it's definitely not the case


if you have more pages indexed that we


think your website is better.


So I think that, at least, is absolutely not the case.


Sometimes it makes sense to have a lot of pages indexed.


Sometimes they're useful pages to have indexed like that.


But it's not a sign of quality with regards


to how many pages that are indexed.


And especially if you're talking about something


in the order of, I don't know, 1,000, 2,000, 5,000 pages,


that's a pretty low number for our systems in general.


And it's not that we would say, oh, 5,000 pages


is better than 1,000 pages.


For us, it's all kind of like, well, it's a small website.


And we make do with what we can pull out there.


And of course, a small website is relative.


It's not like saying it's an irrelevant website.


It might be small, but it might still be very useful.


But it's certainly not the case that just having more pages


indexed is a sign of quality.


OK, thank you so much.


Cool.


LaGrèe, maybe we'll try you again.


There, can you hear me?


Yes.


Good.


This connecting from work and connecting from home


is different.


Anyway, back in mid-July, we started


noticing a lot of errors in Search Console submitted,


but no index.


The URLs themselves do not have a no index on them.


But on the subsequent crawl, they get indexed.


The problem is that we get 300 errors, no index.


And then on subsequent crawls, only five


get crawled before they recall so many more.


So given that they are no index, and granted,


if things can't render or they can't find a page,


they're directed to our page not found,


which does have a no index.


And so I know somehow they're getting directed there.


Is this just a memory issue or like,


since they're able to get subsequently crawled, fine.


Is it just a?


It's hard to say without looking at the pages.


So I would really try to double check


if this was a problem then and is not a problem anymore


or if it's still something that kind of intermittently happens.


Because if it doesn't matter, if it doesn't kind of take place


now anymore, then like whatever.


It does still take place now.


Like it just crawled on 10.4.


And then on 10.5, it recrawled again.


And the ones that got recrawled were indexed fine.


So I don't know if it's caching it somehow


or if it was just a memory issue.


I'm not sure.


Yeah.


My hunch, without knowing your site,


is that something with the rendering


is sometimes going wrong.


And it's reaching that error page that you mentioned.


And if that's something that still takes place,


I would try to figure out what might be causing that.


And it might be that when you test the page in Search Console,


nine times out of 10, it works well.


But kind of that one time out of 10,


when it doesn't work well and redirects to the error page


or we think it redirects to the error page,


that's kind of the case I would try to drill down into


and try to figure out is it that there are too many requests


to render this page or there's something


complicated with the JavaScript that sometimes takes too long


and sometimes works well, and then try to narrow things down


from that point of view.


OK.


So could it be that if there is too much volume,


then the pages couldn't be so many resources were blocked?


And so then it looks like page not found.


And then when there's less traffic on the website,


the resources are able to be rendered?


Could be.


Yeah, could be.


And I mean, what happens on our side is we crawl the HTML page,


and then we try to process the HTML page in kind


of the Chromium kind of Chrome type browser.


And for that, we try to pull in all of the resources that


are mentioned on there.


So if you go to the, what is it, the Developer Console


in Chrome, and you look at the Network section,


it shows you a waterfall diagram of everything


that it loads to render the page.


And if there are lots of things that need to be loaded,


then it can happen that things time out,


and then we might run into that error situation.


So that's kind of the direction I would go.


And if you can't isolate exactly what is going wrong,


then I would try to take the number of requests


there and just see if there are ways


that you can minimize that.


Maybe the developer team can combine the different


JavaScript files or combine the CSS files,


minimize the images, or things like that.


OK.


OK, cool.


Thank you.


Cool.


Christian.


Hi, John.


It's me again.


Yeah, last time we talked about some problems with a website


where we have an e-commerce website where


we have informational stuff and transactional stuff.


And your advice was to separate these content a little bit


into transactional-oriented and information-oriented pages.


But I have another question regarding this.


If you have, let's say, an e-commerce website


and you have a huge blog or a magazine or something


like that where you have loads of informational stuff,


but it's an old section.


And on the other hand, you have all these product pages


and categories and so on.


So would this huge blog with pure informational stuff


give the whole website kind of an informational touch


or character so that Google says, oh, we are not sure


if this is something where people can get information


rather than buying stuff?


Or is this evaluation done on a per-page base?


I don't think we have that documented or defined.


But my understanding is that this


is more of a page-level thing.


Because it's just purely from trying to think of it as,


I don't know, a practical way, like how you would implement it


and look at websites overall.


A lot of websites just have a mix


of different kinds of content.


And then you try to figure out which of these pages


match the searcher's intent and try


to rank those appropriately.


So my feeling is this is something


more that would be on a page level


rather than on a website level.


SPEAKER 2 OK, so we don't have the risk


that by adding more and more text content


that we kind of dilute the product pages or something?


SPEAKER 1 I don't think so.


I mean, you see that with news websites often


that they have kind of the recent events.


But they also have sections for maybe older events


that took place.


Or, I don't know, 9-11.


Or for other big, big events, they kind of


have an isolated archive section.


And those are very different intents.


Like if you want something really now that is happening,


or if you want some kind of informational research


evergreen-type content.


And there, too, we kind of have to look at it


on a per-page basis and not say, oh, this


is a research website because there's


some research content here.


SPEAKER 2 All right, cool.


Thanks a lot, John.


SPEAKER 1 Cool.


Let's see, Kai Shi.


I don't know how to pronounce your name.


I'm sorry.


SPEAKER 2 Just Kai.


SPEAKER 1 Kai, Kai, OK.


SPEAKER 2 Sure name.


So John, I have this question for you.


We are seeing that people are linking to us


through backlinks to our, let's say, subcategory pages.


And the problem is that after some time,


our content comes and goes, which


means that sometimes there is more content appearing


in some categories.


Sometimes the content gets deleted.


And so subcategories can be created


and can disappear as well.


And we are seeing a bunch of 404s from backlinks


because they are linking to subcategories


that no longer exist.


My question here is, is it OK to redirect these people,


these links to the parent category?


And if we do a show, how do we do that with 302, 302,


for example, like a temporary redirect?


Because in the future, this subcategory


might be populated with content again.


Or it's not a permanent redirect.


Like, you know, it does.


Yeah.


So if we see this happening at a larger scale


that you redirect kind of to the parent level,


we would probably see that as a soft 404.


And we would say, well, the old page is gone.


And instead of a 404 code, you're redirecting.


And maybe that's better for users.


But we see it as a 404.


So from a practical point of view,


I suspect there is little SEO difference


if you redirect or not.


If it makes sense from a user point of view to redirect,


then I would just go for it.


It's not that you have a penalty either way.


So that's kind of the first thing.


With regards to 301 or 302, I don't think it matters there.


Because we would either see this as a soft 404,


or we would see it as a canonicalization question.


If it's a soft 404, then the code doesn't matter.


If it's a canonicalization question,


then it comes down to which URL we show in the search results.


And usually, the higher level one


will have stronger signals anyway.


And we will focus on the higher level one.


So that doesn't matter if that's a 301 or a 302.


But my follow-up question to this


is basically, because we do think


it's better for the user to see the category


page instead of a blank page.


Because the parent category is usually very strongly related


to the category.


But my concern here is, if we do this kind of redirection,


I don't know if it could impact future crawling


of the subcategory when the subcategory appears again,


let's say.


Because as I said, content can come in again


in the subcategory.


But if we did a redirect to the parent in the past,


Google maybe, I don't know, maybe it


doesn't crawl anymore, that subcategory.


That's my follow-up concern.


Yeah, I suspect there is a minimal difference.


But I don't know which one would be better.


So that's kind of the first thing.


Because if we see it as a soft 404, it would be like a 404.


And we would slow down crawling of that particular URL


because there's nothing here.


Why do we have to crawl it every day?


If we see it as a redirect, then we would also say,


well, we don't need to crawl this every day


because we focus on the primary URL.


So I think in both of those cases,


we would slow down crawling of that URL


until we get new signals that tell us, actually, this


is maybe something new again.


And the new signals, I think that would be the stronger


sign.


That would be like internal linking or sitemap file,


things like that.


And that would be the stronger sign for us to crawl again.


But I think the slowing down of crawling


would be similar in all of these cases.


It might be like maybe there's a minimal difference between some


of them.


But I don't know which one would be faster, for example.


SPEAKER 2 OK.


OK.


But OK.


But if we include them again in the sitemaps,


that should be enough to let Google know, OK, maybe


take a look at it.


Yeah, I think sitemaps alone is probably not enough.


I would really make sure that the internal linking is also


clear.


Perfect.


Thank you very much.


SPEAKER 1 Sure.


Granit.


SPEAKER 2 Hey, John.


SPEAKER 1 Hi.


SPEAKER 2 So about a year ago, we


saw some significant decrease in traffic.


After the audit, we kind of all the signals


pointed to the site having site quality issues.


We were able to address those issues by February this year.


And by June core update, we saw some increases.


But it's still not to the level where


we used to be before the decrease about a year ago.


So my question is, the site quality issues,


if that's been the case, is this the recovery


that we can expect?


Or can we expect more recovery if we


think we have addressed all the issues identified?


Or is this it?


Yeah.


I think the tricky part here is it's not so much


that we would consider it as a situation where


you have to fix something.


But rather, when it comes to relevance,


if you work on improving the relevance of your website,


then you have a different website.


You have a better website.


So it's not that we would switch back and say,


oh, it's like the issue is fixed,


and we will change it back to the previous state.


But rather, you're saying, well, this is a better website now.


And we look at it and say, oh, it's a better website.


It's not the same or comparable to before.


So it would be kind of tricky to expect that it changes


to the state it was before.


But it's a new website.


It's a better website.


So that's something where when, I think,


especially with core updates, when


you're talking about recovery, it's not so much


you're recovering, but rather, Google


is seeing that you have a better website and reacting to that.


Understood.


And when we talk about site quality issues,


I think from what we've been able to see,


those were mostly, let's say, technical and user experience


issues and not content quality issues,


meaning that content-wise, I think we are very solid.


But we had more ads than you should have on the page,


and that's been addressed.


And overall, the user experience has been improved,


which all this and everything that we did pointed out


for those to be the reason.


Yeah.


I think it's kind of tricky because with the core updates,


we don't focus so much on just individual issues,


but rather the relevance of the website overall.


And that can include things like the usability and the ads


on a page, but it's essentially the website overall.


And usually, that also means the focus of the content,


the way you're presenting things,


the way you're making it clear to users


what's behind the content, what the sources are,


all of these things, all of that kind of plays in.


So just going in and changing everything around the content,


I think you can probably get some improvements there.


But essentially, if you really want


Google to see your website as something significantly better,


you probably also need to work on the content side,


and at least from the focus point of view


and think about where might there be low-quality content,


where might users be confused when they go to my website,


and is that confusion something we


can address with technical issues, with UX changes,


or do we actually have to change some of the content


that we present?


Understood.


And my last question would be, if that job came not


during a core update, but a regular period of time,


should you expect for the core updates


to happen for you to see a recovery if you


are addressing issues, or should you


expect also recoveries to happen in a non-core update period?


I think if you make bigger changes on a website


regardless, you would see a subtle change over time


as we reprocess things and re-understand the signals.


And if there was something from the core update that


was significantly impacting it as well,


then you would see a jump during the core update.


But at least you should see that subtle improvement over time.


Thank you.


Cool.


Kamar, oh my, we have two that are almost the same.


Go ahead.


Good morning.


I hope this is the right Kamar.


Yes.


Thank you, John.


So my first, my video is off because I'm


having some bandwidth issues.


All my questions are small, and they're around anchor text.


So when we are writing a page on our own website,


and we curate content, for example, from WebMD,


we are taking a snippet and using it


as a point of reference.


And if we were to give a citation to WebMD,


in other words, source URL, I'd prefer to use it in the footer.


So when the person is reading it,


they don't jump off within the article


and jump off to WebMD to learn more.


So my user stays on the site.


Now, there are two thoughts.


Should we give a link back to WebMD, or should we not?


Because I have not found a reference anywhere


that Google wants you to link back.


So why should I give any more link


that WebMD has that, et cetera?


So that was my first question.


What's the best practice from a Google point of view?


I think if you're quoting something,


then linking to the source always makes sense.


So that's kind of just purely from a usability point of view,


I think that would make sense.


With regards to SEO for your website,


I don't know if you would see any particular change


by specifically linking to other people's websites,


because it's one of those also spammy techniques that


used to be used quite a bit, where you would create


a low-quality page, and on the bottom,


you would link to CNN and Google and Wikipedia.


And then you would hope, well, Google


thinks this is a good page, because it links to CNN.


And that's the reason I was talking about linking.


I gave a reference to WebMD, the authority.


But when we are linking to somebody,


there are some legal issues from an FDC guideline


that when you link, you are kind of giving a recommendation.


And the user might think, OK, I'm


going to click here and do something,


and then I have received letters from lawyers,


hey, stop linking to us, because we don't want to be linked.


So that's why I have not linked to people.


So I wanted to know from an SEO perspective.


Now, the second thing is, if I have a content that is off-page,


in other words, I wrote a piece of content on LinkedIn,


and I'm connecting back to my website,


and the question is about anchor text.


So from an SEO mindset, I can say, OK, I want to link back


to my, if my page talks about SEO services,


I can use an anchor text to learn more,


check my SEO services page.


Now, that's an anchor text that worries me,


because while it's a great anchor text that would benefit


me, and from a web accessibility point of view,


that's the kind of link I should use.


But from an SEO, it's like over-optimization coming back


to me.


So how do I handle that coming back?


Because there are risks levels.


I mean, usually what happens is we look at the web,


and we find all kinds of links.


If you're creating content on multiple platforms,


I would try to use useful anchor text that gives us


more information about the page that you're linking to.


So if that's an anchor text that includes SEO services


or whatever, I think that's perfectly fine,


because usually we have a wide variety of things linking


around, and some of them will be like, click this page,


and that's the anchor text, and others will have


more information about the link.


I wouldn't necessarily over-worry


about using a good anchor text.


Instead, I would try to use good anchor text


and link to your content so that it's clear what that content


is.


Also, with internal linking, the same thing, and not say, well,


I need to vary my anchor text, or I


need to make it look like it's not optimized,


because with internal linking, users


want to know what this link is about,


and you want to give that context.


So I would just include that context.


Right, so my point of view of the link


is that link is contextual.


So wherever I'm linking, I want the user


to click there and find the right information.


The clicked link doesn't matter.


And that brings to the third problem.


So when it's on LinkedIn, it's more trusted,


it's coming back.


But now, there's a third problem is people


are doing guest posts all over.


They're going to these low-quality sites.


They're buying guest posts.


So how does Google determine?


And like you said, if it's a good anchor text, use it.


Now, if it's a guest post and Google does not


know whether it's paid or not, how will Google


then determine that take this link or burn this link?


What is the answer so we are safe from all the angles?


Yeah, I mean, if it's a guest post, then it's a guest post.


And our guidance for links and guest posts


is that they should be nofollow.


So if you're writing these guest posts


to drive awareness to your business,


I think that's perfectly fine.


I would just really watch out to make sure


that the links are nofollow so that you're driving awareness,


you're talking about what you're doing,


you're making it so that users can go to your page.


But essentially, it's an ad for your business.


So from that point of view, I would just make them nofollow.


With regards to guest posts in general,


like how does Google recognize guest posts,


I think that's tricky because we use lots of different signals


to try to figure out what might be a guest post


and how we might need to handle that.


But it's definitely not the case that it's just


like the link anchor text is what makes it problematic.


OK, got it.


Thank you so much.


That's all.


Cool.


OK, let me just take a quick break with the live questions


and go through some of the submitted ones


so that we don't lose track.


Stay tuned.


Hang around.


We'll get to you.


Don't worry.


Let's see.


I think the first one is an interesting one that I got.


It's basically, historically speaking,


SEOs have owned title tags.


And recently, Google prefers to show H1 instead of the title


tags.


And you have to consider that the H1 is


a product of a multi-departmental


discussion, which might not be exactly what the SEO team wants.


Why are you rewriting title SEO titles


when you do not do the same for ad titles?


Why do you do this to SEOs?


So on the one hand, I don't know what happens on the PPC


or on the ad side.


I can't really speak for that.


But in general, I think it's a tricky mindset


to say that SEOs own one particular part of the page,


and that is always mapped one to one


in this part of the search results


because these things change over time.


And it's something like the structured data


that is processed can change over time.


In the past, it was that you would


use microdata and things that are embedded within the HTML


for structured data.


Now, a lot of people use JSON-LD,


which is kind of separate.


But all of these things, they evolve over time.


It's not the case that you can always say,


for any given HTML page, this is exactly what the SEO will do,


and this is exactly what the developers will do


and what the content team will do.


These things just evolve over time.


So from my point of view, it's not so much


that we're doing this to annoy the SEOs,


but rather we're trying to improve


the quality of the search results


so that ultimately people search more.


And when people search more, they go to your websites more.


That's kind of essentially our goals here.


So it is not the case that people at Google


sit around and go like, oh, how can I annoy SEOs this week?


That's definitely not what we do, what we spend our time on.


We have so many other normal business problems and work


problems and technical problems to work on,


and we try to improve the quality of our services.


And sometimes that affects what SEOs do,


and sometimes that doesn't.


And when it does affect what SEOs do,


we do try to let you know about these kind of changes.


I think the submitted noindex, we talked about that briefly.


Let's see.


If there are two competing e-commerce sites that


sell exactly the same product, one website


offers a product at $500, the other at $100,


all SEO signals are equal, would the less expensive website


have a better chance of ranking because there's


such a price difference for the exact same product?


So purely from a web search point of view, no.


It's not the case that we would try


to recognize the price on a page and use that as a ranking


factor.


So it's not the case that we would say,


we'll take the cheaper one and rank that higher.


I don't think that would really make sense.


However, a lot of these products also


end up in the product search results, which


could be because you submit a feed


or it could be because we recognize the product


information on these pages.


And the product search results, I


don't know how they're ordered.


It might be that they take the price into account or things


like availability, all of the other factors that


kind of come in as attributes into product search.


So from a web search point of view,


we don't take the price into account.


From a product search point of view, it's possible.


And the tricky part, I think, as an SEO


is these different aspects of search


are often combined in one search results page, where


you'll see normal web results.


And maybe you'll see some product results on the side.


Or maybe you'll see some mix of that.


So that's kind of looking at that.


If we have 200 sitemap files and 20% to 30% of the URLs


jump from one file to another every week, how bad can it be?


Or should we strictly keep our URLs in the same file forever?


So how bad can it be is hard to say,


because we don't really have a measure for badness


when it comes to sitemap files.


But we would generally try to, or our recommendation


is usually to keep the same URL in the same sitemap file.


The main reason for that is we process sitemap files


at different rates.


So if you move one URL from one sitemap file to another,


it might be that we have the same URL in our systems


from multiple sitemap files.


And if you have different information for this one URL,


like different change dates, for example,


then we would not know which attribute to actually use.


So from that point of view, if you have it always


in the same sitemap file, then it's a lot easier for us


to say, oh, we have the information for this URL here.


And we can trust that information


because it's only there.


So that's something where I try to avoid it,


like these URL shuffling around randomly.


But at the same time, it's usually


not going to break processing of your sitemap file.


And it's definitely not going to have a ranking


effect on your website.


So there's nothing in our sitemap system that kind of maps


to the quality of a website.


I'm learning SEO from multiple sources.


And it feels like a behemoth of information.


Do you have a preferred SEO checklist


that will help make the workflow more efficient?


This is in regards to launching a website for small businesses


or helping existing businesses boost their SEO.


So wow, yeah, I don't think we have any SEO checklists.


So that makes it a little bit harder to get started.


What I would recommend doing is looking at the various SEO


starter guides that are out there.


So we have an SEO starter guide.


They're from various SEO tools.


There are also starter guides available that are usually


pretty good.


And for the most part, the starter guides


that I've seen, they have correct information.


So it's, I think, a lot less the case


that people publish something incorrect when it comes to,


especially the beginning side of SEO.


So I would try to go through those


and think about which aspects actually play a role


or matter for your website.


The tricky part, I think, with all of these starter guides,


at least the ones that I've seen,


is they're often based on an almost old school


model of websites where you create HTML pages.


And for the most part, small businesses,


when they go online, they don't create HTML pages anymore.


They go into WordPress or into Wix


or into any of the other common hosting platforms.


And they create their pages by putting text in and dragging


images in and all of these things.


And they don't really realize that in the back,


there's actually an HTML page.


So sometimes when you go through these starter guides,


it can feel very technical and not really


mapped to what you're actually doing when you're


creating these web pages.


Because when we talk about title elements, for example,


you don't look at the HTML anymore and try to tweak that.


But rather, you try to find the field in your whatever


hosting system that you have and think


about what you need to put there.


So that's something where I think over time,


things will probably shift a little bit to cover


that area a little bit better.


But it's something to keep in mind,


that the SEO starter guides, when you look at them,


they might feel super technical.


But actually, the work that you do


is a lot more like filling in fields


and making sure that the links are there and things like that.


Let's see.


I work in the news vertical.


My team is looking to expand our international presence


and have done work to set up multi-regional subdirectories.


For the most part, pages across the different multi-regional


editions will look the same.


Home page and section pages, like politics or lifestyle,


will have similar content minus a few pieces


unique to the region.


The articles are tricky.


There is not much we can differentiate


across multi-regional subdirectories outside


of modules with related links, which leaves us worried


that duplicate content issues.


How does Google handle duplicate content in the news space?


Is it acceptable?


The content stays the same, but elements of the template


are different.


Should there only be one canonical


across all multi-regional websites?


Wow.


OK.


Lots of different aspects there.


So I think taking a step back first,


it sounds like these are different regions


within the same country.


And it's same language content.


So for example, I don't know, different US states


or different regions within the UK, for example,


something like that.


If these are different countries,


then you have the aspect of geotargeting,


which plays a role.


If these are different languages,


so if you're working, say, in Europe


and you're covering Germany and France and Italy


or something like that, then you have different languages


as well.


That's also a very different aspect.


But if you're talking about within the same country,


same language content, then on the one hand,


it's a little bit easier because you


don't have to worry about all of these technical connections.


But on the other hand, the duplicate content issues


are a lot more visible.


And when it comes to duplicate content,


the tricky aspect on sites like these


is that you essentially end up competing with yourself.


And if you have one news article that you publish across,


I don't know, five or six different regional websites,


then all of these different regional websites


try to rank for exactly the same article.


And that could result in that article just not ranking


as well as it otherwise could.


So because of that, I would recommend


trying to find canonical URLs for these individual articles


so that you can really say, well,


I have this one article on my five regional websites,


but this is my preferred version that I


want to have seen in search.


And then we can concentrate all of our energy,


all of our signals on that one preferred version,


and we can try to rank that one a little bit better.


It doesn't have to be the same version all the time.


So it can definitely be the case that you have one news article


that is within one region kind of the canonical,


and a different news article is more


canonical for another region.


How you pick which region you choose as canonical


is totally up to you.


It can be completely random if you want,


but usually you would try to figure out


where is it most relevant and pick that one


as the canonical version.


So that's for, I think, the individual articles themselves


for the categories and the sections and the home pages.


It seems like that would be something


where the content is more unique and more specific


to the individual regions.


And because of that, I would try to just keep


those indexable separate.


So if you have five different regional websites,


their home page, their category sections,


they would all be individually indexed,


and the news articles themselves would


be mapped to one of these different regions.


So that's kind of the approach that we recommend there.


People do it in lots of different variations.


And this approach also just, I guess if you're curious,


also works across different domain names.


So if you have different domains for individual regions,


but it's all a part of the same news group,


you can still do kind of this canonical shifting


across the different versions.


If you're doing it within the same domain


with subdirectories, that's fine too.


Let's see.


HTML semantics versus SEO optimization.


The question kind of goes into like on e-commerce product


pages, should the title of the product


be marked up as a heading?


And from our point of view, that's totally up to you.


From purely a technical point of view,


it can be the case that the product


is a heading on a page.


I think that makes sense in a lot of cases


because you have kind of your business maybe as a heading.


You have the product or the kind of product


that it is as a heading.


That seems to make total sense from a semantic point of view.


So that's kind of what I would focus on there.


So less trying to squeeze kind of semantics


into what works well for SEO, but rather


trying to do what makes sense from a semantic HTML


point of view.


And usually, that maps fairly well to search as well.


Is Google Discover's personalization


based on Google search only or also on YouTube and other


Google products?


I didn't know offhand.


So I looked at the Help Center content for Google Discover,


and it mentions that it's essentially based


on the searches that you do.


So you have to enable, I think, whatever


the feature is that enables the search history in your account


so that that works.


And that kind of suggests to me that it's


based on your search history.


I don't know how the other Google products map into that.


My assumption is just purely from a data protection


point of view, it would be tricky to map


other products into something like that.


So I kind of doubt that is happening.


What is the best course of action


to take when you have to 301 redirect all of the URLs


to a new set of URLs?


The number of pages will be over 1 million,


and you want to minimize the sandbox effect.


If there is a sandbox effect, how long could it be?


Would we lose ranking that we might never recover?


We plan on doing a one-to-one redirect


and had requested batch redirects,


but that's not a possibility.


So pages, images, URLs, et cetera


would have to flip at the same time.


To me, this sounds like a traditional site move


situation.


You move from one domain to another,


and you redirect all of the URLs from your old site to a new one


and we have to deal with that.


And there is, at least from my point of view,


there is nothing like a sandbox effect.


There is definitely nothing defined


as a sandbox effect on our side when it comes to site moves.


So if you have to do a site move, then do a site move.


Then redirect all of your pages.


It's often the easiest approach is just


to redirect all pages at once.


Our systems are also tuned to that a little bit


to try to recognize that.


So when we see that a website starts redirecting all pages


to a different website, then we'll


try to reprocess that a little bit faster


so that we can process that site move as quickly as possible.


And it's definitely not the case that we would say,


oh, they're doing a site move, therefore we


will slow things down.


But rather, we try to process things actually


a little bit faster when we recognize there is a site move.


I have a website that connects to APIs on a client side


to get data.


Are those URLs being included in the crawling budget?


If you disallow those URLs, would that create any issues?


So I think there are two things here.


On the one hand, if these APIs are included


when a page is rendered, then yes, they


would be included in the crawling.


And they would count towards your crawl budget, essentially,


because we have to crawl those URLs to render the page.


You can block them by robots.txt if you


prefer that they're not crawled or not used during rendering.


Totally up to you.


If you prefer doing that, especially


if you have an API that is costly to maintain or takes


a lot of resources, then sometimes that makes sense.


The tricky part, I guess, is if you disallow crawling


of your API endpoint, we won't be


able to use any data that the API returns for indexing.


So if your page's content comes purely from the API


and you disallow crawling of the API,


we won't have that content.


That's kind of the one aspect there.


If the API just does something supplementary to the page,


like maybe draws a map or, I don't know,


a graphic of a numeric table that you have on a page


or something like that, then maybe it


doesn't matter if that content isn't included in indexing.


The other thing is that sometimes it's


non-trivial how a page functions when the API is blocked.


In particular, if you use JavaScript


and the API calls are blocked because of robots.txt,


then you have to handle that exception somehow.


And depending on how you embed the JavaScript on the page,


what you do with the API, you need to make sure


that it still works.


So if that API call doesn't work and then


the rest of the page's rendering breaks completely,


then we can't index much because there's nothing left to render.


However, if the API call breaks and we can still


index the rest of your page, then that


might be perfectly fine.


So those are kind of the options there.


I think it's trickier if you run an API for other people


because if you disallow crawling then,


then you kind of have this second order effect


that someone else's website might be dependent on your API.


And depending on what your API does,


then suddenly their website doesn't have indexable content.


And they might not even notice because they


weren't aware that suddenly you added a disallow there.


And that might cause kind of like indirect effects.


But that's ultimately all up to you.


Cool, OK.


Still a bunch of questions submitted,


but also lots of people's hands.


Maybe I'll go back to some hands until we kind of finish


the recording here.


And I also have some more time afterwards.


So we can try to answer all of the questions.


Kumar.


Hi, John.


So John.


I have also added my question, the main question


that I had on the question list on YouTube channel.


So I'll repeat the question.


So there are two pages that originate


from the same domain.


The URL is a bit different.


This is a part of the same directory structure.


And the way they are generated, they are generated by Next.js.


So Next.js is a server-side rendered React framework.


And they are being indexed, but I


see one page in the Google Caching,


and the second page is not in the Google Caching.


And I see the same pattern regardless


of how I generate the page.


And I mean, there is no set pattern


that this would be in Google Caching,


this would not be in Google Caching.


Most of my pages are in Google Caching, but now I'm worried,


because I'm currently moving from my Java-based text


stack, which generates all these pages, to total Next.js.


And this problem is, while I was debugging, I found that this is


also a problem with the older Java stack that we were using.


So the question is two parts, basically, why this behavior?


And second is, will this behavior impact my ranking?


I see those pages appearing in search results, which


are not in Google Caching, Web Caching.


But will it impact my rankings?


I have another question, but this is important.


OK.


So first of all, the cached pages


are completely separate from what we index.


So if there is a cached page or not,


it doesn't matter at all for ranking,


it doesn't matter at all for indexing.


Sometimes there are technical reasons


why we don't have a cached page, sometimes we just


don't have a cached page for individual URLs.


The other thing is, if the page is using a JavaScript framework,


then whether or not that JavaScript runs on a cached


page is sometimes tricky.


Because the cached page is hosted on a Google domain,


depending on what kind of JavaScript you have,


where it pulls the JavaScript files in,


sometimes the JavaScript can't run on the Google domain.


And from that point of view, if you look at the cached page,


then it can look like, oh, it's an empty page.


And maybe Google is indexing an empty page.


But that doesn't mean that we would index an empty page.


It's just, well, the JavaScript can't run on this page.


Because the cached page is not the rendered page,


it is essentially just the HTML file that we requested


and a copy of that.


And if the HTML file shows something, that's fine.


If it uses JavaScript and the JavaScript doesn't run


because it's a cached page, that's equally fine.


You just don't see it in the cached page.


AUDIENCE 1 So John, the challenge


is when I search this in Google web cache, I get a 404.


Google tells me that this page is not in the cache.


So typical 404 page that you see for the page in the domain.


And the same page, the structure of the JavaScript file,


the structure, everything is the same.


In the URLs that I've given in the question,


the structure and everything is the same.


It's just a content that is varying.


Are you saying that because while Google was indexing


and was requesting that HTML, there


was a problem with JavaScript, that's why it did not cache?


What could be the reason?


I just want to get to the bottom of this.


Because I have around 100,000 categories


which I need to enable.


Then corresponding to 100,000 categories,


around 200,000 products that I need to enable.


And although what you said is correct,


my SEO team has said that this worked better.


But it is still a problem for me.


Something that I don't understand is a problem.


I cannot find a reason.


And it is giving me ABG if you understand.


Yeah.


The cache page, I would not worry about it at all.


It's a completely separate system at Google.


So we have the system for indexing and ranking.


And the system for the cache page is completely separate.


So if the cache page doesn't show,


I would not worry about that.


That's not a sign of any issue.


And often, it's also something you can't control


if there is a cached page or not.


So I would just ignore that.


OK.


Sounds good to me.


Second is around dynamic rendering.


So before we move to Next.js, we had a regular React


application.


I am more of a tech guy.


I'm not an XE expert.


But then, going through certain articles


on developers.google.com, I found


that there is a headless browser tool called RenderTron.


Maybe you have heard about it, right?


So looking at that tool, does it mean


that I don't need to look at server-side rendering frameworks


like Next.js, where Google is also


investing a lot of money, Google's sponsors Next.js?


So should I stop looking at this?


And should I simply get React, generate my single-page


application, and then use RenderTron for when


Google bots hit me?


I redirect that traffic to RenderTron,


which will serve them the rendered page.


That is OK.


I can't give you advice like which approach would be better.


Because I think it depends a lot on the different factors that


are around your website.


However, I would strongly recommend


you join one of Martin's office hours.


So Martin on my team does JavaScript-based office hours.


I think every other week or once a month, something like that,


I would watch out for those and join one of those


and ask him directly and see if he


can take a look at your site, maybe some of the specifics


from your pages, and if he can give you


some more specific advice there.


So I mean, technically, these approaches should both work.


It's not that one works and the other one doesn't


or that one is better than the other.


But it might be that one is easier in your specific case.


OK.


Mine is an e-commerce website.


But OK, what you're saying is I'll join Martin's office hours.


That's all I had, John.


And thanks a lot for your time.


Thanks.


Cool.


Good luck.


Anna.


Hi, John.


Nice to see you for the first time.


As you might have seen, I have a question on that YouTube site.


The thing is that we have in Google Search Console


two domains.


And like in September, everything


was working fine as the Google Search Console only


saved the last 16 months.


And for some reason, in the other domain,


it just saved in that console only 12 months.


I thought it was just maybe the website or reporting problem.


But we have also a connection to export those data to Kebula.


And we also seen those changes there


that it's saving just 12 months.


And I'm not sure who else should I contact,


because we don't have any local support in Czech Republic.


Yeah.


I don't think you can change that.


So that's kind of the first problem.


Usually, this comes from a situation


where the website was verified before and lost verification


and then was verified again.


And probably like 12 months ago was maybe


when the verification happened again.


And usually, when the website loses all verification,


then we stop processing the data.


And we start processing again when it's verified again.


Whereas if a website was never verified at all,


then we try to recreate all of the old data.


So it's kind of a, I don't know, one of those situations


where you're stuck essentially with the data


that you have available there.


It's not that you can recreate that old data.


What you could try, I don't know if it works,


is to try to verify a subsection of your website.


So if you have a subdirectory or a subdomain,


or instead of doing the domain verification,


doing the specific hostname verification,


see if that will trigger kind of like regenerating


the rest of the data.


But usually, you would just have to kind of, I don't know,


keep it as is.


And in the future, it'll fill up again to the 18 months.


But it won't regenerate the old data.


OK.


Well, I forgot to mention, I'm a web analyst.


And my colleague is the guy from SEO.


So I think that if I told him that to try


to verify those subdomains, then we should know what to do.


Yeah.


Usually, it's easy to just add those separately.


But I can't guarantee that it'll regenerate the data.


I think it should.


But I'm not 100% sure.


This change, we've noticed, it happened maybe two weeks ago.


And on September, everything was working fine.


We've seen the six-month, last six-month raters.


But in this month, everything, those four months


just disappeared.


So that's why I have this question.


It's annoying.


I wish it's something that we would handle differently


in Search Console.


But at the moment, it's not something


that you can force it to recreate that.


Yeah, sure.


OK.


That's all.


Thank you for your questions.


Cool.


All right.


Thanks a lot.


I will take a break here with the recording.


And I'll still be here for more of the questions.


It looks like a bunch of hands are still up.


If you're watching this on YouTube,


thanks for sticking around to the end.


If you'd like to join one of these in the future,


watch out in the community section


on our channel for the next versions of the office hours.


And with that, let me take a break here.


English Google Webmaster Central office-hours from Nov 12, 2019

All right.


Welcome, everyone, to today's Webmaster Central Office Hours


Hangout.


My name is John Mueller.


I'm a webmaster trends analyst here at Google in Switzerland.


And part of what we do are these Office Hour Hangouts,


where webmasters, publishers, SEOs, anyone


can join in and ask questions around SEO.


Bunch of stuff was submitted already.


But if any of you want to get started with the first question,


you're welcome to jump on in now.


I have a question if nobody else does.


Go for it.


I know you hate these questions, and it's been a long time since


I asked them.


But over the weekend, there was probably more chatter


in the SEO industry about a Google update


than I've seen since maybe the Penguin or Panda days.


I did reach out to certain people at Google to find out,


not you, but other people at Google


to find out if there's any statement you want to deliver.


Are you aware that I reached out?


Am I aware that you reached out?


Meaning is there discussion internally at Google about maybe


responding to this question?


I mean, we get questions around potential algorithm updates


all the time.


So I think that's kind of natural.


Right.


Anything you could share about this one?


I don't have anything explicit.


I mean, on the one hand, we make changes all the time.


So it's kind of the balance between, well, we


make all of these changes all the time,


and sometimes some individual changes


are more visible, or at least more visible by some folks


or some sites.


So it's kind of the balance between them


as like, well, do we need to call this one out explicitly,


or is this kind of just a normal change?


And I don't know what the final decision there


is, if there's something decided on that yet.


But it's being discussed internally.


Sure.


No.


Like it always is?


This one's specifically more so than others?


But this is anything more than others?


I don't know.


Not more than others, I'd say.


All right.


I tried.


Thank you.


Yeah.


We try to be as transparent as possible about these things,


but sometimes there just isn't that much


that individual sites can do.


So it's kind of hard to say, well, things change.


They always change.


No, it's true.


I did ask Danny last week at the conference


to release another Google penalty,


because I haven't seen a penalty announced in a while.


Hoping, maybe.


No, I'm just joking.


No.


It's your fault.


Yeah, OK.


At least now people know who to blame.


Yes.


Cool.


All right.


Other questions?


Oh my god, a really long one in the chat.


In Search Console, can I please use a better word rather


than valid when it means something very different,


like valid and indexed?


Um, I don't quite understand how you mean that.


If you want, feel free to jump in.


Hi.


Hi, John.


Can you hear me?


Right, sorry.


I keep nagging you about this in various other places.


So if you go into GSC and you look in, say,


the enhancements reports or something,


if I say look in my mobile one, I have a tiny site.


230 pages or something, each in desktop AMP and a light mobile.


And it'll go and tell me, you have 123 valid pages


under mobile usability.


Now, there are no errors in any of the other pages.


And what it means by valid there is apparently, oh, they're


valid and we happen to have indexed them.


Now, it's inconsistent between different things.


If I look at the new thing that's just come up,


the speed one, I get a different number


of things marked as valid for mobile under there.


So clearly, it can't just be valid,


because otherwise the two of them will give the same number.


And all over the place, there are


things that say valid, which clearly, it


was like the old one, the crawler stats, where


it says pages, when actually, I don't


know whether it means pages or objects,


including other files downloaded.


So it'd be really helpful to me, and I've griped about it


for a long time.


Can we find better words than valid


when we don't mean valid, because the implication


is everything else is invalid?


And as far as I can tell, that certainly isn't true.


Oh, OK.


So primarily with the thought that the things that are not


valid might be valid, except they're just not


looking that way.


If they aren't, call me OCD.


But I can tell you for sure that I have checked my pages


with everything going and have been doing it since 1995.


And I'm pretty sure the pages are valid.


I'm quite happy you don't index them all,


but you should say that rather than imply they're invalid,


I think.


OK.


That was all really.


It's just a gripe.


No, no, that makes sense, because I


think these aggregate reports can


be a bit tricky in that sense in the way


that we report over a sample of the URLs.


And we'll tell you, of the sample, this many are valid.


But essentially, we don't say.


I have so few, I don't think the sampling issue even


applies to me.


Well, I mean, it depends a bit on the site.


But it's something where, especially


in the aggregate reports, we do sample them.


So if you add them up, you won't get the same numbers


across the different tools, because we just


look at a sample.


And it's basically saying, of those that we looked at,


these are OK.


But that doesn't mean the ones that we didn't look at are bad.


But I do get that confusion from time to time.


So maybe we need to find a different way to frame that.


That's all.


I think it's some different wording.


Different wording is always tricky with translations


and all of that.


Yeah.


But OK.


Cool.


Thanks, John.


Sure.


And then a longer question from Matthew.


Do you want to maybe just ask that one directly?


Do you have a microphone, maybe?


It might be helpful to explain to people how it works.


I've never used this interface before.


I had to guess a few times to press the microphone thing


to present.


OK.


So there's a big microphone button


if you hover your mouse over your picture,


or you can unmute yourself.


I can't unmute you.


But feel free to jump on in.


I'll try to read it out loud and see if I get it right.


If not, feel free to correct.


We're an e-commerce store.


We have many indexed pages, search pages


that bring in visitors and who end up purchasing.


A few months now, someone has been building thousands


of links to search pages with illegal and sexual anchor text.


Google indexes these dynamic search pages,


and we rank very high for terms like BuyViagraOnline, New


Zealand for several weeks until Google demotes those pages.


That seems kind of awkward.


If we noindex the internal search,


we lose converting traffic.


On the other hand, manually noindexing


these spammy dynamic search pages


is too time consuming because there


are hundreds indexed every week.


What can be done?


I don't know what the best approach is here.


I think the, so in general, with e-commerce sites,


what we'd recommend is if someone is searching for a page


or searching for something that you don't offer within your site


then essentially you're serving an empty search results page,


which ideally you would be returning a noindex for.


So anytime someone goes to a category that doesn't exist


or does an on-site search that doesn't exist,


it should have a noindex on it or a 404 code


so that we know this is not something


that should be indexed.


So ideally, that would be something


that your e-commerce site setup does.


Theoretically, you could also do this with JavaScript.


I don't know, maybe that's something that could be done


with Tag Manager as well where you can dynamically


set the noindex.


I don't know the details of your setup,


so it's really hard to say how exactly you'd


be able to do that.


The other thing is that generally speaking,


we recommend sites don't let their internal search pages be


crawled and indexed because it's very easy to have things blow


up in the sense that we have suddenly millions of URLs


from your website when actually you just have a few hundred


just because of all these different variations


that people can search and that your server returns


valid content for.


So that's kind of the other aspect to keep in mind.


My general approach to this kind of a problem


is to try to find the queries that you do care about


and make sure that those are indexable


and then just make sure everything else is not indexed.


One way you could do this is instead


of having search pages that kind of are there


to drive traffic, that you have more category pages where


you handpick the categories that you do want to have content


shown for and let those category pages be indexed


while the search pages themselves would be blocked


from crawling and indexing.


So that would still allow you to have


these pages that are focused on a specific category


or type of thing that people are searching


for without opening things up for anything that anyone is


searching for that your site might respond to.


So that's kind of the direction I would head there.


All right.


There are a bunch of other questions that were submitted


on YouTube.


But if anyone wants to kind of get another question


in on the side before we dive into that,


you're welcome to jump in.


Or if not, feel free to jump in during one of the questions


if you have more information or more comments.


Oh, we have someone.


Hello.


Hi.


Hey, I actually have a question for you if that's all right.


All right, go for it.


So I'm working on a Swedish website within the Swedish


market.


So we are writing content in Swedish, et cetera.


Some of the subjects, there are not


like enough information when it comes to YouTube videos.


So to give an example, we are writing about vitamin D


deficit, for example.


We want to post a video about vitamin D from YouTube.


Are there like, if we use an English YouTube video,


how would that affect us?


I don't think that would have any effect.


So I mean, any negative effect in the sense


that we would primarily index your web pages


and we would focus on the textual content on your pages.


We would see that there is a video embedded.


So theoretically, what could happen


is that video is associated with your landing page,


with the web page that you created.


So if someone is searching for a video that's


so if someone is searching for that video in video search,


we could theoretically show your page


as a landing page for that.


That might be kind of awkward, I think,


if there's a language mismatch.


But for most videos, we also have good content


on the YouTube landing pages.


So unless this is a video that is really uniquely


available on your site, then it's


probable that we would just pick the YouTube landing


page for searches for the video and pick your web page


for searches around the text that you have on your pages.


So it's not that we would say this is confusing


and we would treat this as bad.


It's more that, well, we don't have


this direct connection between the video content


and the web page.


And that's fine, too.


All right, thank you very much.


Sure.


All right.


So let's jump in to some of the questions that were submitted.


Can single page websites rank if they cover the topic really


well, or do Google's algorithms automatically


block the page from ranking and prefer sites with tons of pages?


Single page websites can rank.


They can rank fairly well.


There is nothing against single page websites per se.


The one thing I would kind of caution,


though, is that it's a lot easier


to build up your website over time


if you kind of keep building on your existing website


rather than creating new websites


for each individual page that you create.


So if this is a topic that you already have a website on,


I would try to integrate that within your existing website.


If you want to create a single landing


page for something like an ad campaign,


if that's online or offline, then obviously that's fine.


That's something that can work.


But in general, I kind of recommend taking a website


and building off of that rather than creating


individual one-page websites.


Being able to have one stable place for your general web


presence makes it a lot easier to build up value over time


because people will know, well, this website has


created lots of good stuff.


And it's a lot easier to kind of grow that as a website


rather than say, well, this one page is actually pretty good


because people will be pointing at individual pages


on your website too.


But all of those individual pages


are within your website, which makes it a little bit easier


to rank your website rather than just that one page.


And obviously, within a website, everything


is kind of linked well together.


So you have a clean navigation.


You have a way for crawlers to go from one page to the other


to kind of explore things from there.


So I'd say if you really need to just create one page


and put that online, then go for it.


That's fine.


But if you have a way of integrating this


within your existing website, I generally


prefer to have fewer websites rather than a lot more.


We had a manual action for pure spam


because of how our domain was used before we purchased it.


We purged all the backlinks, fixed any issues,


sent two review requests based two weeks apart


over the past month.


But we're still waiting for a response.


How can we follow up and have the manual action checked?


It feels like it's never going to be resolved.


So waiting for the reconsideration request


to come back is really the right approach there.


There's no way to kind of bump yourself up


to the top of the queue.


Sometimes things get a little bit packed in the review queue.


And essentially, you just need to wait


until all of those are resolved.


And then they'll get to your site eventually.


Resubmitting a site with another reconsideration request


won't push it higher, and it won't push it lower.


It'll essentially keep that initial reconsideration request


in the queue, and it'll get processed when it comes.


With regards to pure spam, that's


something that wouldn't be related


to backlinks that would purely be


related to the actual content.


One of the things that the reconsideration team does look


out for, though, is that there's actually a website


live on that website there.


So if you go and just delete your whole website


and then do a reconsideration request for a pure spam issue,


then the reconsideration team will look at it and say,


well, there's nothing here that we would rank.


Therefore, it doesn't really make sense


for us to process this reconsideration request,


because there's nothing here now.


If we remove the manual action, there still


won't be anything here.


So there's nothing big to change there.


So if you're just starting or if you're cleaning up


a pure spam manual action, make sure


that you have normal web content on the website


so that the reconsideration team can look at it and say,


well, there's good stuff here.


We need to make sure that it's indexable.


Another thing maybe to mention, especially with the pure spam


manual action, is that it removes things completely


from indexing as well.


So if the pure spam manual action


has been in place for a while, then our indexing system


will have to start rolling again for your website, which


sometimes takes a few weeks as well.


So when you get the reconsideration request back


and it says, all is clear, you're ready to go,


that doesn't mean your website will suddenly pop back


into the search results.


That means our systems can then start working on it


and maybe a couple of days later, maybe a week or so


later, that content will start to appear in Search Console


and in our indexing system.


That sometimes just takes a little bit longer


to get back into play again compared


to some of the other manual actions.


Or for example, if a website was hacked temporarily


and got the pure spam manual action for a really short time,


then things would still be in our index, more or less,


and we can just reactivate that.


But if it's been out for a longer time,


it takes a little bit longer.


SEBASTIAN Frances families


indexing, given that there are crappy backlinks out there


and references?


I would double check what kind of bad backlinks they are.


If it's something where the previous webmaster really


did some shady link building, then that's


something that you might just want


to disavow on a domain basis.


Maybe take the biggest ones or the ones


that look the worst to you and just disavow those.


But in general, that's something where purely


from an indexing point of view, that wouldn't


be playing a role there.


OK, brilliant.


Thanks a lot for the clarity.


And then in that case, I guess just a waiting game.


We'll wait a couple of days, a week or so,


and hopefully see some results.


Yeah, yeah.


What you'll probably see is that the crawling side


takes a little bit to start rolling again.


And then after things have been crawled,


then the indexing side can happen again.


So if you're watching the server logs, if you have access,


then you should see at some point,


well, crawling is starting.


And then you know it's not going to be too much longer


until indexing is back.


Cool, thank you very much.


Sure.


A year ago, we disavowed a long list of links


we thought were hurting our site.


Within that list, there was a list of about 15 to 20 links


we were not so sure about and disavowed them anyway.


After reviewing our disavow file,


we'd like to check if those links help our site or not.


Should we remove them from the disavow file all at once,


one by one every couple of weeks?


Or is this not a good approach at all?


How should we go about it?


So from my point of view, if you're


unsure about certain links in your backlink profile


and you know these were not created in a bad way,


then I don't think you need to disavow them.


I'd really only try to disavow things


where you're really sure that actually there's


something bad behind that that you aren't able to clean up.


Or if you're in a situation where you're like,


oh, losing sleep because you don't know what Google systems


are doing and you just want to make sure that it doesn't


misinterpret something, then of course, this disavow


is probably not necessary.


But it's good for peace of mind.


So from my point of view, if you don't


want to disavow those anymore, I would just


remove them from the file and move on.


My guess is with 15 to 20 links that you're not sure


are really terrible, my guess is those


wouldn't play a big role in your site's ranking.


So it's probably not something where


you'd see a jump in traffic suddenly because those links


are counted again, but rather maybe a subtle change


over the long run.


But if you're unsure about these and you're


talking about a handful, 20 links, something like that,


then I wouldn't worry about it.


I would just remove them from the disavow file


and let them be processed normally.


SPEAKER 2 Hi, John.


This is actually my question, and I kind of


wanted to follow up on your answer.


So if those links were actually spammy links


and we removed them from the disavow file all at once,


will that affect badly on our website?


Essentially, we would try to process those links when


we try to recrawl those pages, and we would just treat them


as normal links to your site.


And our algorithms do have some protections in there


where when we recognize that something is spammy,


we try to ignore it.


So that's something, ideally, they


would help your site a little bit.


In the worst case, they would probably just be ignored.


SPEAKER 2 Okay, I understand.


Great.


Thanks.


SPEAKER 2 Sure.


Due to a technical problem, we had a period of seven


to eight weeks, a duplicate content issue.


Pages from our main site were one-to-one served


on our verticals without a canonical description.


We fixed the issue recently, and on the wrong pages,


the verticals returned 404s.


Do we have to expect a penalty for this,


or are we already seeing the results of such a penalty?


So maybe the first one before we move on to the other questions,


no.


This would not result in a penalty.


This would not result in a manual action


from the web spam team.


It's also something that algorithmically,


when our algorithms look at your site,


they're not going to say this is a spammy site just because you


have some duplicate content on a kind of like where you're


mirroring things on a per page basis.


This kind of duplicate content is very common.


A lot of sites, for example, have the same content


on the www version as well as on the non-www version.


Technically, that's duplicate content.


In practice, that's kind of the way the internet works.


It's no big deal.


So definitely, from a manual action point of view,


there is no manual action for duplicate content.


The only time a manual action would


come into play with duplicate content


is if your website is essentially scraping


a bunch of other websites.


And it's not so much a matter that kind of like you


have some duplicate content across your website,


but rather your whole website is duplicated from other websites.


There is nothing kind of valuable for us


to index on a website like that.


So that would be a place where the web spam team would say,


maybe we need to take a manual action.


But this kind of thing, where individual pages are just


mirrored, no problem.


I mean, technically, it's good to clean that up.


It improves indexing a little bit,


makes it easier for us to focus on the right pages.


But it's not that you're going to see a big negative drop


in search because of that.


The verticals in question show over 2 million pages


indexed in Search Console.


Is it a good idea to speed up the 404 recognition


process with an upload of a site map for the verticals,


including explicitly the wrong URLs?


Theoretically, you can do that.


In practice, I don't think that would make a big difference.


So what is probably happening in a case like this


is we would index these pages individually.


And when it comes to showing them in search,


we would say, well, we have a few copies of the content.


We will pick one of these pages and just show that.


So we're already folding them together


when it comes to the search results.


So it's not that you need to remove these from search


immediately.


With regards to having them be processed a little bit faster,


having the graphs go back to a more normal situation


a little bit faster, that's something


where you could do something like this, maybe a site map


file of pages that are now 404.


Or if you have redirects in place,


they link to those redirects from the site map file.


That could help there.


I think in this particular case, if you're not


worried about showing something wrong in the search results,


I wouldn't necessarily worry about it.


So you can probably trigger these pages to show in search


if you do a very tricky site query.


But for normal queries, we would probably


pick your primary pages.


Hi, John.


Hi.


So John, actually regarding site map, XML site map,


I was thinking something in my mind.


So how would Google behaves if I have a list of the 404 pages


in XML site map?


So one page it goes and crawls, finds 404.


Does it mean that Google is not interested to further


go to other pages because they all are 404?


Or expose them?


No, no, just because you link to 404 pages in the site map file


doesn't mean the site map file is bad.


So we would still use the site map file.


The important part is the last modification date there,


which tells us that something has changed on these pages.


We would still try to prioritize crawling of those URLs.


And so in a case like this, where you have site map files


linking at 404 pages, we would just try to crawl those.


And at some point, we would say, well, it returned to 404.


We don't need to index this page anymore.


And that's fine.


I suspect we would show this in Search Console as well


and say, you submitted pages that returned 404.


But if you're doing it on purpose,


you know that that's going to happen.


So it's not going to cause any problems otherwise.


Cool.


All right, considering that Google is still


at its early stages to understand natural language


written for non-English content, do


you think it's a good idea to use more structured data


on foreign language sites?


This is, of course, to make it easier for the indexer


to understand the content of the page quickly.


That's a unique question.


I don't think I've seen that before.


In general, we do try to understand the language


on a page a little bit better.


Sometimes it works better in some languages,


and sometimes it works really terribly in other languages.


But it is something that we work on.


With regards to using structured data to kind of explain


the entities on a page, I think that's generally fine.


And it definitely wouldn't have any negative effects.


I suspect for the languages where


we have trouble understanding the language,


it wouldn't play that much of a role,


though, because if we can't understand the entities


within the queries, then we would


have trouble mapping that to content on your pages.


And in those cases, we would kind of fall back


to the situation, well, we understand


that these are sentences or that these are words,


and these words are on that page,


and that page makes sense to rank for these kind of queries


with those words.


And so we would try to do it more on a per word basis.


So I think giving us more structured data on pages


is generally a good practice anyway,


but you're probably not going to see


a big jump in kind of relevant search results for your site


by implementing structured data in a language that is hard


for us to properly understand.


SPEAKER 2 Hi, John.


Can you hear me?


John, I have a quick follow-up question there.


JOHN MUELLER Sure.


Wow.


One at a time.


SPEAKER 2 Xavier, go ahead.


SPEAKER 3 Yeah, thank you.


I just wanted to jump on this structured data thing,


because as you may know, we can't pass everything


we want in a company or with clients


to make them implement structured data and other stuff


because they have priority with IT, et cetera.


As of today, the official communication was implemented.


It will be better for Google, but at some point,


will we have some day the future,


an official announcement saying that Google


is taking into account those structured data to enhance,


in a way, the ranking?


Because as of now, we don't have enough arguments


to tell people, yeah, you need to implement this because


reasons, because maybe Google will use it,


and you see where I'm going?


Now, we want to push that kind of stuff to help you,


but you need to have this to beat this one.


Thank you.


Yeah, I think that's tricky.


So the clear thing at the moment is some of the search results


features, the way that we show things,


they require certain structured data.


And I think for those, the argument


is a little bit clearer in that if you want your search


results to be shown in this fancy way,


then you need to give us the information that we can show


there.


So that's, I think, a little bit clearer.


It doesn't provide any ranking boost,


but at least it's a visible change with the search results.


And sometimes, having a page being more visible in the search


results is useful, just the same as ranking a little bit


better.


With regards to using structured data in general for ranking,


I think that's kind of tricky.


So on the one hand, we do use structured data


to better understand the entities on a page,


and to find out where that page is more relevant.


But that doesn't mean that just because people are doing things


in a technically correct way on a website,


that that page is a better page than it would be otherwise.


So we will try to use that to show it in more relevant search


results.


That would perhaps bring more users


to your pages that actually match


the topics of your pages.


But it doesn't mean that we would show it to more users


or that it would rank better.


So that's something where I don't really


see that changing in the future.


That's something that kind of goes back to, I don't know,


way in the beginning, where people would ask,


is valid HTML a ranking factor?


Because clearly, a page that has valid HTML,


they spent more time on that page.


Where from our point of view, well, they


might have spent more time doing the things technically correct.


But does that mean that the page is actually


a better search result?


Is that really something that provides more value to the user?


Or is it just that someone was a little bit smarter


in creating that page, and the actual content


that people would read is not actually that much better?


So that's kind of the tricky balance there.


And I don't see that changing in the sense


that we would suddenly say, any page with structured data


will have a ranking boost because structured data isn't


visible.


So even that would be something we would be ranking things


higher for technical reasons that users would not even


be able to appreciate.


SPEAKER 2 OK, thank you.


SPEAKER 3 Hey, John, a quick follow-up question there.


Do you see, in the near medium future,


Google moving away from structured data


because it's better able to understand natural language?


JOHN MUELLER To some part, I could see it easier for us


to extract the entities of a page over time.


We already do that.


It's something that I think is, I don't know,


probably still in early stages.


But there are lots of things where we still need to rely


on input from webmasters to be able to highlight


that properly.


So things like the rich search results,


where individual elements of a page need to be extracted,


and we need to be able to understand those properly


and to say, well, this is a review count,


and this is the maximum star rating,


and they have four out of five, and so many people


reviewed something.


All of these details are pretty tricky


to extract automatically, even if you


have a really good system.


So you see some of that with the Data Highlighter tool,


where we do try to extract things based


on patterns on your website.


And that's something where you can try that out


with the Data Highlighter in Search Console


to see how well things can be followed,


even if you clearly define which piece of element


is which part on the page.


So I think part of that will definitely remain,


and certainly in the near future.


So if anything, when talking with the engineers,


they come up with more ways to use structured data


rather than fewer.


The one thing that I have seen, which I thought was pretty


cool, is that some of the engineers


have been looking into finding ways


to recognize when a site could be using structured data


to highlight things in a way that we could understand it


a little bit better.


So for example, if we can recognize


that there are events on your pages,


even if we can't extract those events properly ourselves,


we could theoretically send you an email in Search Console


and say, hey, it looks like you have events on your website.


If you want those events to be shown within the events


features in search, maybe structured data


would be an idea to help out there.


So that's probably more the near future.


And the medium future, I don't see it going away there.


In the really long-term future, I


have no idea how that will evolve.


SPEAKER 2 Got it.


Thank you.


That's helpful.


All right.


Is there anything you can advise as crucial for single-page


application sites, which 3.0x code


should we use to not lose anything from a links equity,


and how long should we keep that redirection?


So I guess those are two different questions.


Single-page applications, we have tons


of documentation on that.


So these are essentially JavaScript-based sites


where you have one HTML file.


And by JavaScript, you're essentially


pulling in different content from the server


and creating a different site under individual URLs.


I think the main thing I would focus on there


is to make sure that you have clear, indexable URLs.


That seems to be the main thing that people


run into when they start looking into this problem.


And apart from that, for JavaScript rendering,


we have a ton of information in our developer documentation.


So I would recommend going there.


In particular, Martin has put together a bunch of videos


as well on this topic.


So lots of information on how to make single-page application


sites work well in search.


I think it's still one of those areas that


is kind of leading edge.


So if you're unsure about all of this,


if you're not that technically versed with JavaScript and all


of these things, then it might be something


to try to get some help for from people who have had


a little bit of experience here, just


to make sure that you're not running into any pitfalls


when you're creating things.


Once you're sure that your setup works well in search,


then that's something where it's a little bit easier to just


say, well, it's working.


My content is being indexed.


I can focus on the content and not so much


on the technical details.


Which 3.0x code should we use to not lose anything


from a links equity?


So there essentially, we try to differentiate


between two redirects, the permanent


and the temporary ones.


Permanent redirects really tell us


that everything should be focused on the destination


page, which means that we forward all of the signals


to the destination page.


A temporary one tells us we shouldn't


trust this redirect to remain there forever.


It's temporary.


So we should focus on the source page instead.


And then all of the link signals are focused on the source page.


It's not that you're losing anything.


It's just like, do you want this page,


or do you want this page to have those signals?


That's kind of the decision that we need to make there.


So use the right redirect type for the type of issue


that you have.


Do link clicks from Google My Business pages


count as organic links or direct clicks?


That sounds a bit like an analytics question


that I don't really know from an analytics side.


Within search, it's organic.


OK, it's organic.


So if we're showing kind of that map, what is it,


the local results one box thing with a link to your website


there, then that would count as an impression.


And if someone clicks on the link to your website,


that would count as a click in Search Console.


So that would be fine.


Within the Maps UI, if you're in Maps and you search


and someone clicks on your website,


we would not count that in Search Console.


What you could do, you can add UTMs for Google Analytics


to track it as Google My Business traffic.


Yeah, you can add the UTM parameters there.


And I suspect you can see that in Google Analytics.


It's not guaranteed that you would


see that in Search Console.


The main reason there is that in Search Console,


we would try to show the canonical page.


And the canonical page is probably


the one without the parameters.


So it's kind of hit and miss in talking with the Google My


Business team here.


They're not so sure which direction


they want to take there, which standpoint they


think is the right one.


So I'm not sure how that will remain in Search Console.


In Analytics, I suspect you'll still be able to see that.


And then I think the question is also with regards to Google


Ads.


Would those count as clicks and impressions as well?


Those would not count in Search Console.


So in Analytics, probably.


In Search Console, they wouldn't count,


because they're not a part of the organic search results.


How important is mobile page speed as a ranking factor?


So we did a bunch of stuff, and we improved our score.


So should we be ranking higher, essentially?


We don't have a measure for the importance


of individual ranking factors.


So that's kind of the main thing here with regards


to kind of the meta question of, is it even worthwhile for us


to work on speed if we can't quantify exactly what


the ranking effect is?


I think it makes sense to focus on speed regardless


of any ranking factor, the main reason being


that when people come to your site


and your site is really slow, they're going to leave again.


So if you care about the people that make it to your site,


then make sure that they have a good experience on your site.


And usually, speed plays a big role in that.


And there are different ways of measuring speed.


There are different metrics, a whole bunch


of different three-letter acronyms,


which I can't remember.


Martin knows most of them, but there


are so many different variations,


especially with regards to the different page types.


So if there's something where you


want people to enter something, then maybe you'd


look at something different than if you want people just


to see your content.


So lots of different metrics out there.


I would try to find the metrics that


are important for you, for your site, for your pages.


I would also use the different testing tools


to find kind of the low-hanging fruit


that you can improve fairly quickly.


Pretty much every site has some low-hanging fruit


that you can improve on, which make it easier for you


to kind of improve your score across the board.


With regards to the individual testing tools,


like Lighthouse or PageSpeed Insights,


the score there is mostly a representation


of we look at different factors, and we came up with this score.


It doesn't mean that this is like a target, where


you should aim to be 100 out of 100 all the time,


or that you'll have a clear ranking bonus if you


have everything set up.


When it comes to ranking, we look at a mix of metrics.


On the one hand, kind of these theoretical metrics


that come out of the testing tools,


but also at field data from things like the Chrome User


Experience Report, where we see what people are actually seeing


when they come to your website.


So it's not that you need to gain the PageSpeed Insights


score and get 100 out of 100 there,


but rather, essentially, if you wanted to gain the score,


you would do it by making a really fast website, which


is kind of what we're trying to encourage people to do.


So kind of summarizing all of that,


there is no specific kind of weight of individual ranking


factors that we can give, so I can't tell you


how important this is.


It does make sense to focus on even regardless


of the ranking elements, and we do use it in ranking.


So it is something where we do have an effect in the ranking


side, specifically around speed.


How often do you check the overall quality of content


on a website as a whole?


Is it monthly or more often than not,


or is it only when the core updates happen?


We essentially have different algorithms


that are running all the time.


And they look at all kinds of signals


that we can collect about your content,


and they try to come up with kind of a notion of quality


and a notion of relevance for individual queries


for the website kind of on a running basis.


So it's not that there is a one-time run where


we go across all websites on the web


and look at all of the content and evaluate the quality.


I don't think that would be feasible.


On the one hand, we can't look at a whole website at one time.


We kind of have to crawl individually and spread


that load out over a longer period of time,


just purely from a technical point of view.


And with regards to the core updates,


these are essentially just different ways


of us evaluating the quality of a website.


So it's not that we would go and look at the website again then,


but rather we have collected all of these signals


about this website, about how we think we should treat it.


And then the core algorithm updates,


they essentially say, well, we use slightly different numbers


when calculating the total relevance


for individual queries.


So it's not that we re-evaluate the website then,


but rather that we recalculate the scores.


We built an SEO optimized site with an SSL certificate


and a one-second loading time.


But it ranks below sites that have no SSL, bad user


experience, and take a long time to load.


What tips do you offer to rank above them?


So these are great things.


These are really good technical things to implement.


I think having HTTPS is a great thing to do,


especially nowadays.


That's kind of expected.


One-second loading time is fantastic.


So that's really awesome.


And I think, in general, you're kind of on the right path there.


In practice, that doesn't mean that your site is suddenly


more relevant than other sites out there.


So I would still kind of take this as a good foundation


to build on and then work to make sure


that your whole website is really fantastic.


And ideally, that it's really significantly better


than the others that are out there so that it's


clear to our algorithms that we should show your website


above the others.


And that's something that's not just based on technical detail,


so where you really need to make sure that the content is


really significantly better.


So if you were, for example, just


to take the content from another website


and host it on HTTPS with a really fast server,


then that doesn't mean that content is suddenly better,


because it's essentially the same as the other website


already has.


So what big difference does it make there?


So really kind of take a step back.


The technical details are fantastic,


but you need to kind of deliver the actual content as well.


Is the Google Sandbox real?


What's the impact on a new website?


So no, there is no Google Sandbox.


I think we've said this tons of times,


but these questions come up all the time.


It's something where we don't have this kind of notion


that a new website should be blocked


from being shown in search until it has reached,


I don't know, a maturity of half a year or a year


or something like that.


We don't have that kind of a notion built


into our algorithms.


However, there are certain things


that could look like that.


So in particular, if you're a completely new website,


then we don't have a lot of signals for your website


for the content that you have there.


So essentially what happens is our algorithms say,


well, we don't really know that much about this website,


so we're going to guess.


And in the beginning, that guess might be positive.


That's something that some folks externally


have kind of framed as a honeymoon period where Google


really likes new websites.


It could also be that that initial guess is a little bit


too low, where we say, well, we're not sure about this,


so we'll put it kind of in the mid-range.


And there's some really good stuff here.


So that mid-range versus the really good stuff


is going to be hard competition.


So you might be ranking a little bit lower


than you would in the long run.


And this is the kind of thing where it's not


that our algorithms are saying we should, by default,


block this website from showing up,


but more that we just don't know how


we should be showing this website ideally in search.


So we start off with some assumptions,


and we kind of see where that settles down over time.


So that's probably the effects that you would see there,


where some sites, when they start off,


they start off really well because we think,


well, this looks really good.


And for other sites, we don't really


know where to place them either.


And maybe that starting point is a little bit lower


than it would be when it settles down for the long run.


And obviously, the settling down part is not static either.


Ideally, you would create a website,


and you would grow your web presence.


You would promote it online.


You would bring people to your website.


People would say, oh, this is fantastic.


I've been waiting forever for this.


And they would kind of link to it from other sites as well


and promote it for you a little bit too.


So that's something that's kind of very dynamic as well.


It's not that you put a website up once


and then you never touch it again


and nothing on the rest of the web


changes for a really long time.


The whole web is dynamic.


So continuing working on a website


to make it better over time is definitely


what I would aim for there.


OK.


I think we're running low on time.


So I thought I'd kind of open it up for you


all to see if there's anything on your mind


that we should be focusing on.


Looks like a bunch of back and forth in the chat.


Yes.


Hi, John, I have one question actually about m.site


and www website.


So I know that Google recommends for a response.


But if the website is already running in this case,


we were just going with m.website.


In this case, John, actually, if I


want to go with A-B testing for www, for www redirection


to new URL and then canonical to old one


I have just gone through.


But how should I manage m.website, especially


when it is mobile first index and I


have to provide the equivalent version or equivalent


behavior in both devices?


Or suppose m. not m. but AMP also is available?


Yeah.


So I think you're making it really hard.


With this kind of a setup, if you have desktop, m. and AMP,


and you're doing A-B testing across different URLs,


you're creating a whole bunch of URLs for essentially one piece


of content.


And that makes it so much harder for us to process properly.


So in practice, you would work with the canonicals


to focus both for the m.AB testing versions.


You would set the canonical from the m.B version


to your main version, assuming that the A version is


the main version.


And the same thing on your desktop.


You would take the desktop B version and canonical


to the desktop main version so that we


can understand that connection a little bit better.


You have the same setup with the AMP pages,


where you have the AMP page, the canonical


set to the desktop version or to the mobile version.


But I think with all of this extra complexity


of different canonicals and different URLs


with the same content, it's something


that you kind of have to assume will cause issues in the sense


that our systems might be confused with regards


to which URL they need to actually index.


So if instead of just one URL with a responsive site,


we suddenly have six or more URLs that


all have the same content, and we


have some canonicals and some cross-linking there,


then it can easily happen that our systems pick


a wrong URL for indexing.


In practice, what that means is we'll index the content


on that URL.


We'll pick that as the canonical and show that


in the search results.


If the A-B testing is minimal, then probably that's


no big deal.


If the A-B testing is significant


with a significantly different content,


then we would index that significantly different


content, and that could be reflected in your ranking.


With regards to different device types,


we do recommend using redirects between the device versions.


So if a desktop user goes to the mobile version,


they would be redirected to the desktop version.


So that's less of an issue.


They would still get to the right version.


However, if you're tracking things in Search Console,


or if you're tracking things in Analytics,


and you're monitoring it by URL, then I


would expect to see some amount of confusion


across those different versions.


That's primarily also the reason why


we suggest moving away from an m-dot version


to just the purely responsive version,


because the less confusion you give us,


the more likely we'll do what you ask of us.