Googlebot: SEO Mythbusting

And then there's mobile readiness or mobile friendliness.


Googlebot: SEO Mythbusting

If your page is mobile-friendly, it makes sure that everything is within viewport


Googlebot: SEO Mythbusting

and you have large enough tap targets and all these kind of lovely things.


Googlebot: SEO Mythbusting

And that just is a quality indicator.


Googlebot: SEO Mythbusting

We call these signals. We have over 200 of them.


Googlebot: SEO Mythbusting

That's a lot.


Googlebot: SEO Mythbusting

So Googlebot collects all these signals and then stuffs them as metadata into the index.


Googlebot: SEO Mythbusting

And then when we rank, we're like, OK, so this user is on mobile.


Googlebot: SEO Mythbusting

So maybe this thing that has a really good mobile friendliness signal attached to it


Googlebot: SEO Mythbusting

might be a better one than the thing where they have to pinch zoom all the way out


Googlebot: SEO Mythbusting

to be able to read anything and then can't actually deal with the different links


Googlebot: SEO Mythbusting

because they're too close to each other.


Googlebot: SEO Mythbusting

So that's one of the many. It's not the signal.


Googlebot: SEO Mythbusting

It's one of the many signals. It's one of the over 200 signals to deal with.


Googlebot: SEO Mythbusting

I had no idea there were 200.


Googlebot: SEO Mythbusting

Right.


Googlebot: SEO Mythbusting

I know that you're not allowed to share what they all are


Googlebot: SEO Mythbusting

because there has to be a certain mystique around it


Googlebot: SEO Mythbusting

because of, I guess, a lot of SEO abuse in the past.


Googlebot: SEO Mythbusting

Yeah, yeah. Unfortunately, that is a game that is still being played


Googlebot: SEO Mythbusting

and people are doing weird stuff to try to game us.


Googlebot: SEO Mythbusting

And the interesting thing with this is with the 200 signals,


Googlebot: SEO Mythbusting

it's really hard to say which one gets you moving in the ranks.


Googlebot: SEO Mythbusting

The weights of each signal.


Googlebot: SEO Mythbusting

And they keep moving and they keep changing.


Googlebot: SEO Mythbusting

So I love when people are like, no, let's do this.


Googlebot: SEO Mythbusting

And then look, my rank change is like, yeah, for this one query.


Googlebot: SEO Mythbusting

But you lost on all the other queries


Googlebot: SEO Mythbusting

because you did really weird and funky stuff for that.


Googlebot: SEO Mythbusting

So just build good content for the users and then you'll be fine.


Googlebot: SEO Mythbusting

I feel like it feels like less effort as well than constantly trying to...


Googlebot: SEO Mythbusting

Yeah. But it's not an easy answer.


Googlebot: SEO Mythbusting

You will pay me to make you more successful on search engines.


Googlebot: SEO Mythbusting

And I come to you and say, so who are your users and what do they need


Googlebot: SEO Mythbusting

and how could you express that so that they know that it's what they need?


Googlebot: SEO Mythbusting

That's a hard one because that means I basically bring the ball back to you


Googlebot: SEO Mythbusting

and now you have to think about stuff and figure out strategically.


Googlebot: SEO Mythbusting

Whereas if I'm like, OK, I'm just going to get you links


Googlebot: SEO Mythbusting

or do some funky tricks here and then you'll be ranking number one.


Googlebot: SEO Mythbusting

That's an easier answer. It's the wrong answer, but it's the easier answer.


Googlebot: SEO Mythbusting

So people are like, links are the most important metric ever.


Googlebot: SEO Mythbusting

I'm like, no, we have over 200 and it's important,


Googlebot: SEO Mythbusting

but it's not that important and chill out everybody.


Googlebot: SEO Mythbusting

But this still happens.


Googlebot: SEO Mythbusting

I'm so glad it's better now.


Googlebot: SEO Mythbusting

I feel actually more at peace in general with SEO as well after speaking to you today.


Googlebot: SEO Mythbusting

That's so nice. That's so good.


Googlebot: SEO Mythbusting

So thank you so much for being with me here and it has been a great pleasure.


Googlebot: SEO Mythbusting

Thanks for answering all of my weird and wonderful questions about the Google bot.


Googlebot: SEO Mythbusting

Perfect questions. Perfect opportunity.


Googlebot: SEO Mythbusting

Did we bust some myths?


Googlebot: SEO Mythbusting

I feel like we did.


Googlebot: SEO Mythbusting

Fantastic. I think that's worth a high five.


Googlebot: SEO Mythbusting

Awesome. Thanks.


Googlebot: SEO Mythbusting

Thanks.


Googlebot: SEO Mythbusting

Join us again for the next episode of SEO Mythbusting


Googlebot: SEO Mythbusting

where Jamie Alberico and I will discuss if JavaScript and SEO can be friends


Googlebot: SEO Mythbusting


English Google SEO office-hours from March 5, 2021

All right, welcome, everyone, to today's Google Search Central


SEO Office Hours.


My name is John Mueller.


I'm a search advocate on the Search Relations team


here at Google.


And part of what we do are these Office Hour hangouts,


where people can jump in, ask their questions around Search,


and we can try to find some answers.


Bunch of stuff was submitted already on YouTube,


so we can go through some of that.


But like always, I think it's always


need to start with live questions.


Anything from your side, anyone who wants to get started.


Yeah, I would like to ask the first question.


OK, go for it.


My name is Vahana.


I'm an IT director of Search Engine Journal.


And one question I have about Core Web Vitals


is, is there a possibility that Google can start,


revise that score in Webmaster quicker?


Like, I did updates on the website


and have found that Core Web Vitals are broken after 20 days


because it's updating in weeks, in a long period.


And I have fixed that, but it will take another couple


of weeks to have that fixed.


Is there a possibility that Google can review and make


those updates more quicker?


Not like 28 days, but a couple of days.


I don't know.


I doubt it because the data that we show in Search Console


is based on the Chrome User Experience report


data, which is aggregated over those 28 days.


So that's the primary reason for the delay there.


It's not that Search Console is slow in processing that data


or anything like that.


It's just the way that the data is collected and aggregated,


it just takes time.


So usually, what I recommend when


people ask about this kind of thing, where it's like,


I want to know early when something breaks,


is to make sure that you run your own tests on your side


in parallel for your important pages.


And there are a bunch of third-party tools that


do that for you automatically.


You can also use the PageSpeed Insights API directly


and pick, I don't know, a small sample of your important pages


and just test them every day.


And that way, you can tell if there


are any regressions in any of the setups


you made fairly quickly.


Obviously, a lab test is not the same as the field data.


So there is a bit of a difference there.


But if you see that the lab test results are


stable for a period of time, and then suddenly they


go really weird, then that's a strong indicator


that something broke in your layout in your pages somewhere.


Thank you.


So does that mean that if I see that I have really


bad, poor Core Web Vitals and manage


to fix most of those issues in one day, the next day,


or rather two days later, that would


be part of these 28 days?


Because would it be a rolling update,


or is it every 28 days that it happens?


I don't know for sure.


I think the Chrome User Experience Report


site has that documented.


I'm not sure if it's like a batched update that is just


done every 28 days or if it's more of a rolling thing.


I know in Search Console we have kind of the overtime data,


and it's not that it jumps every 28 days.


So my guess is it's probably more rolling.


All right, I will check that.


Thank you.


So it can still take 28 days to fully, so after one day,


you're 1 28th of the way there to seeing what the results are,


which is why you suggest doing your own test manually.


Yeah, exactly.


But it makes web development very expensive as well.


It makes things harder, yes.


I mean, it's not that we're trying to make things harder.


It's just the way that this data is collected in Chrome.


It just takes time to be aggregated.


So just to know everybody what did


broke the Core Vape Vital score, seems


something was updated on the Core Vape Vital's calculations,


since it seems better.


So we were hiding the navigation menus ticket


when you scrolled down.


And it showed when you scroll up.


And used test's position to animate that, like hide it.


And seems that position animations


did spoil the Core Vape Vital's.


So even though it's a sticky menu,


and it's not causing layout to shift,


but it's calculated as a layout shift,


because you are animating the position of the element.


So we did change that animation from position to opacity.


Like we are just fading out it, instead of moving away


from the screen by position animation.


Yeah, the cumulative layout shift


is sometimes, I think, tricky to interpret and figure out


what exactly is causing issues there.


Yeah, because in the past, it wasn't causing.


Our Vape Vitals were all green.


One day I found that everything is in red.


What could cause that?


And I found that just our test's position animations, which


even don't cause the layout shifts,


are contributing to layout shift calculations.


Yeah, I think for things where you


feel that the calculations are being done in a way that


doesn't make much sense, I would also get


in touch with the Chrome team.


I think especially Annie Sullivan


is working on improving the cumulative layout shift


side of things.


And just make sure that they see these kind of examples.


And if you run across something where you say,


oh, it doesn't make any sense at all,


then make sure that they know about it.


It's not so much that from Google's search side,


we would try to dig in and figure out


exactly why this is happening with that score.


But we kind of need to rely on the score from the Chrome side.


And our goal with the Core Web Vitals and page experience


in general is to update them over time.


So I think we announced that we wanted


to update them maybe once a year and let people know ahead


of time what is happening.


So I would expect that to be improved over time.


But it's also important to get your feedback in and make sure


that they're aware of these weird cases.


Thank you.


Sure.


All right.


Hi, John.


Yes, you raised your hand.


Perfect.


Exactly.


Great, yeah.


So our website consists of a lot of thousands of guides


that we made to help people understand visa requirements


based on their location.


And the way we've designed these guides


are that they're meant to be a one-stop destination.


But that said, some of the content is overlapping.


And there's a risk that Google might interpret


these pages as doorway pages, even though that's not


our intention.


For example, we have a guide for France visa in Los Angeles


and then France visa in Detroit.


And it's personalized to the city.


But a lot of the content is also similar.


And so what is your advice to make sure it's not interpreted


as doorway canonicalized by Google,


given that we have so many of these?


I don't know.


It's hard to say.


So offhand, just hearing that you have city-level pages


for visa advice feels a lot like you're


headed into the direction of doorway pages.


So that's something where I would


be really cautious about expanding


too much in that direction.


Because obviously, you're going to have things like, well,


all of the country information is exactly the same.


And essentially, the difference is the address and maybe


a telephone number opening hours.


And that's something where I suspect our systems,


on the one hand, could be getting confused


from the content side, and where probably,


if someone from the web spam team were to look at that,


they would say this is potentially too much,


especially if you really have, on a city-level,


country-wide information that feels like you could easily


expand that to 10,000s of cities and claim


that it's all unique content, because it's


for every city individually.


And I have some kind of basic information about each city


as well on those pages.


But in the end, it's all about one country,


and the visas for the other country.


So that's something where I would


try to take a strong look at what you can do to try


to concentrate those pages so that you don't end up


in a situation where you're just taking giant list of cities,


giant list of countries, and then crossing them


in a database and generating pages.


SPEAKER 1 Got it.


And then just a follow-up question.


So we'll definitely look into it.


So across all the pages that we have,


Google's indexed only about 100 of them.


And the primary reason, when I look at the crawl stats report,


is that Google only crawls about 60 pages.


And all the other pages are excluded, discovered


but not crawled, which means that they didn't


want to overload our website.


But we have full success rate and really low response time.


How would you suggest we go about getting the crawl budget


up?


Yeah.


So there are two aspects there with the crawl budget.


On the one hand, kind of the technical limitations


of the server, and on the other hand,


the demand from Google with regards to what we think


is important to index.


And it sounds like the technical limitations on your server


are less of an issue.


If you're saying it's a fast server, it responds quickly,


then primarily what you're probably seeing


is that Google is running into a situation where it's saying,


well, I don't know how useful these pages actually are.


And that's something that Google can learn over time.


But it's not something that you can force.


And even if you can force it for a temporary situation


where Google goes off and indexes tens of thousands


of pages from your site, if then it still determines,


well, actually this site is not as useful as I thought it was,


then it will just drop those pages from the index over time.


So assuming this is still the Visa level site,


then that's something where I would strongly


recommend starting off with a much smaller set of pages,


a much smaller set of content so that it's something


that Google can index.


Google can learn over time that actually this


is really good content.


And based on that, it can expand and say, well, OK,


instead of just 100 pages, I'll go and crawl 1,000 pages


or go and crawl 10,000 pages and try


to get those indexed as well.


But especially when you're starting off


with a site that has a lot of content potentially,


I would try to find a way to decrease the size so that you


can grow and become strong first and then expand


to showing more content.


Awesome, thank you so much.


Hi, John.


Hi, so this is Raminder.


I have a question regarding sitemaps,


which is similar to the previous discussion.


So we have a new site, and we have older sitemaps


ranging from 2009 till 2015, say.


And we're facing some of the sitemaps


do have certain issues like images.


Your image URLs have migrated, and because of that,


we're getting issues in the older sitemaps.


So just wanted to check with you how important are the older


sitemaps for the bot, and since we feel it's already


archived in that case.


And secondly, does it help the crawl budget


by removing the older sitemaps?


Yeah, good question.


So with regards to sitemaps, we use


that to better understand which pages we


need to crawl and update.


And if you know that you have sitemap files that


point to pages that don't exist anymore


or that you don't care about anymore, then essentially, I


would try to remove those pages.


So if this is a sitemap file that was useful,


I don't know, a couple of years ago


and your website has migrated in the meantime,


you have a different URL structure,


then telling us about those old URLs


doesn't give us any useful information.


So I would consider regenerating the sitemap files


and really making sure that they contain


the current set of URLs from your website


that you care about.


And that's less a matter of crawl budget, I think,


but more just kind of like general website hygiene


in that you're making sure that Google understands


what you care about and is able to process


that as easily as possible.


OK, you got my answer.


Thank you.


Sure.


Mohak, I think.


You already answered my question.


Oh, OK.


Emre?


Emre.


Perfect.


OK, so recently we implemented structured data


for voice search, that's speakable structured data.


And you would like to understand what will be the impact


and is there any way that we can somehow


understand what percentage of the traffic


come from the voice search or is there any trick


that we can use to extract this information from the search


console?


I don't think you can see anything with regards


to voice search there at the moment.


So that's something people have asked about on and off.


Not for a long time now, but every now and then.


But I don't think there's anything in Search Console that


gives information about how the speakable markup is used


or when that's shown.


So it's almost something that you'd probably


have to try out yourself and see,


is it doing what you think it's doing


or is it not that useful?


OK, thank you.


And I have another question.


Sure.


It's about how we can force Google


to refresh the resources, like the cache, resources cache.


For example, if we change something


with our JavaScript file or, I don't know,


CSS or something like this, how we can force


Google to refresh this file?


OK, so you mean with regards to rendering the page


or with regards to?


Exactly, exactly.


Because we had an issue and the page


was returning an empty page because of this caching issue.


So we, let's say, redeployed the site.


And we saw that Google continued to use the old JavaScript file.


So what we can do about this?


I mean, to force Google to refresh the cache?


I don't think you can easily force it to refresh that.


But there are two things you can do.


One is to serve the files with the appropriate caching headers


so that Google knows approximately how long it


could be cached.


It's a bit tricky there because the caching


that we do for rendering is very different from the caching


that a browser usually does for users.


So it's helpful, but it's not the perfect solution.


The other thing that I've seen people do


is to update the URLs when they make significant changes


within the content.


So I have seen some that use something


like a content hash in the URL for the JavaScript files,


for the CSS files.


And in general, when we try to render an HTML page


and we see a link to a new JavaScript file, which


would be based on the content hash, for example,


then we would know that, oh, this is a new JavaScript file.


We need to take a look at it.


Whereas if you keep the same file names for JavaScript


and CSS, then we might be tempted to just reuse our cache.


AUDIENCE MEMBER 1 OK, thank you.


JOHN MUELLER Sure.


Esther, I think you're next.


AUDIENCE MEMBER 2 Hey, John.


Hi.


We're currently conducting a large site migration


after our company was acquired by another company.


So we're basically lifting and shifting some pages over


to our website amongst our existing content


and changing the domain.


Are there any top five tips on what I should look out


for whilst I'm doing this?


JOHN MUELLER Yeah.


I think the most important part is really


to track the individual URLs so that you


have a clear map of what previously was


and what it should be in the future.


And based on that, on the one hand,


to make sure that you have all of the redirects set up


properly.


So there are various tools that you


can use to kind of submit the list of the old URLs


and check to see that they redirect to the right new ones.


The other thing I would watch out for


is all of the internal linking so that you really


make sure that all of the internal signals


that you have as well, that they're forwarded


to whatever new URLs.


Because what sometimes happens or what I've sometimes


seen with these kind of restructurings


is that you redirect the URLs, you move them over,


but you forget to set the rel canonical.


You forget to set the links in the navigation


or in the footer somewhere.


And all of those other signals there,


they wouldn't necessarily break the navigation,


but they make it a lot harder for us to pick


the new URLs as canonicals.


So that's kind of the effect that you would see there.


It's not so much that it would stop ranking,


but it's more that, oh, we just keep the old URLs for much


longer than we actually need to.


SPEAKER 2


Yeah, sure.


Hi, everyone.


Let me just grab the next one in the list,


and I can get back to you.


Dwayne?


SPEAKER 3


No issues.


SPEAKER 4


Thank you.


So we're rethinking how we display our publication dates


on site, on the front end.


And I wanted to know what the official stance is


on whether we need to have both the published date and update


date, our last modified date, to comply with structured data.


I know that most of structured data


requires that you have some physical element that's


marked up.


So I just kind of want to see what you think is best there.


We have a Help Center page on dates.


I would double check that.


I don't think it's necessary to specify


any date in structured data.


So if you give us some date information,


that helps us to better understand it.


Usually what happens with regards to dates


is we try to pick a date that is relevant for the page.


And we do that based on the structured data


that you have on the page, but also based on things


like what is on the page itself.


So as much as possible, like a lot of SEO signals,


I think it's important to make sure


that all of these things align, that if you're giving us


a date in one place, that it's actually


a date that's visible on the page,


that the time zone is correct, that you specify there,


all of these kind of small details


as well, just that it matches the other information


that we find.


SPEAKER 2


The reason we're thinking this is


because we're seeing that sometimes we


aren't seeing the proper modification


of the page in search.


So we want to be able to have that work properly.


So thank you.


Yeah.


Our systems kind of work in a way


that we pick up all of the dates that we can find


for this piece of content.


And then we try to figure out which ones kind of match up


the best.


And based on that, we try to pick the right date.


And sometimes if you have, I don't know, a news site,


then you'll have related articles or things


like that, where you'll also have dates kind of mentioned


in the text.


And if we don't find clear information in the structured


data for the date, then we might pick


one of these other random dates on the page as well.


SPEAKER 1 You can just follow up one more time on that.


So how does Google decide to show the last modification


date?


Does it need to reflect a significant change


on the page?


Or is there something else you can figure out?


I don't think we have any specific guidelines on when


you should change the dates.


But for practical reasons, I would


try to do it then when you have significant changes


on the page.


So it's not like I just swapped an ad out in the sidebar kind


of thing.


But really, if it's in the primary content


and you made some significant changes,


then that feels like something that you'd want to flag there


for users as well.


SPEAKER 2 Not necessarily me changing the publication date,


but will Google then pick up the modification date


if I made the substantial change versus just showing me


the publication date?


Sorry.


Yeah.


I don't know how we decide between publication date


and last modification date.


I'm not really sure how you'd be able to easily influence that.


I could imagine that we try to see when significant changes


happen on a site or on a page.


But I don't think we have any explicit guidelines on that.


OK, thank you so much.


Sure.


Hi, John.


Hi.


This is Midea Bass.


Again, I am here with a very simple basic question


as I started learning SEO.


So this time, my question is related to headings,


heading structure, basically.


Is it mandatory that we should structure our headings


in ascending order or we can leverage these structure


as per our need?


It's not necessary to have this theoretically correct order


of the headings.


And the number of H1 headings that you have


is also something that is kind of open.


I think for a large part, it makes sense


to try to have a semantically correct heading structure,


because there are things like screen readers that also


rely on headings to navigate to the right part of the page.


So it certainly makes sense to try to get that.


But I wouldn't see it as something


where you would see significant changes in SEO


if you change the order of your headings.


Another question is related to H1 tags.


What's the optimal or ideal numbers of H1 tags


we can have in a page?


Because I have read so many articles of industry experts


that there should be only one H1 tag from the perspective


of SEO.


So what does you recommend?


You can have as many as you want.


So I think in many cases, you have one primary heading.


But if you have multiple, that's fine.


With HTML5, it's also the case that by design in HTML,


you have multiple H1 tags.


So it's something where there is definitely not


a hard limit with regards to the number of primary heading tags.


I think from a kind of a focus point of view


that you have a clear focus on your page,


it makes sense to have one primary topic.


But that can be something that you highlight in other ways.


It could be like it doesn't necessarily


need to be the H1 tag.


It could be something that you have with a big title.


Or if this is your home page, then maybe you


have other ways of structuring what you think


is the primary topic of the page.


So can I say Google gives the importance


on the basis of its structures of headings to the content?


To some extent.


It's not only that.


But it does help us to understand what you think


is important for the page.


And if you say, well, this is the primary heading for my page


and what this page is about, and we


see it matches what you have in your content,


then that's always useful for us to know.


Thank you.


And Rob, you're so patient.


Am I, though?


So can you give us your current take


on the supported image types for the Core Web Vitals and Page


Speed?


Because you guys still seem to be recommending file types


that you don't actually support or are not really


the optimal image types to use.


And I say that because when I look at our Page Speed Insights


scores, they seem to be fine.


We can save 0.15 seconds on JavaScript,


0.9 seconds on deferring off-screen images.


But we can save 12 seconds by changing the file types


to image types like JPEG XP or JPEG 9000 ABC


that you don't even support.


So can you clarify that?


I haven't seen that.


So it's hard for me to say.


I can share my screen.


Maybe not.


No, I mean, just because since we're


talking about images and things, I


don't want it to suddenly be like, oh, it's


like you're showing copyrighted images on your YouTube thing.


So that's the primary thing I'm a little bit worried about.


But if you want to drop your URL in the chat,


like the one that you're looking at,


then I can take a look at that afterwards.


It's just our UK home page, which I'm sure you know.


OK.


But in particular, the time because we


have no problem updating the image to a file type


that you like.


But why is that still the recommended file type


when you guys don't really support JPEG 2000?


I don't know.


It feels like we should support that.


You're saying we don't support it in image search?


Well, as far as I know, and our developer


and whenever you search, Chrome doesn't support JPEG 2000


but recommends it as the recommended file type


to speed up your pages.


That seems kind of an awkward mix.


It seems to be conflicting information.


Yeah.


But it might be also a sign that there are other image


optimizations that you could do to decrease the file size.


But it's still a problem.


But it does it.


It never mentions file size.


Just type.


They're all PNG, which is JPEG.


JPEG 2000.


That seems like forever ago.


OK.


I'll take a look at that.


Interesting.


Cool.


Let me run through some of the submitted questions.


And then we'll have more time for questions from you


all along the way as well.


I'm managing one of the biggest news sites in our country.


I want to improve internal linking for news.


And I'm thinking about including latest news


section in the sidebar so that the latest articles are linked


not only from the home page but also from most pages on the site


as well.


Is this a good approach?


Or is only linking from the home page enough that these,


whoops, enough and those sidebar links


won't necessarily move the needle


or pass additional value?


Usually, this is a good approach.


If they're new or important pages on your website


and you link to them from across the rest of your website,


then that helps us to better understand


that you care about those pages.


And you want them to be crawled and indexed as quickly


as possible.


Usually, when it comes to news sites,


we focus a lot of our crawling on the news site map


that you have and on so-called hubs, which are usually


the home page, the category pages, pages that we


can refresh very quickly where you link to the individual news


articles.


So probably for a reasonably sized news site,


we should be able to pick up all of the new content that way


already.


So it's more a matter of giving us


a signal for the importance of those other pages.


So that's something where if you think


that these new articles are not performing as well in search


as they could be, then linking to them like this


probably makes sense.


If you have other articles that you care about


that you want to be more visible in search,


and that could be something that is older,


it could be something that maybe is a cause


that your site cares about, maybe it's


important information that you need to share for the long run,


then I would definitely also include


that in something like a sidebar or footer links.


So I think overall, it's a reasonable strategy.


I think especially for non-news websites,


it definitely makes sense.


So if it's an e-commerce site, for example,


and if you have new products or related products,


you link to those in the sidebar,


that's a fantastic way for us to get that information.


For news site, we probably can already


find those pages in other ways, and it's


more about giving a subtle hint of importance.


What's the best practice for anchor text


wording on internal links as well as external links?


For example, using the website name, the blog post title,


exact match, or LSI keywords.


So first of all, we have no concept of LSI keywords,


so that's something you can completely ignore.


I think it's interesting to look at LSI when you're


thinking about understanding information retrieval


as a theoretical or computer science topic, but as an SEO,


you probably don't need to worry about that.


With regards to internal links, you're


giving us a signal of context.


So basically, you're saying, in this part of my website,


you'll find information about this topic,


and that's what you would use as the anchor text


for those internal links.


So that's something where, on the one hand,


usually that's something that you


want to give that context to users as well.


So the kind of internal links that you


would use for users usually match what


you would use for SEO as well.


So that's something where, luckily, there's


a nice overlap there.


With regards to external links, if you're linking out


to other people's websites, the same thing.


It's like, supply some context why people should go and click


on this link, what kind of extra information it gives.


With regards to links to your website from other people's


websites, usually that's something


you wouldn't have control over anyway,


so I'd be kind of cautious about what you need to have there.


Second part of the question, is it true


that the more links you have on a page,


the more page authority gets divided amongst the links?


If so, is there a range to be within?


So we do use page rank.


We don't use page authority in our systems.


We do use page rank as a way of kind of understanding


the internal structure of a website a little bit better,


and it does take into account the links on a page.


And we kind of forward the signals across these pages.


But it's not the case that there's


any optimal number of links that you need to have on a page


to make that work well.


Essentially, if you're creating a website that


has a reasonable structure for users


where they can navigate around your website


in a reasonable way, then that would work for search as well.


And especially if you're getting started,


then that's something where I wouldn't necessarily


focus on the number of links on a page


and use that as a metric to try to figure out


how to keep your pages running.


I think at some point, if you have a very large website


and Google has trouble understanding the structure


or has trouble crawling it properly,


then the internal linking structure


is certainly something to look into.


But especially with smaller, newer sites,


that's something you probably don't


need to worry about too much.


John, can I ask a follow-up to last question?


Sure.


Yeah, actually, I just want to know,


if I am linking to other pages on my website from my article,


does Google try to understand the other article which


I am linking to by looking at the article from which I'm


linking?


I mean, Google say, this is 500-word text.


From here, you are linked to this page.


So we may try to understand from 500 words,


get some information about that page which I


am linking to internally.


Does Google do that?


A little bit, but not so much in that random words on a page


will impact how linked pages are handled.


So we take that into account with regards


to understanding the context of the pages that you have there.


Usually, the anchor text is the most important part there.


With images, it's something where


we do take into account the content around that image.


So that's kind of in the direction


that you're saying there.


And it's less in a case like that,


that any random text on a page will count towards that,


but more really around this image or around the link there,


what kind of information is available.


Just for better understanding the page which


you are linking to.


Yeah, exactly.


And you can kind of guess some of this


if the destination page is blocked by robots.txt.


Then there are still some situations


where we would show it in search,


even if it's blocked by robots.txt.


And we would only be able to show it


in search based on the links that we have to that page.


So that's one way to kind of see, well,


what kind of information is Google picking up


from those internal links?


Yeah, OK.


Thank you.


Sure.


My blog has been completed since two months.


I posted content in January.


And in starting, it was an index.


But now, it's not in Google anymore.


What is the reason for the de-indexing of the post?


And how can I get it back into Google?


I don't know.


It's really hard to say without any additional information


there.


So the one thing I would recommend in a case like this


is go to the Search Central help forums, post your URL,


some keywords that you are targeting there,


and try to get some advice from the community.


Sometimes there are technical things


that are wrong on a site that you


need to fix to make it so that we can index it properly again.


Sometimes it's more a matter of quality of individual pages


or of the website overall.


And that's something where the feedback is sometimes


hard to take.


But it's sometimes really useful to get that feedback anyway.


So I would make sure to just provide all of the information


that you can give there with regards


to the pages, the rest of your site, what


all has been happening there.


If there's a web spam manual action that took place


with regards to that page or the site in general,


then you would see that in Search Console.


So probably you've seen that.


For the most part, if you're using a common CMS system,


then probably the technical details are kind of OK.


And it's really more a matter of the quality side of things.


Our log files show that Googlebot


finds a lot of JSON files, even more than normal pages.


Might these kind of pages exhaust our crawl budget?


Does Google crawl API routes, or is this somehow


internally prevented?


If we disallow our API in robots.txt,


how does that affect the XHR requests?


Would they still be able to deliver content?


So super good question.


Yes, all of these requests do count towards the crawl budget.


So that's something where essentially every request we


make to the server that goes through the Googlebot


infrastructure is something that we would count as a request


that we make.


I mean, it is a request.


And we would try to limit that based


on the technical limitations that we


feel apply to your server.


And based on that, we might say, well, we can get more


or we can get less content here.


So in a case like this, you're probably


more looking at the technical limitations


when it comes to crawl budget and less the crawl demand


side of things.


And it's something where if you just look at the server logs


and you see a lot of JSON files that are being requested,


it's not necessarily a sign that something is broken.


It's not necessarily a sign that the crawling


is limited due to these JSON requests that are made.


It's just, well, we happen to make a lot of these.


And it could be the case that we make a lot of these requests,


but we can still crawl enough of the normal content,


and it doesn't really affect the normal crawling


and indexing of the site.


So that's something I would try to figure out ahead of time.


There's no simple way to see that.


There's no information in Search Console


directly that says, oh, you've exhausted


the technical limitations that we think apply to your server.


So that's something where you almost


need a little bit of experience and a little bit of guessing


when looking at the graphs and looking at your servers


and testing individual URLs to figure out,


is it possible that Google is running


into limitations with regards to what it can crawl,


or is it possible that Google is crawling as much as it wants


and it's OK with that?


So that's the first step there.


With regards to blocking these JSON files,


if you do find out that these requests are causing problems,


I mean, one approach is to speed things up.


So the technical limitations are less


of an issue, which could be to move some content


into a static CDN somewhere so that those requests are easier


cacheable, that they go a little bit faster


than the rest of the requests to your site.


That might be an option.


If you decide that you do need to block


the crawling of these JSON files,


then you just need to keep in mind that this does include


rendering of pages that kind of try to embed these JSON files


or that request these JSON files.


So if there is content on your site that is only


visible after rendering pages that request these JSON files,


then that content will no longer be indexable


because we can't get to those JSON files anymore.


So that's kind of the downside of blocking things there.


It's similar if other people on other websites


are using your APIs and kind of using your APIs to generate


content for their pages.


If you block those JSON files from being crawled,


then those pages on their site won't


be able to get that content indexed.


It might be that you don't really care about that,


and that's fine for you.


But it's kind of something to keep in mind there.


Yeah, I think pretty much it with the, yeah, go for it.


Yes, John.


We have a discovery platform, olivash.in,


and we are facing some issue related duplicate canonical


thing.


So the issue is Google is picking the wrong pages instead


of choosing original pages.


Which I think.


OK.


So let me add, I'm from the same team.


So basically, we have 20,000 cities,


and each city has different events on them,


basically listing the events.


So since a few months, we are seeing Google


is picking up wrong city instead of other cities.


If you search about events in Washington,


it will show a result of San Francisco.


And when we check canonical, it has picked up the wrong


canonical.


And the whole page and the content, everything


is different.


Still is picking up a whole different city


for around 2,000, 3,000 pages are affected like this.


There are two possible things that I've


seen that could be playing a role here.


And the first one depends a little bit on the way


that you have your site set up.


I didn't check in this case.


But if you're using JavaScript to generate


some of this city level content, and for whatever reason


we can't process the JavaScript properly,


then it's possible that we see more of a generic page


for some cities.


And then we could say, well, this generic page


is the same as that other generic page.


And then we treat them as duplicates.


And we pick one of them as canonicals.


That's something where you could probably recognize that fairly


quickly if you use like view source on the page.


And you see, well, it pulls in JavaScript.


It doesn't have in the HTML the actual content


from the individual cities.


You can double check some of that with the inspect URL tool


as well in Search Console to see what is the content that Google


picks up after rendering.


The tricky part there is if it's really with JavaScript,


sometimes there are weird, flaky issues,


depending on the server, depending


on the setup that happened, where sometimes we


can render it properly.


Sometimes we can't.


And that's something that's sometimes a bit hard to judge


if that's happening or not.


The other thing is if it's not a JavaScript-based site where


you're pulling in the content with JavaScript,


this kind of duplicate detection can still


happen even if the HTML content is different.


So what tends to happen on our side


is we have multiple levels of trying


to understand when there is duplicate content on a site.


And one is when we look at the page's content directly


and we kind of see, well, this page has this content.


This page has different content.


We should treat them as separate pages.


The other thing is kind of a broader predictive approach


that we have where we look at the URL structure of a website


where we see, well, in the past, when we've


looked at URLs that look like this,


we've seen they have the same content as URLs like this.


And then we'll essentially learn that pattern


and say, URLs that look like this


are the same as URLs that look like this.


Even without looking at the individual URLs,


we can then sometimes say, well, we'll


save ourselves some crawling and indexing


and just focus on kind of these assumed or very likely


duplication cases.


And I have seen that happen with things like cities.


I have seen that happen with things like, I don't know,


automobiles is another one where we saw that happen, where


essentially our systems recognize that what you specify


as a city name is something that is not


so relevant for the actual URLs.


And usually, we learn that kind of pattern


when a site provides a lot of the same content


with alternate names.


So with an event site, I don't know


if this is the case for your website.


With an event site, it could happen that you take one city


and you take a city that is maybe, I don't know,


one kilometer away.


And the events pages that you show there


are exactly the same because the same events are relevant


for both of those places.


And you take a city maybe five kilometers away,


and you show exactly the same events again.


And from our side, that could easily end up


in a situation where we say, well, we checked 10 event URLs.


And this parameter that looks like a city name


is actually irrelevant because we checked 10 of them


and it showed the same content.


And that's something where our systems can then say, well,


maybe the city name overall is irrelevant.


We can just ignore it.


So what I would try to do in a case like this


is to see if you have this kind of situation


where you have strong overlaps of content


and to try to find ways to limit that as much as possible.


And that could be by using something


like a rel canonical on the page and saying, well,


this small city that is right outside the big city,


I'll set the canonical to the bigger city


because it shows exactly the same content.


So that really every URL that we crawl on your website


and index, we can see, well, this URL and its content


are unique.


And it's important for us to keep all of these URLs indexed.


Or we see clear information that this URL you know


is supposed to be the same as this other one.


You have maybe set up a redirect or you have a rel canonical


set up there.


And we can just focus on those main URLs


and still understand that the city aspect there


is critical for your individual pages.


So that's kind of the approach I would take here.


I'll also pass this on to the team


internally so that they have an example of something


where we're picking it up wrong and we could


be doing a better job there.


But I'd still try to see if there's something


that you can do on your side as well to try


to limit this kind of overlap with multiple URLs.


SPEAKER 2


If you want to stick the URL into the chat,


there's quite a lot of experience with geographic event


sites, as you know.


So we can have a look at it as well.


SPEAKER 1 Of course, yeah.


SPEAKER 2 Rishi, if you want to just paste it in.


SPEAKER 3 I just dropped the URL in the chat.


And one more thing that I have noticed


that there is some pattern that the smaller number of cities,


like in length, like four or five character length cities


are affected the most instead of the larger name cities.


SPEAKER 1 Yeah, that's something that could also


be happening there.


And that, I mean, our systems, when


they try to understand the duplication across URL


patterns, it sometimes picks up weird things there.


So especially, like if you mentioned shorter city names


and longer city names, I could imagine a case


where you have abbreviations as well for the same city.


And then we might learn, well, the shorter version


is the same as this longer version.


Therefore, we can do that more likely with shorter URLs.


But I don't know, maybe some examples,


if you could post them in the chat,


then I can pick those up as well and forward them onto the team.


Or Rob has some tips.


He's been working with event sites since forever.


SPEAKER 3 Sure, I will drop some things.


SPEAKER 1 Cool.


Let me just see what else we have.


So I have a bit more time we can hang around after the recording


stops as well.


Let's see.


One question here is someone has a bunch of XML sitemap files,


and two of them have a status of can't fetch,


sitemap couldn't be read, the files are valid,


and why don't they work?


It's hard to say.


That's something where it would be useful to have specific


example URLs.


And one way you can give us that information


is maybe in the help forum.


So to post some of the sitemap URLs that are working


and some that are not working.


The folks in the help forums can escalate individual issues


as well so that Googlers can take a look


and see if there's something on our side that's


maybe blocking that as well.


I have occasionally seen these kind of cases


where you have a bunch of sitemap files and some of them


work and some of them don't.


Sometimes they're just technical quirks on the server side


that are happening.


Sometimes they're weird things on our side


So it's sometimes useful to look at the specifics.


One thing I have seen is sometimes


just changing the URL of the sitemap file


kind of resets Google's opinion of that URL


and gives us a chance to take a look at the kind of new sitemap


URL.


So that might be kind of a tricky workaround.


Cool.


OK.


Let me take a break here with the recording


and I'll stick around for more as well.


So if any of you want to hang out a little bit longer


or chat off the record, you're welcome to stay.


If you're watching this on YouTube,


thank you for watching.


And if you'd like to join one of these events in the future,


make sure to watch out for the event listing on our YouTube


page or subscribe to the calendar


that we have on the Search Developer site.


All right.


With that, I wish you all a great weekend.


English Google SEO office-hours from August 13, 2021

All right, welcome, everyone, to today's Google Search Central


SEO Office Hours Hangout.


My name is John Mueller.


I'm a search advocate here at Google in Switzerland.


And part of what we do are these Office Hour Hangouts


where people can join in and ask their questions around search.


Bunch of questions were submitted already,


but we also have people here live.


So maybe we can start with the people who are here now


and already have their hands raised.


Let's see, Michael.


I think you're first on the list.


Good morning, John.


Hi.


My question is around Core Web Vitals.


And in Search Console, what concerns me


is I've seen an increase in my URLs


that have been bucketed into the poor category.


But unfortunately, with PageSpeed Insights,


it's impossible for me to validate that or corroborate


that information.


Do you have any information on, I guess,


which tools Google is actually using for the algorithm?


And again, what concerns me is the trend


that this has happened after the release of the latest core


updates.


So I'm curious if you could give me


any insight and guidance into what tools to trust.


OK.


So I think the whole topic of speed


is surprisingly complicated in that you look at it


and it has three measurements.


You're like, oh, I can figure three things out.


But there are just so many things involved.


I'm sorry to interrupt, John.


This is actually a CLS issue.


So this one is not speed, but similar problem.


Yeah, exactly.


And I think maybe first of all, just to be clear,


the whole page experience update is completely unrelated


to the core updates.


So the core updates that you might see happening


or that we announce, they're more about understanding


relevance of the site, so more about the quality


and the content of the site.


And the page experience update is purely


on the core web vitals and the other page experience factors.


So that's maybe one thing to keep in mind,


that these are really separate.


The in-search console, what we show


is essentially field data, which we also call real user


metrics or RUM data, which is essentially


data that's collected from users when they go to your website.


So that's essentially the scale on which we operate there,


I don't know, the most pure measure


that we can get, essentially, because this


is what users actually see.


And PageSpeed Insights and some of the other tools,


they are so-called lab tests, which are essentially


kind of emulations of the field data, where basically they


run similar calculations, and they make assumptions


and say that, on average, users are probably like this,


have this kind of connection, this kind of a screen,


do this kind of thing on their phones,


and we'll try to estimate what those metrics could be.


And because of that, you might see differences


across the data in Search Console and the data


that you see when you run the tests yourself,


because the tests, when you run them yourself,


are essentially estimations.


And in Search Console, we see what users actually saw.


So that's kind of the, I think, the main difference there.


And that applies to speed, CLS, first input delay, all of that.


And I think, especially with regards to CLS,


that's something where sometimes you


have weird things that play in there, which


are kind of hard to figure out, where, for example,


if users in your country see the page completely normally


and users in the primary location of your site,


where most users are, they see maybe a pop-up or something


that comes up in between, which shifts the CLS around.


Then when you test it yourself, you


might think, well, this metric looks OK,


but what users actually see is slightly different.


With regards to CLS, I think that's sometimes


super hard to pinpoint.


What I would recommend doing there


is trying to make some simplified pages


and trying to see if you can see what users might be seeing so


that you can narrow down a little bit which


elements on the page are actually causing the shift.


What you can also do is instrument the pages yourself.


So in web.dev, we have some JavaScript libraries


that you can use where, essentially, you


add the code to your pages, and then it


reports through Google Analytics or some other analytics tool


that you use the metrics that users are actually seeing.


That lets you get data a little bit faster.


Because in Search Console, because of the way


that we aggregate the data, that's


always about 28 days delayed.


So if you make a change now, it takes almost a month for you


to see that in Search Console.


And if you instrument the pages yourself,


you can do things like A-B testing and track that.


You can see, essentially, if you release something today,


you can check tomorrow what users actually saw there.


And you can narrow things down a little bit faster.


And once you've figured out which elements on the page


are causing the CLS shift, then you


can work to improve that on the site.


So that's the direction that I would go there,


using the field data that you see in Search Console


as a way of recognizing, oh, there's an issue,


and then trying to narrow that down iteratively


and using the lab tests to confirm that you're


on the right track there and to try to reproduce


what users are actually seeing.


Thank you very much.


JOHN MUELLER-ROSENZWEIGER-MARTIN LEEUWENER-MARTIN LEEUWENER-MARTIN


Let's see.


Julian.


Hey, John.


How's it going?


Hi.


So I have a two-part question about Google evaluating website


quality.


So in the past Office Hours, you've


talked about if the overall quality of a website


is an issue, specifically if it has tens of thousands of pages


or more, that aggressively de-indexing low-quality pages


is a good idea, which makes sense.


So removing bad pages from the index that


haven't had a chance to meet a certain quality threshold just


yet with good user-generated content,


and I guess all of this in an effort


to make Google reconsider the quality of the website


as a whole.


So we have a website in that camp.


And in the last couple of months,


we've made an effort to do just that.


And in the past Office Hours, you've said it can take months


or even up to half a year for Google


to recalculate the quality and treat a website differently


in search.


So question one would be, do you stand by that time frame?


Can anything affect the time frame?


And is it correlated to the size of a website?


And then a quick follow-up for question two


is, if we're a small team, and that's


with SEO being our primary acquisition channel,


how would you suggest measuring that that was indeed


the problem that we tackled?


So for instance, would you look for gradual improvement,


or would it be flat for many months,


and then you would see things shoot up if we indeed


did do the right thing?


Because I guess it's a scary proposition


to remove thin pages from your index


and then wait six months and then see if you've done enough.


Does that all make sense?


Yeah, I think the timeline is probably about right.


I'm sure there are situations where it


can go a little bit faster.


But on a whole, that's probably something


which takes that, I don't know, half year or longer almost


for things to kind of settle down.


And it's, I think, partially due to kind of us


needing to re-index everything on the site,


understanding the site again, partially also just


because all of the quality signals that we collect,


they just take a long time to kind of be built up.


So that's something where a couple of months, half a year,


maybe even a bit longer, I think that would be kind of the norm.


I'm sure there are situations where it can be faster,


but for the most part, it's probably in that range.


With regards to recognizing when you're kind of on the right


track, I think that's super hard.


Because on the one hand, you have the situation


where you're removing pages.


So you might see kind of like a drop


in traffic from those pages.


I think overall, if you look at the metrics for your site,


the pages that you remove are probably


pages that don't get a ton of page views.


So maybe that's something that wouldn't pull down


your overall traffic from search or not that much.


But it takes a long time for things


to kind of start seeing more and more traffic


and kind of being picked up as being higher quality.


So one thing I would try to do is


try to find some other proxy metrics


that you can use for recognizing the quality of your site.


And that could be something like looking into analytics


and looking at things like, I think


like they call it the engagement rate now


or other metrics where you have kind of time on site,


where you have something where when you look at the metrics


now or when you compare to the metrics


that you had maybe a month ago before you started working


on this, you can tell that actually those were pretty bad.


And with these improvements, we see kind


of this shift in user behavior.


And it's not so much that we would use that user behavior


directly in search, but it's something


that's more like a leading indicator for you


to let you know that you're on the right track.


So that's kind of the direction I would look at it there.


On the one hand, you shouldn't see a big drop from the pages


that you remove.


You should see kind of like this slight subtle shift


in user behavior through the other metrics that you track.


And then over the course of, I don't know, a couple of months,


you probably see like a gradual increase.


And then at some point, you see maybe a bigger jump happening.


AUDIENCE 1 OK.


And thanks.


That really helps.


So would one strategy that you recommend


be could you test a small subset of pages,


take some of the better quality ones,


and maybe do like a campaign around them to sort of get them


shared around a little bit if you believe


that they're good quality?


Would that expedite the reconsideration


of just those pages by any chance?


Or is it kind of more like, no, if you have a website quality


issue as a whole, those might see a suppression too


no matter what?


JOHN MUELLER I think for tracking the metrics yourself,


if you're looking at analytics, that's definitely


a good way to look at it.


Where if you can drive traffic there and say,


it's like instead of, I don't know,


50 people a day coming to these pages,


you have 1,000 people coming to those pages,


you have a little bit clearer metrics.


From an SEO point of view, from a Google search quality


point of view, I don't think taking a small subset


of a site and improving the quality of that


would be enough to make us say, oh, this


is a different part of a site, and this is high quality,


and this is kind of medium quality.


We'll treat it differently.


I think for the most part, we would look at it


or try to look at it as an overhaul thing.


And it might be that the small set that you improve


kind of moved the overall average up a little bit higher,


but that's not something I would kind of wait for.


Because if you're talking about a half a year


and you improve 10% of your pages,


and after half a year you realize,


oh, I need to improve even more, you've


kind of wasted a lot of time.


Yeah.


Yeah, that makes a lot of sense.


OK, cool.


Thank you so much.


Sure.


Tom.


Hey, John.


A couple of months ago on one of these Hangouts,


I was asking some questions about the book structured data


and getting an entry point in the search results.


So we went away, and we made some changes


to how we had implemented that and made it all correct,


because there were some errors.


And you very kindly, the last time we spoke,


offered to give the internal team a nudge


or basically just give someone a heads up


to check the inbox on that Google form.


So basically, it's just a question


to see if that would be possible for you to do


or if I should send you a tweet to give you


just a little nudge next week or what the best process would be.


Probably easiest just to send me a short tweet


and some kind of detail of what it actually was.


So I can double check what they said there.


Yeah.


Amazing.


Thank you very much.


That was it, short and sweet.


Or drop some information here in the chat,


and I can pick that up afterwards.


All right, perfect.


Cool.


Jason.


Cool, thanks, John.


So some background.


Our systems and content delivery network


are designed to let all real users view our content


and filter out some bots while letting others,


like Googlebot, through.


And just for some additional context, earlier this year,


we changed our server monitoring suite,


and we thought we'd carried over all of the reporting needs


we had to ensure that Googlebot could access our content.


Unfortunately, it seems we'd missed some and noticed


in Search Console some 500 series errors cropping up


starting late last month.


The question came up from our technology team


whether this represented real user impact


and why we would look specifically


at Googlebot and not real user metrics


to prove that there's an issue here.


So given that context, I have a few questions.


The first is just to get the technology concern out


of the way, from your perspective,


how does Googlebot view 500 series errors,


and could you give any clarity on established thresholds


at which point Googlebot will crawl source content less


based on those errors?


We don't have any strong thresholds on that,


but essentially what happens with 500 errors


is we'll try to retry them.


And if we continue to see that there are 500 errors,


then we will slow down crawling.


then we will drop those URLs from what they index.


So that's something where if every now and then


individual pages have a 500 error, it's no big deal.


We'll retry them.


They'll remain indexed.


And the next time we retry them, that's fine.


But if a large part of a site consistently has 500 errors,


then we might assume that maybe we're causing the problem.


And we'll slow down crawling of the whole site.


And at some point, we'll say, well,


it looks like these pages are really gone.


We're going to drop them.


So that's essentially the effects


that you would see there.


And if you're talking about a large site


and wondering what percentage of 500 errors is OK, I don't know.


My feeling is if you're seeing something more than 1%,


then that sounds like something is broken


and probably would be something where


we would start to slow down.


But I don't think we have any hard thresholds where


we'd say this many requests and this many errors


means that's not slowing down.


OK, cool.


Thank you.


Also, our server logs show a 200 given


to Googlebot at the same timestamp


as the 504 in Search Console.


Our content delivery network is telling us


that if Googlebot gets a 504 from a CDN,


then they'll automatically try to fetch from the origin.


Could you confirm, deny, or possibly confuse that for me?


I don't think we do anything special with regards to 504.


But I'd have to double check.


So in the Search Developer documentation,


we just put up a page with all of the HTTP status codes


and how we react to that.


I don't think we have anything where


we would say we would defer to the origin other than a CDN


because we don't actually see that difference.


Because from our point of view, we access the domain name.


And if the domain name resolves to the CDN,


then that's what we get.


We're not going to say, well, we will take a different IP


address and then try again.


OK, cool.


That actually will help further that discussion with our CDN.


So thank you.


Final question.


If I manage a website for a T-Shop,


will setting all of my pages to a 418 help me rank better?


I'm a T-POD investor.


Probably not.


I actually don't know what we would do with that.


We'd probably drop them as 400 errors.


So that seems like a bad idea.


Sorry.


I just love the 418 status code.


It's just so fun.


So weird.


Yeah, I don't know how that snuck in there.


I wonder if we can add something new.


Cool.


Daniel.


Yeah, so I've got clients currently going


through a site migration.


And they've changed the architecture quite a bit.


I shifted some things around.


I just wanted to share an example of the current site


on the new site.


So in the chat, I'm sent over the current site.


So if you navigate to that page, you'll


see that on that page, their subcategory pages


are split into tiles.


So you've got all adhesive vinyl, foil sheets, et cetera.


So these are all on-page links.


And then if you look at the current staging site,


they're doing away with those internal links for that page,


say, or across all those top category pages.


And they're inserting them into the main navigation.


So my main concern going through a site migration


is always a tense.


I feel like an intense experience


to make sure that we don't lose any visibility, that the old


sites are really, really talking to the current one.


So my question is around how you handle links and navigation


as opposed to on-page links.


My understanding is that if we've


got, say, a lot of backlinks, natural backlinks that


go into that category page, all that link acquisition,


so all that link equity is getting


split between all the links off the page.


But if we no longer have those links on that specific page


and we've got them into the navigation,


is Google really treating the navigation as the same?


Because I would have thought that all of a sudden,


those individual links now become thousands and thousands


of links into the subcategory.


So my main concern is that we go from one site to the other,


and then we start to see rankings drop,


because that page now longer doesn't necessarily


doesn't have the power that it did, because it's now


not a link off a page, but a link off the navigation.


So I wanted to get a good idea, because I'm


sure they'll absolutely kill me if we do lose any visibility


during this process.


So any information you can give me is guidance.


At the minute, I've advised them to revert back


to the tiled type of way, because it's a collection type


page.


A chop fight probably won't work.


So it'd be good to get your feedback and guidance


on the different structures.


Yeah.


I think they can both work.


So I wouldn't necessarily say that one is good


and the other is bad.


It's kind of tricky for me to say


what the actual effect will be, because it's


kind of like a new site structure.


What I would watch out for is to try to avoid changing the URLs.


So with regards to the URLs that you had there,


one is kind of like a page slash adhesives,


and the other one is collection slash all adhesives.


So you're changing page URLs there.


That's something that always kind of means


we have to restructure what we have indexed for your site


and kind of understand what the context of the new pages is.


You have to make sure that the redirect's in place


or what have you there.


So that's kind of the one thing there


with regards to moving those links from a collection


page to the navigation, where probably that navigation is


shared across the whole site.


I think both of those approaches could work.


What probably would happen here is the category pages


would lose a little bit of visibility


in favor of the kind of lower level pages


that you're linking to directly from the navigation.


Because it's


SPEAKER 2


Oh, so it would flip then.


Because I would have thought it was the other way around.


OK, interesting.


SPEAKER 1 Because you're kind of linking to the lower level


pages from across the site rather than just


linking to the categories from across the site


in that chain, right?


Yeah, they both will be in there.


Say if we call it the main category and subcategory pages


should be in that navigation.


Well, I would have thought that all the main category


pages will be in the navigation and then the subcategory.


But they have, say, over 3,000 collection pages.


So they all won't fit.


So these other pages will be on like an A to Z page as well.


And that's another thing that concerns me.


Because now we're kind of taking these URLs


and put it into the corner of the website,


all linking up one page.


So I think that that might be an issue.


So I think if there are really important pages,


they shouldn't exist there.


But they have to exist in the main navigation.


Would you say that that would be a fair assumption?


Yeah, I think kind of like the A to Z pages,


that's something where you're basically telling us


these are a lot less relevant than they used to be,


which might be fine.


Where if you can say, I really want


to concentrate on these other things,


then you kind of have to balance that and say,


I really want this.


So I have to let go of this.


And that might be perfectly fine.


And similarly, with kind of that switch


from focusing on the category pages for search


to kind of the next level or the lower level pages,


again, that's something that is almost


like a strategic decision, where if you


see that the competition for category level queries


in search is such that you will still be visible there.


Because you're really strong in that area,


then maybe it's OK to say, well, I'll


make sure that the lower level pages, the long tail,


is a lot stronger.


And you're kind of balancing almost


like short tail versus long tail.


And that's almost like a strategic decision, where


there's no right or wrong.


But it's more like, well, in your situation,


in your market, with your site, what makes more sense?


Do we need to concentrate more?


Can we already kind of spread out a little bit broader?


And from my point of view, it's not


a matter of technically correct, because both of these


could be technically correct.


It's more like, what do you want to focus on?


Yeah, so we would just say that if these, we'll


call them subcategory pages.


But if they went into the navigation,


that Google sees them as more important,


because we've put them in the main navigation.


So in theory, fingers crossed, they wouldn't lose visibility.


Maybe they might gain or at least


maintain what they've got.


I know it's difficult to tell.


I mean, so I don't know the whole site.


So it's hard for me to say offhand just


by looking at these two pages.


But I'm assuming that the main category pages were already


linked across the site on the old version.


And with the new one, the main category pages


will be linked, as well as subcategories and maybe even


products, something along that line.


Yeah.


So from my point of view, what would happen there


is the main category pages, because they're


a part of this bigger set of pages


that are now linked across the site,


they would lose a little bit relative strength.


So some of that strength would be distributed


to the other pages.


So it's not that the main pages would


remain the same because they're still linked


from the same pages.


It's just from all of those same pages,


you now link to lots of other pages.


So you're kind of spreading the value a little bit broader.


Throughout the navigation is what you're saying.


Yeah, that's interesting.


OK.


And is it considered thousands?


So I have to take up so much time.


But is it considered thousands?


Because again, it's every single page is a new URL.


And we've got 100,000 pages.


So is that 100,000 internal links?


Or are you just looking at them?


You know it's a navigation, so you're just looking at it.


I don't know.


I think we would still count them as internal links.


But I wouldn't worry so much about the number.


So one of the things that I like to do with my test sites


is to do something like running Screaming Frog over them


and generating that tree graph or something.


They have something like that, where you have kind of like how


the internal links are spread out.


And based on that, you sometimes see, well,


it's all concentrated in the middle,


or it's like it has clear category structures there.


And that's essentially a shift that you're doing there.


You're kind of taking things that


were linked further out and bringing them closer in,


which means you're kind of like spreading


that value in that internal circle a little bit broader.


That's very interesting.


Thank you very much.


I appreciate that.


Thank you.


Sure.


That gives me something to go back with.


OK.


Cool.


Let me go through some of the submitted questions.


I'll get to the rest of you as well.


Don't worry.


And we have a little bit more time afterwards


to kind of go through the remaining stuff as well.


But just to make sure that the submitted questions don't get


ignored, 20% of my pages are not getting an IEX.


It says they're discovered but not crawled.


Does this have anything to do with the fact


that it's not crawled because of potential overload


of my server, or does it have to do


with the quality of the page?


Probably a little bit of both.


So usually, if we're talking about a smaller site,


then it's mostly not a case that we're


limited by the crawling capacity, which


is kind of the crawl budget side of things.


If we're talking about a site that has millions of pages,


then that's something where I would


consider looking at the crawl budget side of things.


But smaller sites, probably less so.


With regards to the quality, when it comes to understanding


the quality of a website, that is something


that we take into account quite strongly with regards


to crawling and indexing of the rest of the website.


But that's not something that's necessarily related


to the individual URL.


So if you have five pages that are not indexed at the moment,


it's not that those five pages are the ones


we would consider low quality.


It's more that, well, overall, we consider this website maybe


to be a little bit lower quality,


and therefore we won't go off and index everything


on this site.


Because if we don't have that page indexed,


then we're not really going to know if that's


high quality or low quality.


So that's kind of the direction I would head there,


is if you have a smaller site and you're


seeing that a significant part of your pages


are not being indexed, then I would take a step back


and try to reconsider the overall quality of the website


and not focus so much on technical issues for those


pages.


Because I think for the most part,


sites nowadays are technically kind of reasonable.


If you're using a common CMS, then it's


really hard to do something really wrong.


And it's often more a matter of the overall quality.


The other thing to keep in mind with regards to indexing


is it's completely normal that we don't index everything off


of a website.


So if you look at any larger website or any even


mid-sized or smaller website, you'll


see fluctuations in indexing.


It'll go up and down.


And it's never going to be the case


that we index 100% of everything that's on a website.


So if you have 100 pages and, I don't know, 80 of them


are being indexed, then I wouldn't


see that as being a problem that you need to fix.


That's sometimes just how it is for the moment.


And over time, when you get to 200 pages on your website


and we index 180 of them, then that percentage


gets a little bit smaller.


But it's always going to be the case


that we don't index 100% of everything that we know about.


Let's see.


Question about home page versus landing page.


When it comes to SEO rankings, if I have a small brochure


website, should I be focused on keywords


I want to rank for on the home page?


For example, would I want to rank for a roofing company


and location?


I could set up the home page as construction company


with different surfaces.


But I also want to rank for roofing.


Is there best practice to follow for these situations?


So I think this is kind of similar to the question


before where it kind of boils down to kind of what you want


to focus on.


And especially if you're a new website,


then picking some specific area to focus on


and kind of building up your strength there


makes a lot of sense.


And sometimes it makes sense to pick an area where maybe there


is not a lot of competition, maybe where


you're one of the few companies in that area that


offers that service, just so that you can kind of slowly


build up your reputation and the things


that you do on your website for those kind of queries.


And then from there, you can kind of expand.


So that would be generally my recommendation here


when it comes to smaller businesses.


Don't just create the website and put everything on there


all at the same time in the hope that maybe one of these


will end up ranking.


Because what will usually happen is we'll look at all of this


and say, well, there's a bit of everything here,


but there's no really strong things here


that we should focus on for ranking.


The other thing to maybe also keep in mind


if you're a really small business


is if you're locally active, then sometimes


the traditional SEO things are not the most critical things


that you need to do in the sense that maybe a lot of the traffic


that goes to your website is more from local searches.


And when it comes to local searches,


the Google My Business side of things


is slightly different there.


And maybe that's worthwhile to kind of say, well,


I'll put everything on my website


just as a place to kind of collect things,


but I'm not using that in a sense of actively doing SEO


and trying to rank for, I don't know, roofing services


worldwide because I'm just focusing on my small area


where I'm actually active.


And in cases like that, sometimes just focusing


on the Google My Business side of things


might make a lot of sense.


Let's see, what would be a better strategy,


having a 404 page for pages that don't exist,


or redirecting any nonexistent page to the home page?


Our website is recovering from a malware attack


in which tens of thousands of pages that were made


were redirecting to some shady website.


We're fixing it, but now we probably


have 150,000 pages with 404 errors in Search Console.


Yeah, I don't know.


Recovering from malware is always super annoying


and a lot of work.


So hopefully it goes well.


And hopefully you can lock things down


to avoid this kind of situation in the future.


In practice, in a situation like that,


I don't think there's any big difference with regards


to doing a 404 page or redirecting to the home page.


If you redirect to the home page,


we'll probably see that as kind of a 404 page for the page


that was redirecting there.


And we will treat that similarly.


So we would call this a soft 404 page.


I think in practice, both of these will be similar in terms


of the time that it takes to settle down.


What I would try to do in a case like this


is focus on the most important pages of your website


and make sure that all of those work really well


and make sure that those are updated in search.


And all of the rest, I don't know, probably


404 is the easiest approach here,


where if you just remove those pages,


it returns 404 by default. And I would just


give that maybe a half a year or even longer to kind of settle


down and drop out of the index.


If you actively keep looking for those pages,


we will continue to show them to you because we think, well,


you're looking for this URL, so we will show it to you.


But for users, probably they see your primary pages


in the search results, and they would not even notice that.


So maybe there was also some hacked content on your website.


So it's easy to kind of get dragged into a situation


where you're focusing on an issue that actually users


don't see in practice.


And that's something to kind of watch out for, I think, here.


So I think 404 or redirecting both of those are OK.


Some people also use sitemap files


to make sure that we crawl them a little bit faster.


I don't think that really matters


when you're talking about 150,000 pages that


used to have malware on them.


I would just let them drop out naturally.


Let's see.


Long question.


And this is from Mike.


And I see you're up next, Mike.


Maybe you can just run us through it briefly.


MIKE POOLE-CHAPMANNITSKY-SUNDERSEN-MARTIN-MURPHY


Yeah, sorry.


I tried to put a short thing down, but I just couldn't.


So probably best if I just probably


will do quicker like this.


So I'm just trying to decide whether it's worthwhile.


Create unique URLs for each country


that the site is relevant for.


The site's relevant for, say, like five


English-speaking countries.


And at the moment, there's just one URL.


And since the majority of the traffic is from the US,


that's fine.


And if the user's not from the US,


then there's a pop-up which says, click here,


and it will load then their country's content


and set a cookie so that that's not a problem in the future.


And now, as I came across, you were talking around


in a webmaster's from 2016 where you were saying,


like, if you've got lots of very, very similar content,


maybe you're talking about identical content.


It may not be the best idea to create


unique URLs for different countries


and to use HREFLang and geotargeting as well.


I'm just doing lots of research on this.


I'm still just completely, I don't


know whether it's going to be counterproductive


to do this in terms of, oh, it's going to reach more.


I know with HREFLang, what I seem to understand


is it basically just swaps out so there'll


be no ranking boost.


And so the only benefit there would


be an improved in bounce rate or a lower in bounce rate, which


could improve ranking, which may be worth it,


may not, with regards to the tech side of things.


But with regards to the geotargeting,


I guess the first question is, with regards to,


if I was to use subfolders, by geotargeting,


does that just mean setting up the country


in international targeting in the search console


to each country?


And given that there'll be a lot of duplicate content


across each country, is that a good thing?


Would you say that this is a sensible thing, given


there's a considerable amount of duplicate content?


Or would you say, no, don't do that, just stick with the,


is it going to be counterproductive to do that?


It depends, to be kind of cliche, I guess.


What kind of site is it?


Is it like an e-commerce site where


you have products that are for sale,


or is it more like a handful of services?


It's got reviews for TV shows on streaming services.


So it would be, so someone will be searching for good horror


movies on Netflix or something like that, which could be,


most of the time, I mean, sometimes they'll


add in their country in there.


And a lot of the pages, they'll have the review,


but they'll be in the list.


But the list will change, because the availability


will be different across each thing.


So that means it definitely needs to be,


that means it would be very useless to the user to have,


for them to be like, oh, this is available.


But there will be significant amount of duplicate content,


because a lot of the shows are the same across each,


particularly the English-speaking countries.


Now, so maybe taking the easiest one first,


when it comes to geotargeting, that


would help us with ranking and boost the ranking


when we think that a user is looking for something local.


So for example, if someone is searching


for washing machine manuals, then we


could just show washing machine manuals globally.


They're not looking for something local.


But if they're looking for washing machine repair,


then obviously, they're probably looking


for something more local.


And in a case like that, we would


try to show the more local content.


When it comes to reviews of TV shows or movies,


I suspect there's not that much of a local intent


when it comes to the queries there.


So probably, you would not see a significant change


with regards to ranking, with regards to geotargeting there.


So that's kind of my assumption there.


I don't know the market that much.


If people are actually looking for something local


when they're looking for, I don't know,


horror movie reviews or something like that,


it might be.


But offhand, I would assume it's more of a global thing.


So geotargeting probably doesn't make that much sense.


I mean, if they're looking for horror movies on Netflix,


they'll definitely want the list on the Netflix one.


But yeah, it would be slight changes across it.


So I can see that.


Yeah.


If you wanted to do geotargeting,


you could use folders, and you just


verify those folders separately in Search Console,


and then set the geotargeting in Search Console there.


But could that be counterproductive, though,


because it's duplicate a lot of duplicate content?


Could that just be like, you know?


I think it could, because you're kind of diluting


the value of your content across all of those different country


versions.


But it's also something where I think, like in your situation,


probably geotargeting wouldn't make much sense then.


OK, good.


OK, thanks.


When it comes to hreflang, I could


see that being something that helps there.


But you also have the situation there


that you essentially create separate URLs.


And they have to be indexed separately,


and they have to be ranked separately.


And when we show them in search, we can swap them out.


But they have to be kind of like standing on their own.


And if you're taking something like a horror movie review


page and just creating different country versions of that,


I could imagine that that kind of reduces


the strength of your pages relative to the rest


of the competition that is out there.


So if you're like by far number one for this kind of a query,


then going off and doing hreflang


for different country versions, like go for it.


But if you're really in a strong competition area


where maybe you're ranking number five or number 10


in the search results page, then adding hreflang there


just makes it a lot harder for you to grab more foothold


there.


So that's kind of the direction I would head there.


The nice thing about hreflang, though,


is you can do it on a per-page basis.


So you can try this out for individual areas.


You can try it out for things like maybe detail pages


on your site, maybe the home page,


maybe specific categories, and just kind of see


what happens there.


Is it like really something where


you see a change in user behavior


when they go to those pages, if they're


coming from the right country?


Or is it something where you don't


see a significant change?


If you don't see a significant change in how users react,


then probably it's not worthwhile to go off


and implement that.


Yeah.


That's cool.


That's really helpful.


Thanks.


Can I just ask something slightly different


that hopefully will be short, a different section?


If there's a problem with the server


and there's a very long response time, which Google


has really lowered the crawl rate, which is also


annoyingly half at the same time there's


been lots of thin and duplicate content pages been indexed.


To get the crawl rate, is it better to leave the setting


in, to manually set the crawl rate to be the maximum,


if that is above what was previously being crawled?


Or is it better to let Google just do some tests itself


using its custom crawl rate?


I think both would work.


I suspect if you set it to let Google figure it out,


it'll pick up a little bit faster


because it'll be able to go past that setting


that you applied there.


But both of those should work.


And it's also something where as soon as we can recognize


that your server is actually a lot faster,


then usually within a couple of days, maybe a week or so,


we'll actually pick up crawling anyway.


Cool.


Is it all right to have just a final question?


Is it all right to have lots of thin and duplicate content,


no indexed, or is that going to harm the?


That's perfectly fine.


Yeah.


Good.


Thanks.


Thank you very much.


Sure.


Maybe we'll just go and move on to live questions again.


Contact mentor.


Hello, John.


That is Anirudh, actually.


So nice meeting.


So my question is, so actually I have a site


which is around like three months old.


So the thing is when the name of my site


is searched, like contact mentor, so that time


there is another site which is kind of writing


a review on my site.


So the review is kind of like very spammy.


Like it is kind of written like my website


is from suspicious domain provider, domain registrar,


and show low ratings, and also kind of without any evidence,


they're showing kind of a wrong thing.


And also like in terms of domain registrar, I'm using GoDaddy.


So there is nothing suspicious from that side.


And if I check from other like popular sites


which show reviews, so my site is showing fine.


So the thing is, so I had emailed their admin


that the information like why is it that way.


But I get a response, auto reply response,


to kind of pay money to get the site removed.


So is there a way like Google can help me in this case?


And also there is press pilot reviews on that site.


So that also they have written the same things.


Many people have written the same thing


that I am telling you now.


I don't think we would do anything special in a case like that.


It's more something where over time you kind of build up


the value of your own website.


And more and more, we'll just focus on your website


in the search results.


So that's not something where we would go off and say,


oh, the information for this one page


is not good on this one site.


We will remove it from search, or we will rank it lower


We wouldn't do that kind of manual intervention.


So that's something where it's more essentially the best


approach here is kind of to just to move forward and keep


working on your own website and ignoring the other ones there,


even if they are sometimes ranking for your name as well.


And it's, in a sense, similar to what is considered reputation


management, where essentially someone is creating


some content on your name, in this case on your site's name


that you don't like.


And if you look up reputation management,


there are a handful of strategies


that you can do there, which include things like making sure


you have some social profiles with your name


or making sure that other people actually


talk about your site in the way that you think is correct


and making sure that the good information is out there


as well.


And that's something that just takes time.


It's not something where you can focus on the bad content


and try to get that removed.


You essentially have to move forward in a positive way


and say, oh, I see there are people trying to pull me back,


but I will continue my path in moving forward


in a positive way.


AUDIENCE 1 So the thing is, my site is kind of a mentorship


site, so even one person kind of sees


that it is a significant loss.


And also, I have no issue with bad information,


but if it is false, like completely false,


and I can kind of show that, then would that help anyway?


JOHN MUELLER We wouldn't see that as spam, though.


It's like a lot of information on the web is false.


And there are some situations where


it's really critical for us to make sure


that the information we show is correct, which


is areas around finances or medical information.


But if it's information about another business,


about another website, then people


have different opinions, and sometimes they


post something incorrect.


They mix things up.


That's not something where we would go in and say,


we need to fix that manually.


And even in cases where it's like medical sites


or where it's financial sites, it's


not that we go in there and then manually say,


oh, this is bad medical information.


We will hide it in the search results.


But rather, we'll try to improve our algorithms


to recognize the correct information


and to highlight that a little bit better.


And that's not kind of like based on individual sites.


It's based on a broader and better understanding


of the web as a whole.


So even there, we would not go in and manually hide sites


just because they have some information that we don't like.


AUDIENCE MEMBER 1 Yeah, I get it.


Thank you for answering.


No.


It's hard work sometimes.


And sometimes people are jealous of the successes that you have.


So it's like, I don't know.


It's annoying, but you kind of have


to move forward in a positive way.


Let's see, Nathan.


AUDIENCE MEMBER 2 Hi, John.


So I've been checking a few pages in Search Console, which


aren't indexed.


But they are indexable.


And the canonicalization seems to be all in order.


And I'm trying to see how I can figure out why


these pages aren't indexed.


Is there a way that we can see that either in Search Console


or via any other way?


OK.


Sometimes, not what I like, but OK.


Next question.


No, I mean, it's something where even internally trying


to figure out why a page is not indexed,


that's really complicated.


And we don't have anything along that line externally available


in Search Console where you can say,


but why don't you like this page, Google?


Our loss.


Thanks.


Oh, well.


Let's see, Ritu.


I can barely hear you.


Very quiet.


A little bit, yes.


Better.


Yeah.


So I have a few questions.


One question is related to sitemap.


I see some pages are not present in sitemap,


but still appearing in Search Console as a 404 and not


calling.


What can we do for these type of issues?


You suggest me and I?


You can just leave them.


So we use sitemaps to help us additionally crawl,


but it doesn't replace crawling.


So if a page is not in a sitemap,


it doesn't mean we won't crawl it.


So if it's a 404 and you say the page doesn't exist,


then that's fine.


It doesn't cause any problems.


OK, we can leave it.


Yeah.


OK.


So one more question.


If any parent and parent category is de-indexed,


it will impact a child's subcategory pages


for any parent category experience.


So if the main category page is not in the index?


Yeah, parent category is de-indexed.


It can affect the subcategories there,


because usually the subcategories


would be linked from the main category page.


It depends a little bit on how you set up your website,


but usually they're linked from there.


And if that category page is not in our index,


then maybe we don't have links to the subcategory pages,


and maybe we won't be able to crawl them.


If you still link to the subcategory pages


like from a menu on the website, then it


doesn't matter so much, because then we can still find


those subcategory pages.


OK.


One more question.


It is related to like, I have awesome blogs.


And the periods are declining, to a large extent declining.


And I have implemented so many techniques to improve them,


but I'm not getting any results.


Can you suggest me like any good techniques


which I can implement to improve these periods,


declining periods in the CFP, or blog periods,


blog optimization?


Apart from social media promotion,


apart from like content optimization,


if I can go further, any techniques?


To kind of improve ranking in general,


I guess, for those keywords.


Yeah.


I don't think there is one simple approach that you


can do to kind of improve the-


Yeah, there are various techniques,


but I can suggest any good techniques which are helpful


for me to manage the results.


So I think what I would do in a case like this


is maybe take a step back and go through the SEO starter


guides that we have.


So we have an SEO starter guide.


There are various SEO tools that also have kind of SEO starter


guides, which I think are really good.


And kind of work your way through those.


And see if there are any aspects that you're missing.


A lot of those things are things you're going to say,


oh, this is obvious.


You already know all of this.


But there are going to be individual things in there


where you say, oh, this is actually a good idea.


And this is not something that we have done in the past.


And that, I think, is kind of a way of going from,