used to nest structured data within HTML content.


JSON-LD Structured Data: Where to Insert in a Page? #AskGoogleWebmasters

Like RDFa, it also uses HTML tag attributes for this.


JSON-LD, in comparison, is a JavaScript notation


embedded in a script tag in the page head or in the body.


The markup is not interleaved with user visible text, which


makes nested data items easier to express.


So as mentioned, the JSON-LD structured data


can be either in the head or the body of the page.


So it sounds like we can answer your question with a solid yes.


JSON-LD can also be injected into your pages


using JavaScript if that suits your pages better.


Thanks for watching.


And remember, if you have any questions,


post them on Twitter using the hashtag AskGoogleWebmasters.


See you next time.


["1- motivationsHealthierMSM synth-


Should I worry about crawl budget?

Hello, and welcome to another episode of Search Off the Record, a podcast coming to you from


Should I worry about crawl budget?

the Google search team, discussing all things search and having some fun along the way.


Should I worry about crawl budget?

My name is John.


Should I worry about crawl budget?

No, no, no way.


Should I worry about crawl budget?

No, my name is not John.


Should I worry about crawl budget?

My name is Martin.


Should I worry about crawl budget?

And I'm joined today by Lizzie and Gary from the search relations team, of which I'm also


Should I worry about crawl budget?

part of.


Should I worry about crawl budget?

Hello.


Should I worry about crawl budget?

Hi.


Should I worry about crawl budget?

Oh my god.


Should I worry about crawl budget?

Okay.


Should I worry about crawl budget?

All right.


Should I worry about crawl budget?

I would like to talk to you about something that does end up being a question that I've


Should I worry about crawl budget?

been asked everywhere pretty much I go.


Should I worry about crawl budget?

And I'm always wondering, we do have docs on this, but I do see that people are very


Should I worry about crawl budget?

confused about it.


Should I worry about crawl budget?

And that's crawl budget.


Should I worry about crawl budget?

What's that?


Should I worry about crawl budget?

Yeah, that's a really good question.


Should I worry about crawl budget?

Apparently, obviously, everyone wants everything to be indexed as quickly as possible, be it


Should I worry about crawl budget?

a new website that just come online, or be like websites that have plenty of pages, and


Should I worry about crawl budget?

they want to like frequently change those.


Should I worry about crawl budget?

And they are worried about things not being crawled as quickly.


Should I worry about crawl budget?

And I usually describe it as a challenge with a balance between not overwhelming the website


Should I worry about crawl budget?

and also spending our resources where it matters.


Should I worry about crawl budget?

So, we have docs on it.


Should I worry about crawl budget?

But why is seems everyone seems very nervous about this.


Should I worry about crawl budget?

Why is that?


Should I worry about crawl budget?

Why is everyone so concerned about crawl budget?


Should I worry about crawl budget?

I think it's partly a fear of something happening that they can't control, that people can't


Should I worry about crawl budget?

control.


Should I worry about crawl budget?

And the other thing is just misinformation.


Should I worry about crawl budget?

Because there were, I don't know if there still are because, you know, I haven't been


Should I worry about crawl budget?

on Twitter for four hours now.


Should I worry about crawl budget?

And there were some blogs back in the days where people were talking about crawl budget,


Should I worry about crawl budget?

and it's so important.


Should I worry about crawl budget?

And then people were finding that and then they were getting confused about like, okay,


Should I worry about crawl budget?

so do I have to worry about crawl budget or not?


Should I worry about crawl budget?

And of course, the default state is usually, yes, I want to worry about it because I'm


Should I worry about crawl budget?

human and I want to stress myself about everything and my grandmother.


Should I worry about crawl budget?

Wait, are you me?


Should I worry about crawl budget?

I'm not.


Should I worry about crawl budget?

I'm just a sentient blob.


Should I worry about crawl budget?

A blob.


Should I worry about crawl budget?

Okay.


Should I worry about crawl budget?

So, a blob?


Should I worry about crawl budget?

That's less interesting.


Should I worry about crawl budget?

But let's say you were an interesting blog.


Should I worry about crawl budget?

Do you need to worry about crawl budget?


Should I worry about crawl budget?

I think most people don't have to worry about it.


Should I worry about crawl budget?

And when I say most, it's probably over 90% of sites on the internet don't have to worry


Should I worry about crawl budget?

about it.


Should I worry about crawl budget?

But it doesn't mean that people will not worry about it, right?


Should I worry about crawl budget?

Because, again, people.


Should I worry about crawl budget?

Do you think it has to do with the fact that it's a number and people get stressed about


Should I worry about crawl budget?

having the right number?


Should I worry about crawl budget?

Well, I don't think I've ever seen it quantified though.


Should I worry about crawl budget?

I mean, in Search Console, it shows you the number.


Should I worry about crawl budget?

It's like, is this number good or bad?


Should I worry about crawl budget?

Like, it's the crawl range.


Should I worry about crawl budget?

I haven't played with Search Console and whatever is in there for 12 years now.


Should I worry about crawl budget?

So I wouldn't know what is there.


Should I worry about crawl budget?

But externally, at least, it's not quantified.


Should I worry about crawl budget?

It's just you have to worry about crawl budget because...


Should I worry about crawl budget?

Right.


Should I worry about crawl budget?

So maybe that's part of the confusion is, what is a good crawl budget?


Should I worry about crawl budget?

Worrying about it.


Should I worry about crawl budget?

How much is it?


Should I worry about crawl budget?

Is it good or bad?


Should I worry about crawl budget?

How do you know if the budget is enough for your site?


Should I worry about crawl budget?

Or it's sort of a hand wave, hand wave?


Should I worry about crawl budget?

I mean, yes.


Should I worry about crawl budget?

But then we also try to define it for people a bit.


Should I worry about crawl budget?

And we put up a blog post in 2013, 2014, something about what crawl budget means for Googlebot


Should I worry about crawl budget?

or something like that.


Should I worry about crawl budget?

I don't remember what was the title.


Should I worry about crawl budget?

I'm going to link it in the show notes.


Should I worry about crawl budget?

It's 2017.


Should I worry about crawl budget?

Oh, it is?


Should I worry about crawl budget?

Yes.


Should I worry about crawl budget?

Okay.


Should I worry about crawl budget?

And that is the title.


Should I worry about crawl budget?

Like, Fact Check You.


Should I worry about crawl budget?

Live.


Should I worry about crawl budget?

That's great.


Should I worry about crawl budget?

Now I am nervous.


Should I worry about crawl budget?

But we are off the record.


Should I worry about crawl budget?

No worries.


Should I worry about crawl budget?

Oh, it is off the record.


Should I worry about crawl budget?

But continue talking into your microphone because it's off the record.


Should I worry about crawl budget?

This is all very confusing.


Should I worry about crawl budget?

So is crawl budget apparently.


Should I worry about crawl budget?

We put up a blog post which tried to demystify a little bit.


Should I worry about crawl budget?

It was also partly demystifying it for us because for the longest time we were saying


Should I worry about crawl budget?

that we don't have the concept of crawl budget.


Should I worry about crawl budget?

And it was true.


Should I worry about crawl budget?

Like we didn't have something that could mean crawl budget on its own.


Should I worry about crawl budget?

The same way we don't have a number for EAT, for example.


Should I worry about crawl budget?

And then because people were talking about it, we tried to come up with something like


Should I worry about crawl budget?

at least somehow define it.


Should I worry about crawl budget?

And then we worked with two, three, four teams, I don't remember, where we tried to come up


Should I worry about crawl budget?

with at least a few internal metrics that could map together into something that people


Should I worry about crawl budget?

externally define as crawl budget.


Should I worry about crawl budget?

And that's how we came up with that blog post.


Should I worry about crawl budget?

And then we defined it as number of URLs Google bot can and is willing or is instructed to


Should I worry about crawl budget?

crawl.


Should I worry about crawl budget?

For a given site.


Should I worry about crawl budget?

For a given site, yes.


Should I worry about crawl budget?

And for us, that's roughly what crawl budget means.


Should I worry about crawl budget?

Because if you think about it, we don't want to harm websites because Google bot has enough


Should I worry about crawl budget?

crawl capacity to bring down sites.


Should I worry about crawl budget?

And like my spammyguy.com keeps being harassed by Google bot every now and then.


Should I worry about crawl budget?

And then we don't want that because people don't like it for some reason.


Should I worry about crawl budget?

And then Google bot is instructed to crawl from a site.


Should I worry about crawl budget?

That's the part where indexing is telling Google bot that we need a new version of this


Should I worry about crawl budget?

URL or we saw this URL and we want to see what's underneath.


Should I worry about crawl budget?

Okay.


Should I worry about crawl budget?

All right.


Should I worry about crawl budget?

But crawl budget itself does not say anything about the quality or something, right?


Should I worry about crawl budget?

So if I have a website and I have very few pages, my crawl budget is likely not going


Should I worry about crawl budget?

to be huge unless the pages change very frequently.


Should I worry about crawl budget?

Right.


Should I worry about crawl budget?

So I think that's another misconception that people think like, oh, but we uploaded this


Should I worry about crawl budget?

new blog post and it has been two days and we still haven't seen it crawled.


Should I worry about crawl budget?

Well, yeah, it just takes a while until we get around to it.


Should I worry about crawl budget?

But that's because the internet is large.


Should I worry about crawl budget?

And pretty much infinitely large.


Should I worry about crawl budget?

And we don't have enough storage because that's another thing that some people don't realize


Should I worry about crawl budget?

that when you index stuff, you have to store it somewhere.


Should I worry about crawl budget?

And that somewhere is either a hard disks or memory modules or it's a SSDs.


Should I worry about crawl budget?

It costs money.


Should I worry about crawl budget?

It doesn't make sense to invest infinite money in those storage mechanisms because most of


Should I worry about crawl budget?

the stuff that you can find on the internet will not be ever seen.


Should I worry about crawl budget?

Yeah.


Should I worry about crawl budget?

Like the stuff that search engines generally index, not just Google, but most search engines


Should I worry about crawl budget?

is stuff that people will likely look for.


Should I worry about crawl budget?

All right.


Should I worry about crawl budget?

Got it.


Should I worry about crawl budget?

Okay.


Should I worry about crawl budget?

That makes perfect sense to me.


Should I worry about crawl budget?

And I think another thing that contributes to the wariness that people offer or have


Should I worry about crawl budget?

towards crawl budget is that there are things that are not as easy to understand about it.


Should I worry about crawl budget?

For instance, if I build a website that requests a bunch of resources in the background, that


Should I worry about crawl budget?

counts into my crawl budget, no?


Should I worry about crawl budget?

Right.


Should I worry about crawl budget?

So I think then we have a lot of caching usually if it's get requests.


Should I worry about crawl budget?

But then post requests, you can't cache.


Should I worry about crawl budget?

So that would consume more.


Should I worry about crawl budget?

If I have like 10 pages and all of them make a post request to some API, for instance,


Should I worry about crawl budget?

that would count into my crawl budget each time.


Should I worry about crawl budget?

Because it can't be cached.


Should I worry about crawl budget?

So these are like technical considerations where people are like, oh, we built this thing


Should I worry about crawl budget?

with this API and it makes post requests.


Should I worry about crawl budget?

And now we have a crawl budget issue.


Should I worry about crawl budget?

And I don't think-


Should I worry about crawl budget?

How do they know that it's a crawl budget issue?


Should I worry about crawl budget?

At what point is it that they have so many resources or so many URLs that it's become


Should I worry about crawl budget?

a problem?


Should I worry about crawl budget?

Or how have they pinpointed the crawl budget as an issue at all?


Should I worry about crawl budget?

That's a really, really interesting question.


Should I worry about crawl budget?

Because pretty much all the time I've been asked this, people were very, very worried


Should I worry about crawl budget?

like, oh, we have this issue.


Should I worry about crawl budget?

We are relaunching our website with this architecture and it's going to have crawl budget issues.


Should I worry about crawl budget?

And I look at it and I'm like-


Should I worry about crawl budget?

Going to have.


Should I worry about crawl budget?

Yeah.


Should I worry about crawl budget?

Doesn't even have.


Should I worry about crawl budget?

They assume it's a crawl budget problem when they are very far from it, especially if it's


Should I worry about crawl budget?

small websites with like 20 pages.


Should I worry about crawl budget?

I'm like, no, very unlikely to be an issue.


Should I worry about crawl budget?

But people are worried about it.


Should I worry about crawl budget?

And I'm not exactly sure where it comes from.


Should I worry about crawl budget?

I think it comes from the fact that a few large scale websites do have articles and


Should I worry about crawl budget?

blog posts where they talk about crawl budget being a thing.


Should I worry about crawl budget?

It is being discussed in SEO training courses as far as I've seen.


Should I worry about crawl budget?

It's being discussed at conferences.


Should I worry about crawl budget?

But it's a problem that is rare to be had.


Should I worry about crawl budget?

It's not a thing that every website suffers and yet people are very nervous about it.


Should I worry about crawl budget?

I think it's also conflated with other things like indexing not willing to schedule crawls.


Should I worry about crawl budget?

Discovered but not indexed.


Should I worry about crawl budget?

Oh, that's a search console thing, right?


Should I worry about crawl budget?

Because like we said, we don't have infinite space.


Should I worry about crawl budget?

So we kind of want to index stuff that we think that, well, not we, but our algorithms


Should I worry about crawl budget?

determine that might be searched for at some point.


Should I worry about crawl budget?

And if we don't have signals, for example, yet about a certain site or a certain URL


Should I worry about crawl budget?

or whatever, then how would we know that we need to crawl that for indexing?


Should I worry about crawl budget?

And some things you can infer from like, for example, if you launch a new blog on your


Should I worry about crawl budget?

main site, for example, and you have a new slash blog subdirectory, for example, then


Should I worry about crawl budget?

we can sort of infer based on the whole site, whether we want to crawl a lot from that slash


Should I worry about crawl budget?

blog or not.


Should I worry about crawl budget?

But the blog is a new type of content that might be updated more frequently.


Should I worry about crawl budget?

So how can we tell if that is like, it's just new and we're not sure if it's going to be


Should I worry about crawl budget?

newsy, like how frequent it's still to be determined.


Should I worry about crawl budget?

But we need a starter signal.


Should I worry about crawl budget?

And the starter signal is?


Should I worry about crawl budget?

Infer from the main site.


Should I worry about crawl budget?

Oh, okay.


Should I worry about crawl budget?

All right.


Should I worry about crawl budget?

So the main site is, I don't know, like it's yearly.


Should I worry about crawl budget?

But you both are thinking small because you're saying only yearly or whatever, like update


Should I worry about crawl budget?

frequency, but it's not just update frequency.


Should I worry about crawl budget?

It's also the quality that affect the main site has.


Should I worry about crawl budget?

So for example, if we see that a certain pattern is very popular on the internet, like slash


Should I worry about crawl budget?

product is very popular on the internet and people on Reddit are talking about it, other


Should I worry about crawl budget?

sites are linking to URLs in that pattern, then it's a signal for us that people like


Should I worry about crawl budget?

the site in general.


Should I worry about crawl budget?

While if you have something that people are not linking to and then you are trying to


Should I worry about crawl budget?

launch a new directory, it's like, well, people don't like the site, then why would we crawl


Should I worry about crawl budget?

this new directory that you just launched?


Should I worry about crawl budget?

And eventually if people start linking to it.


Should I worry about crawl budget?

I mean, not crawl it at all versus crawl budget.


Should I worry about crawl budget?

But it still goes down to crawl budget.


Should I worry about crawl budget?

Yeah.


Should I worry about crawl budget?

If we decide that we don't want to spend crawling on that pattern, then that's a decision towards


Should I worry about crawl budget?

spending the crawl budget elsewhere.


Should I worry about crawl budget?

Right?


Should I worry about crawl budget?

Or just lowering the crawl budget.


Should I worry about crawl budget?

Not spending your budget is also a budget decision sort of.


Should I worry about crawl budget?

That's interesting because it crawl budget, like whose budget is it?


Should I worry about crawl budget?

Google's budget for your site or your budget that you get to choose to spend?


Should I worry about crawl budget?

That's interesting because yeah, I want to say Google bot's budget.


Should I worry about crawl budget?

Google bot can spend this much on this site.


Should I worry about crawl budget?

Oh, I can't contribute any like, please go over here.


Should I worry about crawl budget?

But you are influencing that with server capacity, for example.


Should I worry about crawl budget?

True.


Should I worry about crawl budget?

Because if we can, or Google bot can get content faster and it doesn't matter if it crawls


Should I worry about crawl budget?

with one QPS or one trillion QPS, queries per second.


Should I worry about crawl budget?

QPS is queries per second.


Should I worry about crawl budget?

Can you say it even more softly?


Should I worry about crawl budget?

Queries per second.


Should I worry about crawl budget?

I'm hungry for some queries.


Should I worry about crawl budget?

Good night time with you.


Should I worry about crawl budget?

Right.


Should I worry about crawl budget?

So it's Google bot spending time on your site.


Should I worry about crawl budget?

Oh, time.


Should I worry about crawl budget?

So it's not necessarily the number of URLs.


Should I worry about crawl budget?

But it's access.


Should I worry about crawl budget?

It's also how long it takes to access each URL.


Should I worry about crawl budget?

So would you get more URLs if it's faster?


Should I worry about crawl budget?

Wait, wait, wait, wait, wait, wait, wait, wait, wait.


Should I worry about crawl budget?

So the time it takes to access a URL that just influences your crawl budget, it doesn't


Should I worry about crawl budget?

decrease it.


Should I worry about crawl budget?

Well, it might decrease it.


Should I worry about crawl budget?

But by Google bot accessing every URL you are spending from your crawl budget.


Should I worry about crawl budget?

So you have, I don't know, like a thousand tokens per day.


Should I worry about crawl budget?

Google bot is going to crawl up to a thousand euros.


Should I worry about crawl budget?

That's not how it works.


Should I worry about crawl budget?

But for this example.


Should I worry about crawl budget?

Yeah, for this example.


Should I worry about crawl budget?

And then you can decrease crawl budget, like your budget allocation, right, by signaling


Should I worry about crawl budget?

Google bot that your server cannot handle the load that Google bot puts on it.


Should I worry about crawl budget?

And those signals are stuff like slowing down serving URLs to Google bot or 503, 429 status


Should I worry about crawl budget?

codes, HTTP status codes.


Should I worry about crawl budget?

Those are all signals that signal Google bots that it needs to slow down and then the budget


Should I worry about crawl budget?

will also go down.


Should I worry about crawl budget?

Okay.


Should I worry about crawl budget?

After you do that, say you then pick a different server, you fix your problem and you want


Should I worry about crawl budget?

to tell Google it's fine again.


Should I worry about crawl budget?

How do we say like permanently like, no, we're not coming back here.


Should I worry about crawl budget?

Like you told us no, so we're not coming back.


Should I worry about crawl budget?

Or how would you get us to come back?


Should I worry about crawl budget?

It's not permanent.


Should I worry about crawl budget?

It's going to increase as the server becomes more healthy.


Should I worry about crawl budget?

So we'll like check again, but it will take us a while.


Should I worry about crawl budget?

Well, technically, you can also stop us from crawling the site.


Should I worry about crawl budget?

Like you can serve 503, 404 robots.txt and then we are not going to crawl anything on


Should I worry about crawl budget?

the site until we get robots.txt or certain conditions are met.


Should I worry about crawl budget?

Don't we say like not to do that for a certain, like for long term, because then we might


Should I worry about crawl budget?

not come back ever?


Should I worry about crawl budget?

Well, it's not ever.


Should I worry about crawl budget?

We don't say ever.


Should I worry about crawl budget?

We didn't say that, but.


Should I worry about crawl budget?

It is not forever.


Should I worry about crawl budget?

It's a longer period of time, but it can be days, two weeks or whatever.


Should I worry about crawl budget?

If something is linked on the internet, then it's extremely likely that we are going to


Should I worry about crawl budget?

crawl that where the links are pointing, like that URL where the links are pointing, because


Should I worry about crawl budget?

that's a good signal that the target page is important enough for many people to link


Should I worry about crawl budget?

to it.


Should I worry about crawl budget?

So if you are a spammer, for example, and you want something indexed, then that's how


Should I worry about crawl budget?

you would go about it.


Should I worry about crawl budget?

I do see now why people are a little worried about crawl budget, because I can see, for


Should I worry about crawl budget?

instance, if you were to launch an e-commerce store and you have lots of products and you


Should I worry about crawl budget?

haven't had an e-commerce store beforehand, but you had an offline store beforehand, you


Should I worry about crawl budget?

might have like a million different products, maybe.


Should I worry about crawl budget?

And then you have a million URLs.


Should I worry about crawl budget?

And obviously, I want to sell today and not in two weeks, not in four weeks.


Should I worry about crawl budget?

On the other hand, I might have technical difficulties, because I just set up that new


Should I worry about crawl budget?

e-commerce store.


Should I worry about crawl budget?

So maybe Googlebot is like, oh, there's a million new URLs that are being linked from


Should I worry about crawl budget?

somewhere.


Should I worry about crawl budget?

So I want to visit all of these URLs.


Should I worry about crawl budget?

But then after 100 URLs, the server slows down due to technical problems.


Should I worry about crawl budget?

And then Google assumes, oh, we are overwhelming the server, so we are backing off.


Should I worry about crawl budget?

And then that slows down crawling and thus indexing and thus things showing up in search


Should I worry about crawl budget?

results.


Should I worry about crawl budget?

And that might make you nervous.


Should I worry about crawl budget?

And there's no button for me to press, no toggle, no switch, no slider to adjust.


Should I worry about crawl budget?

So I don't have control over it.


Should I worry about crawl budget?

Google will figure it out.


Should I worry about crawl budget?

But I, as a site owner, don't have control over that.


Should I worry about crawl budget?

Well, and it's also like a fluid thing.


Should I worry about crawl budget?

Like it could expand or become smaller.


Should I worry about crawl budget?

Like if we find that now you have this whole new section of stuff that we need to allocate


Should I worry about crawl budget?

more budget, that could be okay.


Should I worry about crawl budget?

And for you to sort of just like a strategy game, chicken and egg thing, like, well, if


Should I worry about crawl budget?

I have enough crawl budget, then I can launch this site or do I launch the site and then


Should I worry about crawl budget?

I get the crawl budget, like which thing happens?


Should I worry about crawl budget?

And is it something that you can plan for or is it more of a look at this to see if


Should I worry about crawl budget?

there's an issue?


Should I worry about crawl budget?

Back to your point about you don't have control over it.


Should I worry about crawl budget?

You don't have direct control over it.


Should I worry about crawl budget?

You do have indirect control over it.


Should I worry about crawl budget?

Aha.


Should I worry about crawl budget?

Okay.


Should I worry about crawl budget?

How does it might be?


Should I worry about crawl budget?

Because like, for example, if you go to webmaster to search console, search console, search


Should I worry about crawl budget?

console, search console, and you see that the response, I'm trying to think of that


Should I worry about crawl budget?

report where we have the graphs about like how many bytes we downloaded, crawl stats,


Should I worry about crawl budget?

crawl stats, crawl stats.


Should I worry about crawl budget?

Thank you.


Should I worry about crawl budget?

Settings page.


Should I worry about crawl budget?

And then you see it takes more time to download stuff.


Should I worry about crawl budget?

And then that might be an indication that you need more either processing power, more


Should I worry about crawl budget?

bandwidth, more something, or that you want Google bot to slow down.


Should I worry about crawl budget?

Oh, so it could be something that you want or don't want, like, is it intentional on


Should I worry about crawl budget?

your part?


Should I worry about crawl budget?

You could correlate that with visitors traffic, for example.


Should I worry about crawl budget?

And then you could say that, okay, when Google bot was visiting the site, I also had a thousand


Should I worry about crawl budget?

visitors on my site.


Should I worry about crawl budget?

And I want visitors to have priority.


Should I worry about crawl budget?

So don't do that.


Should I worry about crawl budget?

Like the max QPS that you queries per second that Google bot can do on my site is seven.


Should I worry about crawl budget?

Yeah.


Should I worry about crawl budget?

Okay.


Should I worry about crawl budget?

Fair enough.


Should I worry about crawl budget?

So that seven is ballpark spread out throughout the day.


Should I worry about crawl budget?

And then you are not going to get into the situation where both the visitors and Google


Should I worry about crawl budget?

bot want to visit your site at the same time.


Should I worry about crawl budget?

How do you pick what number is enough?


Should I worry about crawl budget?

I just picked seven.


Should I worry about crawl budget?

I know, but like what's like a normal number?


Should I worry about crawl budget?

Seven is a normal number.


Should I worry about crawl budget?

The site content, like if it's a news or live updates, like where they're posting like live


Should I worry about crawl budget?

updates from some global event or something hourly, like something very frequent.


Should I worry about crawl budget?

I mean, I would prefer if people thought about it before they pick a number and they try


Should I worry about crawl budget?

to estimate how much is too much.


Should I worry about crawl budget?

And for that, you need some idea about how servers work and how HTTP connections work,


Should I worry about crawl budget?

how TCP connections work.


Should I worry about crawl budget?

Because for example, if you have HTTP2 enabled on the server and Google bot is scrolling


Should I worry about crawl budget?

through H2 from your site, then you are going to have a much better time because then we


Should I worry about crawl budget?

can just open a connection and then stream through that instead of overwhelming your


Should I worry about crawl budget?

server with eight, nine, 10 connections at the same time.


Should I worry about crawl budget?

Because all those connections that we have to make will take up resources.


Should I worry about crawl budget?

That's why H2 is more efficient in general from resource usage perspective.


Should I worry about crawl budget?

How you estimate that basically correlating what Google bot was doing on your site with


Should I worry about crawl budget?

what you see in search console.


Should I worry about crawl budget?

You have the access logs for your site.


Should I worry about crawl budget?

And then you can analyze those logs, see when Google bot was accessing through how many


Should I worry about crawl budget?

connections, how fast, like how many requests per second.


Should I worry about crawl budget?

RPS.


Should I worry about crawl budget?

Once you have that, then you can correlate it with what you see in Webmaster to search


Should I worry about crawl budget?

console.


Should I worry about crawl budget?

Okay.


Should I worry about crawl budget?

But also some critical thinking about the type of content you have and if that makes


Should I worry about crawl budget?

sense for your site, for your whole site.


Should I worry about crawl budget?

It's not like, oh, I can just do this for my blog.


Should I worry about crawl budget?

Yeah.


Should I worry about crawl budget?

You can only do this for the whole thing.


Should I worry about crawl budget?

So yeah, I don't think it's simple, but I also think that it's not for everyone.


Should I worry about crawl budget?

It's not like everyone has to worry about it all the time.


Should I worry about crawl budget?

And the other thing is that, because you mentioned the discovered, not indexed or something,


Should I worry about crawl budget?

like if you see those, then it's something about your content or that people don't know


Should I worry about crawl budget?

about your content.


Should I worry about crawl budget?

So you recently updated the documentation about large site owner guide.


Should I worry about crawl budget?

No, it was you.


Should I worry about crawl budget?

Definitely.


Should I worry about crawl budget?

And you added a line that people who should care about crawl budget, it should be also


Should I worry about crawl budget?

like not just people with large sites or frequently updated, but the third bullet is people who


Should I worry about crawl budget?

are seeing a large portion of their site being discovered, but not crawled.


Should I worry about crawl budget?

Right.


Should I worry about crawl budget?

Which we haven't talked about.


Should I worry about crawl budget?

We just talked about discover not indexed.


Should I worry about crawl budget?

And we did a discovery not crawled and that's the bucket where we just don't want to crawl.


Should I worry about crawl budget?

And that could be down to at least a couple of things.


Should I worry about crawl budget?

And one of them is technical.


Should I worry about crawl budget?

And I think we did the update for the technical part.


Should I worry about crawl budget?

So like a large portion, like 80%, I look at the report and it's like 80% discovered


Should I worry about crawl budget?

of my URLs and like, no, I'm moving on to the crawling stage.


Should I worry about crawl budget?

And that's because something's wrong.


Should I worry about crawl budget?

So one of the some things is quality, which is not necessarily that you have spammy content,


Should I worry about crawl budget?

but it could be just that people are not looking for that kind of content.


Should I worry about crawl budget?

Like it could be empty search result pages or something that we, for whatever reason,


Should I worry about crawl budget?

once indexed and then we don't ever want to do that again.


Should I worry about crawl budget?

And the other part is the technical part where the server doesn't have enough capacity to


Should I worry about crawl budget?

handle the crawl requests.


Should I worry about crawl budget?

And then you have the important part of your site, let's say the blog or the slash news


Should I worry about crawl budget?

where we want to crawl a lot and we know that people are looking for that stuff.


Should I worry about crawl budget?

And then we don't have time.


Should I worry about crawl budget?

We have a budget allocation, but we run out of budget allocation for the rest of the site.


Should I worry about crawl budget?

And then we don't get to slash blog or whatever.


Should I worry about crawl budget?

But we could choose to expand it if we saw that the other pages were good, like you said,


Should I worry about crawl budget?

but not if it's the server problems that you're mentioning.


Should I worry about crawl budget?

Because if we just can't, like we can't look at it because of technical problems.


Should I worry about crawl budget?

If it's a technical thing, then it's only you who can solve it.


Should I worry about crawl budget?

It's a you problem.


Should I worry about crawl budget?

It's not us, it's you.


Should I worry about crawl budget?

And there's not that much that we can do on that part because what we could do theoretically


Should I worry about crawl budget?

is that we shift over the traffic from slash news to slash blog.


Should I worry about crawl budget?

But that's not necessarily a good thing because on slash news you might be publishing amazing


Should I worry about crawl budget?

content that everyone is looking for.


Should I worry about crawl budget?

And then we switch over to slash blog and then no one is looking for it.


Should I worry about crawl budget?

It can also be a split where we are decreasing crawling on slash news and then increasing


Should I worry about crawl budget?

on slash blog if we see that people are linking slash blog, for example.


Should I worry about crawl budget?

But yeah, it's down to either content quality or budget allocation.


Should I worry about crawl budget?

But again, that's probably not something that you had to worry about.


Should I worry about crawl budget?

There's also a section on that cross stats settings page for file type.


Should I worry about crawl budget?

So you can filter to see which part is having a problem.


Should I worry about crawl budget?

Is that something else that people should look at?


Should I worry about crawl budget?

This is search console, right?


Should I worry about crawl budget?

It's in search console.


Should I worry about crawl budget?

Yeah, you look at search console, it's good.


Should I worry about crawl budget?

Okay.


Should I worry about crawl budget?

It's like information.


Should I worry about crawl budget?

Like what do you do with it?


Should I worry about crawl budget?

Like nothing.


Should I worry about crawl budget?

I don't know what I would do with it.


Should I worry about crawl budget?

Like it's a PDF.


Should I worry about crawl budget?

PDFs.


Should I worry about crawl budget?

Right.


Should I worry about crawl budget?

I don't think I would care about the PDFs because they are just like normal URLs and


Should I worry about crawl budget?

whatever.


Should I worry about crawl budget?

But maybe...


Should I worry about crawl budget?

Even if all the crawl budgets are eating up with the PDFs, the PDFs are eating up all


Should I worry about crawl budget?

the crawl budgets.


Should I worry about crawl budget?

That's what I meant to say.


Should I worry about crawl budget?

I'll give you a wild example from a real world website where someone was like, oh, only 41


Should I worry about crawl budget?

percent of our crawls are going to the HTML and 15 percent go into the JavaScript files.


Should I worry about crawl budget?

Is that a problem?


Should I worry about crawl budget?

So PDFs specifically, I don't think it's a problem because just don't publish PDFs if


Should I worry about crawl budget?

you don't want them indexed or just robot them out because that might help or just put


Should I worry about crawl budget?

up an extra robot stack, no index, because that will surely help.


Should I worry about crawl budget?

What if you just want more budget for them?


Should I worry about crawl budget?

It's the same budget.


Should I worry about crawl budget?

It's not one more budget.


Should I worry about crawl budget?

It's like for the site you have one budget and that's it.


Should I worry about crawl budget?

But JavaScript is trickier because there you don't necessarily bring in content with JavaScript.


Should I worry about crawl budget?

So you might.


Should I worry about crawl budget?

You might though.


Should I worry about crawl budget?

You might.


Should I worry about crawl budget?

Yes.


Should I worry about crawl budget?

OK.


Should I worry about crawl budget?

That's why I said don't necessarily bring in content.


Should I worry about crawl budget?

And there you might be wasting because we need to crawl everything to see the rendered


Should I worry about crawl budget?

version of the page or we might retrieve it from cache.


Should I worry about crawl budget?

But that's a complication in this case.


Should I worry about crawl budget?

If you need to crawl the JavaScript files, then that means that you are potentially wasting


Should I worry about crawl budget?

crawl budget on that instead of your actual content, textual content.


Should I worry about crawl budget?

So it might make sense to bundle things up like have fewer JavaScript requests.


Should I worry about crawl budget?

And that might help with reallocation of crawl budget.


Should I worry about crawl budget?

But then rendering is also caching a lot, like caching very aggressively.


Should I worry about crawl budget?

So it's probably not that much of a problem.


Should I worry about crawl budget?

And going back to the original question, I can see how for JavaScript, for example, it


Should I worry about crawl budget?

could be a very useful thing, that new report where you can see how much of the crawl budget


Should I worry about crawl budget?

is spent on JavaScript versus HTML.


Should I worry about crawl budget?

I'm wondering if you can tell based off of the proportion, like if that's a normal amount


Should I worry about crawl budget?

or if it should be something.


Should I worry about crawl budget?

I guess if the person is bringing up that they think it might be a problem, like if


Should I worry about crawl budget?

it's 15 percent, I don't know, it's that high or not high compared to what?


Should I worry about crawl budget?

That's a moment where you definitely want to sit down with the people who are technical


Should I worry about crawl budget?

on your team, because if it's a website where the JavaScript is purely decorational and


Should I worry about crawl budget?

doesn't add any value to the content or doesn't add content itself, then 15 percent sounds


Should I worry about crawl budget?

OK.


Should I worry about crawl budget?

But let's say it's like 35 or 90 percent of your crawl is going into that Java.


Should I worry about crawl budget?

Why?


Should I worry about crawl budget?

Yeah, it doesn't add any value.


Should I worry about crawl budget?

Exactly.


Should I worry about crawl budget?

Then you might want to do something about it and maybe just have an X meta robots header


Should I worry about crawl budget?

in the HTP request forbidding it to be indexed and crawled.


Should I worry about crawl budget?

And then maybe that helps because you could wait the JavaScript.


Should I worry about crawl budget?

Actually you can robot the JavaScript.


Should I worry about crawl budget?

Yeah.


Should I worry about crawl budget?

Yeah, true.


Should I worry about crawl budget?

So you robot the JavaScript out so that you don't have requests wasted on JavaScript.


Should I worry about crawl budget?

If it doesn't add to the content, it will fail in rendering.


Should I worry about crawl budget?

But it doesn't matter because it doesn't change the content of the site.


Should I worry about crawl budget?

Oh, right.


Should I worry about crawl budget?

And in rendering, you might get these weird warnings that we couldn't load this resource


Should I worry about crawl budget?

because it was roboted and then you're like, yeah.


Should I worry about crawl budget?

That's the idea.


Should I worry about crawl budget?

So it requires understanding what your website does on a technical level to see if that is


Should I worry about crawl budget?

well spent or if it's wasted.


Should I worry about crawl budget?

Right.


Should I worry about crawl budget?

And not all the warnings or errors in Search Console are bad.


Should I worry about crawl budget?

But they have these errors as well, like crawl errors.


Should I worry about crawl budget?

I think on 404, Josh, the Search Console tech writers put something up that says that if


Should I worry about crawl budget?

you have 404, that's not necessarily a problem.


Should I worry about crawl budget?

It could be completely fine.


Should I worry about crawl budget?

Just make sure that you check that those URLs shouldn't exist.


Should I worry about crawl budget?

Yeah, that's working as intended.


Should I worry about crawl budget?

There's no replacement or anything like that.


Should I worry about crawl budget?

And then that's fine.


Should I worry about crawl budget?

But I understand the concern because, again, that is crawls or requests that could have


Should I worry about crawl budget?

been spent elsewhere.


Should I worry about crawl budget?

Good point.


Should I worry about crawl budget?

Wrong.


Should I worry about crawl budget?

Awesome.


Should I worry about crawl budget?

That's good.


Should I worry about crawl budget?

Now we learned something.


Should I worry about crawl budget?

404, 410, they don't waste crawl budget.


Should I worry about crawl budget?

Neither do robotic URLs.


Should I worry about crawl budget?

Because we didn't get back anything, just the status code.


Should I worry about crawl budget?

If you do, on the other hand, a soft 404, that's a good job.


Should I worry about crawl budget?

You're not putting anything in the index and wasted crawl budget.


Should I worry about crawl budget?

Okay, cool.


Should I worry about crawl budget?

Cool.


Should I worry about crawl budget?

No, that's a good triple.


Should I worry about crawl budget?

I wonder what people, or if people have more weird questions about crawl budget, maybe


Should I worry about crawl budget?

you could ask people to add John Moo questions about crawl budget.


Should I worry about crawl budget?

And we can bring him back here.


Should I worry about crawl budget?

We can also just write YouTube comments.


Should I worry about crawl budget?

Oh, no, I will ignore those questions.


Should I worry about crawl budget?

No, but that's why we bring John back.


Should I worry about crawl budget?

Well, maybe I don't.


Should I worry about crawl budget?

You're also mean.


Should I worry about crawl budget?

So if you out there do have questions about crawl budget, feel free to comment them either


Should I worry about crawl budget?

on Twitter or better on the YouTube comments.


Should I worry about crawl budget?

If you listen to the show elsewhere, do go to our YouTube channel as well.


Should I worry about crawl budget?

Or just add John Moo.


Should I worry about crawl budget?

Anywhere.


Should I worry about crawl budget?

Go to Myspace.com slash John Moo.


Should I worry about crawl budget?

Okay, fair enough.


Should I worry about crawl budget?

Or Friendster.com.


Should I worry about crawl budget?

That's also the place where the cool kids hang out.


Should I worry about crawl budget?

I mean, John is still on Google+.


Should I worry about crawl budget?

So am I.


Should I worry about crawl budget?

Yeah, me too.


Should I worry about crawl budget?

So you can also try to add us on Google+, I guess.


Should I worry about crawl budget?

I have author annotations on my Google Plus profile.


Should I worry about crawl budget?

That's very professional of you.


Should I worry about crawl budget?

With that being said, thank you so, so much for clearing up some of the questions and


Should I worry about crawl budget?

confusions about crawl budget.


Should I worry about crawl budget?

I think this was really interesting.


Should I worry about crawl budget?

And I didn't know the 404 thing about crawl budget.


Should I worry about crawl budget?

That was news to me.


Should I worry about crawl budget?

And I do hope that you all enjoyed the conversation, learned something, had some fun along the


Should I worry about crawl budget?

way.


Should I worry about crawl budget?

And next time on Search Off The Record, we'll be taking a look at dealing with non-web page


Should I worry about crawl budget?

content like PDFs.


Should I worry about crawl budget?

Oh, I don't know what those are.


Should I worry about crawl budget?

That's gonna be fun.


Should I worry about crawl budget?

So thanks a lot.


Should I worry about crawl budget?

We've been having fun with these podcast episodes.


Should I worry about crawl budget?

I hope you, the listener, have found them both entertaining and insightful too.


Should I worry about crawl budget?

Feel free to drop me a note on Twitter or chat with us at one of our next virtual or


Should I worry about crawl budget?

in-person events we go to if you have any thoughts.


Should I worry about crawl budget?

And of course, don't forget to like and subscribe.


Should I worry about crawl budget?

Thank you so much and goodbye.


Should I worry about crawl budget?

Bye.


Should I worry about crawl budget?


English Google Webmaster Central office-hours from November 29, 2019

All right, welcome everyone to today's Webmaster Central


office hours hangouts.


My name is John Mueller.


I'm a webmaster trends analyst here at Google in Switzerland.


And part of what we do are these office hour hangouts,


where webmasters and publishers can join in and ask


about their websites in search.


Since it's a bit of a holiday, I guess


maybe there won't be that many people, but we'll


see how it fills up.


A bunch of questions were submitted already on YouTube,


so we can run through some of those.


But before we get started with them, maybe one of you


has something on their mind that they really


want to talk about in the beginning.


If not, that's OK too.


You're also welcome, of course, to jump in in between.


Hello, John.


Oh, hi.


Hi.


Do you take question about Google My Business?


I can't really help much with Google My Business.


OK.


Is it specific to Google My Business,


or does it relate to something you've shown in service?


It was about the maps.


For example, we got an issue with people creating fake,


positioning fake listing around our main establishment.


And they are doing it multiple times.


We are trying to remove them by doing the reports.


And every time, after a little bit of time,


we see something appearing with false images, false address.


Do you know if there is a solution for this


to I don't know.


No.


So I would recommend, on the one hand,


using the tools that you have available there,


and on the other hand, maybe checking in


with the help forum for Google My Business


to see if the folks there have some more information.


OK, thank you.


OK.


All right, any other questions around web search


that maybe I can help with?


Otherwise, I'll just get started with what was submitted.


And like I mentioned, feel free to jump in


if there's something that you want more information on,


or if you have related questions, any of that.


All right, fairly long question.


Let me see if I can simplify it a little bit.


In Search Console, it's written that the unexpected 404 errors


might be generated by Google bots trying


to follow links in JavaScript, Flash files,


or other embedded content.


For example, some Google Analytics link.


In Search Console help, it's also written that this error


has no effect on crawling and ranking of your site.


Is this always true?


I'm asking because we have on our site many cases


like this, specifically related to image files.


Checking our logs related to Googlebot image,


we see that last October, Googlebot image requested


more than 3,000 resources.


And we know for sure that these requests


are related to 404 URLs generated


by Googlebot trying to follow links in embedded content.


Could this be a problem?


Could it be affecting Googlebot images' ability


to crawl our actual images?


So I guess the one thing that you don't have mentioned


in the question here is, are your images otherwise


being indexed normally?


And if your images are being indexed normally,


then this is not something that I would worry about.


What can sometimes happen is that sites


embed their own images with the wrong URL.


And then when users look at those pages,


those images don't work.


And then when we try to crawl and index those images,


we can't get to those images either.


So that would be the kind of situation


where I'd say that's something that you'd need to fix.


However, if it's just the case that in addition


to all of the images that we have crawled and indexed


on your website, we try out a number of URLs that return 404,


then that's absolutely not a problem.


For the most part, especially when


we're crawling images, a 404 error


is something that we can process really quickly.


It doesn't distract us from processing the images that


are really on your website.


So in that regard, I would not worry about this.


But I would really double check if you're already


looking at your server logs, double check


that we're actually crawling the URLs that you do care about.


And if we're not crawling the URLs that you do care about,


then double check why Googlebot might be missing out on those.


We're looking to incrementally modernize our WordPress website


with some modern web frameworks like React.


Since the site is PHP WordPress, we currently


don't have the infrastructure to server-side render


JavaScript and React code.


So we're thinking about possibly foregoing


server-side rendering in favor of just server rendering


of structured data, like JSON-LD.


For the short term, our long-term goal


is to move away from WordPress.


So this would mean that a site's HTML would only


be client-side rendered, but there would still


be some server-side rendered structured data.


Is this feasible?


Would this cause significant SEO hit?


How does Google consider structured data in search


results and in rankings?


The alternative would be dynamic rendering.


However, we're wondering if we can get away with not managing


puppeteer clusters.


So there's a lot packed into this question.


And I feel if I just say yes or no,


it's probably not going to be a great answer for you.


In practice, these kind of changes


can have a significant effect on your SEO.


In particular, what you're essentially doing


is changing the layout of your website


and moving from something that is pure HTML to something that


has a different layout and that uses a different technology,


namely client-side rendering JavaScript, which


means along that path, there are a lot of small changes


that add up, and they can have a significant effect on how


Google indexes your pages.


On the one hand, layout changes can have an effect,


especially if you're making changes


that would affect the internal navigation of a website


or the URL structure of the website.


That's something that can have a significant effect.


That's even without looking at the trickier problem of moving


to a JavaScript framework.


So that's something that can definitely have an effect.


The JavaScript framework part is something


where I would be kind of cautious taking that on,


in the sense that it can definitely work.


It's not something that by design would not work,


but it's also something where you


need to make sure that you're implementing it


in a way that does actually work for search.


So what I would recommend doing there


is setting up a bunch of test pages, either on your website


or on a testing website, and using the various tools


and the documentations that we have available,


really double check to make sure that everything on those pages


is actually indexable.


And then that's something where you could step by step


try to take that and see how does this


react within my normal website.


So in particular, what I usually recommend doing


is taking some pages where you see enough traffic from search


that there are things that you can measure there,


and then converting those to your new setup,


maybe kind of splitting that up, depending on how you have


your server set up, and really double checking


to see what is kind of the midterm effect from changing


those pages over to the new infrastructure that you have,


the new frameworks that you're using,


the new layouts that you're using,


and really double checking that those changes are OK for you.


So I wouldn't just see is it indexable or not,


but I would kind of leave it running for maybe a month


or so to see where does it actually settle down.


Because especially when you make bigger changes,


you will see some fluctuations for kind of the short term.


And the question here, I think, is more in the long term,


will these fluctuations kind of settle down


at a state that are similar to what we had before?


And that's something that you can really only test.


So from that point of view, I wouldn't


take this as something like, oh, I will just swap out


the whole back end and add some JSON-LD structured data


to these pages.


And we'll see what happens.


But I'd really try to take this on in a controlled way


so that you know along the way where things go wrong, where


things are actually working out well.


And as always, with any kind of site redesign,


it's not always the case that the changes that you see


will be negative.


It's very possible that you do a revamp of your website,


and that results in Google understanding your content


a lot better, and Google being able to crawl and index


your pages a lot better, and then kind of the ultimate


effect being more visible in search.


So changes alone are not something


where I would say that would be a problem.


It's really a matter of where these changes go.


And then I have my website is getting thousands of links


from a porn site through some domain


where it's kind of redirecting through an error page,


apparently.


What is happening, and what's the best solution


for this problem?


Is my website being penalized due to this?


So these kind of links from spammy sites are really common.


I wouldn't necessarily worry about that.


One thing I did notice looking at your website


is that it's built on an expired domain.


So there used to be completely different content there.


And that, I think, is probably the bigger effect


that you'd be seeing.


Not so much kind of the links from adult content sites,


but really it's like, well, your website


is not what it used to be.


There's like a really big change there


with regards to the way that the content used to be


and what you have there now.


So on the one hand, I wouldn't expect


that your current website ranks the way that the old domain


used to rank.


I would expect that things are slightly different.


And on the other hand, if you're building on an expired domain,


then obviously all of the things that this domain


has collected over the years, they're still around there too.


So it's not that you can just say,


well, there are some random adult content


links linking to my website, and I


don't know what the effect is.


Rather, you're building on a very shaky foundation already.


And whether or not those particular links


are the problem or the whole shaky foundation,


that's kind of the question.


For new websites that have been internally scored well enough


to be given a chance for head keywords,


I've seen Google continuously test their pages


by giving them small bursts of traffic a few days or weeks


apart.


It is as part of the testing signals


it's collected over time on those pages,


sort of like Google saying, you're new, and you're good,


and I want to trust you, but I want to take my time.


I'm not aware of any particular system on our side that


would be saying, well, we need to kind of double check


how these pages work, and we will send them some sample


traffic from time to time.


But in general, with new websites,


it is something where we don't have a lot of signals.


So if we don't have a lot of signals,


we have to guess sometimes.


And our algorithms might end up trying things out.


Our algorithms might end up kind of guessing


to see what the stable state could be.


And then over time, over a couple of months,


you'll see that things settle down


into something more stable.


And this is something that can go both ways


in the sense that maybe our algorithms will


guess in the beginning that this is actually something that


is really, really important, and we need


to show it a lot more visibly.


And later on, we realize maybe it's not that critical.


And similarly, it can happen that we look at it and say,


well, we're not really sure how to show this in search.


And over time, we learn that this is actually


something that is really good and worth showing


a little bit more visibly.


So it can go up.


It can go down.


But especially in the beginning, it's


something where it's a little bit


of an unclear situation for us.


Now that Google can index JavaScript,


we're thinking of using more JavaScript to increase


our website's performance.


At the moment, all of our content is in the initial HTML.


IT says loading the content using JavaScript


would decrease page loading time.


If we made this and also implemented dynamic rendering


so that our content would be indexed faster,


what could happen?


Is it a terrible idea, or should it be fine?


Is it better to leave it as is?


This is kind of, I guess, similar to the previous question


where someone wanted to swap out WordPress against some React


setup.


In general, this is something that you can


do in a way that works well.


It's also possible to do it in a way that causes trouble.


So like with the previous question,


I would be cautious in kind of going down this direction,


but I wouldn't say that it's impossible to do that.


There are lots of speed improvements


that you can do with static HTML.


Sometimes there are really cool improvements


that you can do with JavaScript as well.


So I would try these things out, test them out,


test them out with your actual pages.


Like if you can do something like an A-B test


where you have some set of pages where you know you're getting


good traffic from search enough that you can measure it,


swapping those out and seeing what


the reaction is over maybe a course of a month or so.


Some questions regarding AMP.


My website is in WordPress.


Like, oh my gosh, so many WordPress sites.


And I've enabled AMP for it.


Please give me a hint about native AMP


and how Google sees it as there will be no canonical URL.


Also, I want to ask how Google ranks AMP pages.


Is there any specific criteria for AMP pages,


or is it the same as a normal web page?


So I don't know which AMP plugin you used.


I know Google has been working on one, which seems


to be working out really well.


If you're using native AMP, I assume


you don't have the legacy HTML pages anymore,


but rather you switch completely to AMP.


In general, that's kind of the approach


that I recommend because then you just have one URL,


and you don't have this problem of having an AMP URL


and a normal kind of HTML URL.


If you can switch completely to AMP,


then I think that makes a lot of sense.


Obviously, not all sites are really


suitable to switching completely to AMP.


So that's something you have to figure out.


If it's mostly a content site, and it's hosted on WordPress,


and it's more a matter of using a plugin


to just turn it on and off, then that


seems like a pretty easy approach.


We rank AMP pages the same as we rank any other pages.


So there is no inherent advantage to using AMP.


There is no ranking boost from AMP.


Some search features do require AMP pages


so that we can show them properly


because of the pre-caching and all of that.


But otherwise, there's no ranking advantage


from using AMP.


Obviously, AMP pages can be a lot faster than the legacy


HTML pages.


You can make really fast HTML pages, too.


So it's not that you need to use AMP to have a fast page,


but they can be quite fast.


And speed for us is a ranking factor, especially on mobile.


So all of these things add up.


But ultimately, I'd say if you think


that it works well for your website to have AMP pages,


and you can go to a pure AMP setup,


then I don't see any problems with that.


I just wouldn't expect any magical boost


to the top of the search results from just switching to AMP.


SPEAKER 2 Hey, John.


You want to take a question on APIs?


JOHN MUELLER Sure.


So in Discover, sometimes our website shows AMP pages,


and sometimes mobile pages show up.


The template is the same.


The speed is the same.


The story are different.


But sometimes mobile pages are showing up,


and sometimes AMP pages are showing up.


Also, sometimes bigger images are showing up,


and sometimes smaller images are showing up.


SPEAKER 3 So the reason for the images part


is because it has a direct impact on the CTS.


So do you see any logic there, like why AMP site not AMP,


while the speed is pretty much same in both the templates?


JOHN MUELLER I don't know.


My hunch would be that it's more a matter of us


having discovered and recognized the AMP version of those pages.


So if we do know there is an AMP version of the page,


and it's properly connected with the canonical back and forth,


then that's something where we would


try to show the AMP version as much as possible


on mobile devices where that would be compatible.


So it's not something, at least as far as I know,


it's not that there is any kind of heuristic that would say,


oh, in special cases, only then show the AMP version,


otherwise show the kind of traditional mobile version.


So it feels to me it's more a matter of timing and maybe


proper understanding of those two pages


that we can actually understand.


This is the HTML version.


This is the AMP version.


SPEAKER 2 Or do you consider the first crawled page type


as one of the source to rank in APS Instantia?


Meaning like when you picked up the AMP site first


because I'm in mobile indexing, and then you didn't even


hit my AMP site, then you suddenly picked up


and then showed up there.


Because when we looked at this page type, there's no error.


Everything is perfect.


And fortunately, what happens is the first page


is AMP site, and the second link is, again, AMP page.


So there is no correlation that we are not at all


coming there as an AMP, but the same category, same folder,


one in AMP site, one in AMP site.


OK, so you have basically desktop pages,


AMP site version, and the AMP version?


Or do you just have kind of responsive and AMP?


No, we have three different versions.


OK.


Yeah.


So what I would make sure there is that from the mobile version


you also have a link to the AMP HTML version so that if we


pick up the AMP.


Yeah?


I guess the thing that I'm not completely sure about there


as well is from the AMP version, would you


have a link back to the m dot version


or to the desktop version?


We have both.


Canonical.


We have like desktop.


Canonical is the best version.


I could see maybe that might be something that's confusing us,


but I need to double check with the team on that.


I can push to the URL.


And second question is like, for the videos,


the image always goes in the smaller icon


than the bigger icon.


Yeah.


And that has a direct impact in graphics.


That's something.


So for AMP pages, we can show the bigger preview image


in Discover.


And what you can also do is use the meta tag for, what is it,


max image size, I think, where you can specify large


to let us know that for this particular page,


we can also use a large preview image.


So that's something that would also need to be on, I guess,


on the mobile and on the desktop page.


So for videos, what we have noticed


is all the websites are showing only mobile pages.


Earlier, it used to be AMP pages, but now not only us,


but competitors also, all websites, even YouTube.


Any video that shows up in Discover


are mobile only never achieves AMP since last couple of months.


OK.


That's weird.


If you can send me some examples,


I'm happy to take a look at that with the team.


Sure, we will do that.


Also, one last question on APIs is that, I mean,


there are multiple, but what we're generally seeing in India


is that the cache rate or the refresh rate of APIs


is pretty bad.


So in the morning, we see yesterday articles,


which I thought which is OK.


But in the afternoon, I'll see like Sanas ago articles,


which is not based on the real time.


So do you see any cache issue that is happening,


or is that something we have to deal with that?


I'm not aware of anything specific.


But I haven't heard from other people either about this.


But I'm happy to take a look at that, yeah.


OK.


Hey, John, I have a question.


Sure.


Yeah, hi.


So being a news publisher, we see most matters most, right?


So being a news publisher, we publish


all the news in real time.


But in Google News, Carousel, or AMP, Carousel,


generally I see three weeks ago article, two weeks ago article,


14 hours ago, 20 hours ago.


And there are some news publishers, as you know,


there are domain crowding where only two or three publishers


are coming.


But everybody is contributing to Google.


So I don't understand what is the reason


that the old articles are there and only two, three publishers


are ranking for every query.


It's something I can't really say in general,


because it really depends on the query and the timing there.


But it's definitely not the case that we would say we


try to show older content.


I completely agree on the timing on the point.


But I just want to understand why we are showing,


like in the morning, there was some big news


in the parliament, Indian parliament.


And if I search on Google, I can see articles two weeks ago,


one week ago.


And it's like four hours ago when the news breaks.


I don't know.


That seems suboptimal.


Yeah.


I think, John, to add to that, in India, basically, this


is the biggest problem we always see is that the timestamps


don't generally get updated, especially


when you have an event, right?


It goes really for a toss.


This is something that's a common feedback.


I think we already shared once.


But I think this pretty much we are living today in the search.


OK.


So the timestamps in particular are not


don't match the article or?


I mean, while the article updates,


Google still shows that six hours ago, seven hours ago.


But interestingly, sometimes what happens


is that the article title that is still showing in Google


is the latest one.


But the timestamps will show six hours ago.


Interesting.


If you all can send me some examples,


I'm happy to take a look at that.


Sure, we can share some examples with you.


Cool.


All right.


So completely different question.


My competitor is sending fake traffic


to my website using a bot in huge amount.


How does Google see it?


Will I get penalized for that?


What do I do?


With regards to SEO, this has absolutely no effect at all.


So you can ignore it.


You can block it, whatever you want.


If I use the same website template


to create multiple websites, will Google


consider it plagiarism?


And does it affect SEO?


So using the same template itself


is not so much of a problem.


But using the same content on multiple websites


essentially means that you kind of dilute


the value of your content.


So it makes it a lot harder for us to recognize,


this is actually something really good and important.


If you're saying, well, it's across all


of these different websites, it's all the same content,


then when it comes to trying to rank things,


we might say, well, something else looks really important.


And this is kind of like each of these individual pieces


has a little bit of value, but not enough


to actually be shown competitively in the search


results.


So that's something where just using the same template


is no problem.


Using the same content across multiple sites,


that's generally just kind of a bad practice.


Does AMP increase direct traffic?


So direct traffic is when people come to your site


directly by entering the URL or kind of using the bookmarks.


And that's not something that AMP really directly affects.


I mean, indirectly, maybe, in the sense


that if you have a really great and fast website,


that could be something where people might say, oh, I


like this.


I'll continue coming back.


But directly, there is definitely


no effect on direct traffic for AMP pages.


I have a site that supports multiple languages.


Sometimes we have content in four or five languages,


but you can still see a page in untranslated content.


We're being flagged as having an issue with reciprocal hreflang


tags.


So I think the general issue is you have multiple language


versions of your website, and you


don't have all of the content in all of those language


versions.


So in general, that's kind of OK in the sense


that you can have hreflang for those pages anyway,


because probably the template of the website is translated.


So for example, if you don't have a piece of content


in Korean in this case, but the template of the website


is in Korean, then that's still a useful page


to point Korean users to.


So the hreflang there would still be OK.


At the same time, you could also say, well,


I don't have this page in Korean.


I don't even want to send Korean users


to the English version of the article


on a Korean templated website.


Then that might be something where you say, well,


I just won't use the hreflang on that Korean page.


So from my point of view, you can do it either way.


Either include that page in the hreflang set,


or don't include that page in the hreflang set.


The other thing to keep in mind here


is that hreflang is handled on a per page basis.


So if we can recognize that, say,


three or four of the five hreflang links


that you have on a page are OK, then we will just use those.


And if we can recognize that that one is not OK,


then we will just skip that one.


It's not that there is a negative effect on the hreflang


set for that piece of content.


It's also not that there is a negative effect on the website


itself if one of the hreflang links does not work.


So from that point of view, it's not


going to be critical if you know that this page is


kind of iffy with regards to translation


and the hreflang set up for that particular page is not OK.


My main service page is not ranking even


in the first 100 pages of Google,


where all other pages are performing well.


I doubt this is due to duplicate URLs,


as I couldn't find any old URL in a different file


format and different protocols.


This might have happened when the website was converted


to WordPress and the pages redesigned long ago.


I've redirected 301, redirected all these URLs


to the present service page.


How will I inform Google to recheck


the URL for duplication or if the issue is solved?


So I don't completely follow what has happened here


and what is happening with your pages at the moment.


But you can use the Inspect URL tool


to let us kind of re-index a specific page.


So if you're worried that maybe something shifted along the way


and it's like maybe the content isn't really what


or the content has been updated and you want to make sure


that Google has kind of taken a new version into account,


then using the Inspect URL tool and submitting that to indexing


is definitely something that you can do.


I wouldn't be shy about using something like that.


That's kind of what it's meant for.


With regards to changing website structures,


like I mentioned with the other cases,


you can make website structure changes.


And sometimes these just take a while to be reprocessed.


So depending on what you mean with these pages


were redirected long ago, that might be something where maybe


it's still being processed.


If it was, I don't know, a couple of weeks ago


or maybe they're already processed and already kind


of in that stable new state if you


made this change, say, several months or maybe even years ago.


What's the best course of action if you have a website,


pages drop completely out of search index,


and you've received no manual action notifications?


Yeah.


Hard to say.


So I think, first of all, it's worth


saying that we don't index all pages.


That can be completely normal.


Sometimes with the new kind of features in Search Console


where you have the coverage report,


it can be tempting to kind of focus on some of the issues


there and say, I need to fix this problem.


I need to get all of these URLs indexed.


When in practice, it's just, well,


we've indexed a lot of pages from your website.


And at the moment, we don't feel that it's


worthwhile to index even more.


And that's not something where there's


a technical reason involved.


It's more that, well, we're not sure if it's really


worthwhile to spend so much time on these things.


So that could be, for example, if you


have a ton of different URLs on your website


and they're all kind of similar, if you have, I don't know,


a directory of all businesses in your country


and their phone numbers, and we've crawled through 1 third


of those, and we're like, well, these


are all things that we've seen before.


Do we actually need to index all of these or not?


So that's kind of one aspect there.


The other aspect could also be that maybe there


is a technical issue involved.


It's actually worthwhile to kind of dig


into those individual pages to see what exactly


has been happening there.


So that's kind of the general split


that I would try to focus on is first


try to get kind of some rough understanding of,


is this perhaps just the case that Google


is indexing as much as it feels is necessary for my website?


And maybe some of these other things


would be worthwhile to kind of rework in a way


to really prove to Google and to users


that this is really important content.


That's something where maybe it's


worthwhile to have a second pair of eyes


look at these issues with you and really give you


some more honest feedback, if possible.


And on the other hand, if you feel


that these are actually really good and important pages


for your website, and for some reason


they're just dropping out, then that's


something where I would try to dig in for technical reasons


and try to figure out why these particular pages are not


being indexed.


Is maybe another version of that content being indexed?


Is maybe there's something weird happening with my server


that I could look into?


All of those more technical questions.


Are attachment pages bad for SEO?


Will Google penalize a website if its attachment pages


are indexable?


So assuming these attachment pages are for attachments


that you have maybe on a blog, and these


are reasonable things on a reasonable website otherwise,


then there's nothing against having those indexed.


So there's definitely not a penalty


that on our side where we would say attachment pages are


indexable, therefore we will demote the website.


That's definitely not the case.


The general question I'd say is more,


are these attachment pages valuable for your website?


Is there something of value that you're


providing on these pages that when users go there,


they say, oh, this is really fantastic website.


I will stay, and I will look around a little bit more.


I will recommend this to my friends.


Or are these attachment pages essentially just


placeholders where when people go there,


they're like, how did I end up here?


And what do I need to do next?


And why did Google send me here?


Then that's the kind of pages where I'd say,


probably not worth getting those indexed.


But that's not an issue of attachment pages or not,


but rather what content do you want to have indexed?


And that's something where you can decide,


this is a good use of my time or a good use of my visibility


in search.


Or maybe this is something where I'd rather


have people actually go to different pages


within my website.


John, if you have one minute to explain about why YouTube


is actually taking up the entire listing page.


I'm sure it's a Google product.


Do you want to throw any light around that


to give us some indication that on videos,


why is YouTube so important than a personal publisher who


actually has the video?


I don't actually know how much we're showing YouTube.


So that's kind of tricky.


But in general, we spend a lot of time


to make sure that we don't give any of our own services


any preferential treatment in search in that regard.


So YouTube pages are video pages like any others.


They obviously work to try to make these pages work well.


I think, in particular, YouTube almost


has a little bit more challenging than other video


pages in the sense that most users on YouTube


probably go to the app.


So that's where they put most of the content.


And kind of the mobile version of their pages


is not really that well suited for search.


And with mobile-first indexing, we


try to index the mobile version of those pages.


So that's something where they struggle with SEO just as


much as any other website.


So from that point of view, I don't


think there is any inherent bias to say


we should show more YouTube content in search


than anything else.


But there is a lot of content there.


And they have a lot of good content.


So from that point of view, I think


it makes sense to show some things from there.


But obviously, there are lots of other video sites, too.


And we need to make sure that we can show their content as well.


Over the years, I've been in touch


with a number of these video sites.


And sometimes they do things really well.


And sometimes they also struggle with SEO, like anyone else.


And especially for really large websites


where you have a lot of content that is in video form,


where it's hard for us to recognize what is actually


in the videos if we don't have any text around that,


then that can be sometimes really tricky


to crawl and index really large sites like that.


But that's something where everyone struggles with SEO.


And especially bigger sites, they also


have the inherent organizational challenges


that come together with any bigger business


in that you want to make some changes on these pages


because there's something technically wrong.


Then it just takes a really long time


until you have all of those permissions.


So I don't know.


From that point of view, I don't particularly


believe that we have any inherent YouTube bias


that we'd show more content from YouTube in search.


But they do a lot of things well.


There's a lot of good content there.


Other sites do well, too.


And we really need to make sure that we're showing


kind of a balanced mix of sites.


I think we'll share some samples where the publisher


first publishes and then goes to YouTube,


trying to take advantage of YouTube as a dot


usual dot com, maybe bigger in terms of ranking.


But maybe it is taking up the first slot


and then the publisher comes.


There will be reasons, but we'll share the samples


with you to dig in separately on that.


OK, I'm happy to take a look.


I think the other aspect is also tricky if you have a video


that you host on YouTube and you also


put it on your own pages.


Then that's something where our algorithms


try to figure out which is the best landing page to show


there.


That's always kind of a tricky situation.


But happy to take a look at some examples.


And second question, follow up to the video.


So kind of forward looking, I would always


expect to see changes in search.


I don't think that really helps you at the moment,


but I know the team is always working


on trying to find the right balance there.


And that's something where generally what happens


is we have a number of algorithms that


try to determine which of these kind of search results


blocks to show above the others.


And essentially, they're trying to determine


what is the most relevant for users


for those specific queries.


And sometimes it can happen that videos are shown on top,


sometimes images, sometimes a news block.


That's something where I think we generally


get the mix fairly well, but sometimes maybe we don't.


And in particular, I could imagine


for non-English content, it might be even trickier for us


to understand what specifically is being searched


for with this query.


Are they looking for more up-to-date news,


or are they looking for background information?


Are they looking for images or videos on this topic?


Sometimes there are also videos that


are indexed that are very news friendly in the sense


that they're really fresh.


And it's also worth showing together with news content.


But this kind of mix is something


that I know the teams work really hard on,


and it's really tricky to get right,


because I don't think there is one answer that always works.


Yeah, I mean, we do take that kind of feedback seriously.


But it's also important from our side


that these search results work well for users.


So kind of having feedback from publishers saying,


I want my content to be shown on top rather than someone else's


content, that's something where, OK, we


can kind of have a reasonable discussion about that.


But ultimately, what's important for us


is that users get the search results that


are relevant to them at the time when it is relevant for them.


And sometimes publishers see these kind of issues


a lot clearer than maybe users would


be able to formulate them.


So that's always useful to have.


But sometimes it's also kind of just that conflict


between publishers would like to have more visibility in search,


and we think that maybe there are other things that


are also important to be shown in search.


Yeah, I think it's not something like where one publisher wants


to rank better, but because of this,


YouTube is something which is getting benefited.


So it's always like the YouTube, videos ranks first,


and then the YouTube always takes the slot completely.


So that is the kind of question that I


ask you to spend some time to talk about the YouTube,


because Google is benefiting, but not the publisher,


but the YouTube is taking the benefit out of it.


So that's where I think the feedback is a question.


Not to rank between the publishers,


but the news versus videos, and videos takes advantage.


If that is the case, then YouTube is taking 90% share.


Yeah, I don't know.


I could also imagine situations where maybe some fairly


fresh videos are really kind of comparable to news content.


So I don't know the specific queries that you're looking at.


Sure.


Yeah.


All right, and a question relating


to kind of one of the previous Office Hours.


So some websites are copying their content,


and they're using a piece of JavaScript


that kind of adds a link back to the website with a hash


after a fragment identifier.


And the general question is, is this a problem


or is this not a problem?


From our point of view, that's perfectly fine.


If you have JavaScript that includes kind of a link


when people copy and paste content from your site,


then that's something that kind of makes sense.


The fragment identifier at the end of the link


is something that we would drop because we


try to use the normal kind of URL instead.


So with the path and the file name


and the fragment identifier, we would drop.


So that's something where we would


see that as a normal link to that normal URL.


Would Google consider these to be natural links?


I think there are two aspects here.


On the one hand, if these are scraper sites that


are copying and pasting content from your website,


then probably those links are not something


that we would value that much anyway.


So it's not something that would be high on our radar.


On the other hand, if people are putting content


on their website and linking back to you as a source,


then that seems like a normal natural link to me.


So I wouldn't necessarily worry about this.


But kind of how I mentioned there, on the one hand,


these are probably not going to be that useful links


for your website anyway.


So on the one hand, I wouldn't worry


about this as being a kind of natural or unnatural link


because they're not valued that much.


But also, I wouldn't expect to see any positive effect


from these kind of links because probably those sites are not


going to be things that we'd say are kind of useful references


for a website.


Does Bert affect?


Sure.


Hi.


Yes.


Yeah, hi.


So we are a new publisher.


And we have a business news website


which happens to follow all the Google guidelines, the news


guidelines, and everything.


The site maps are working perfectly fine.


The feeds are working perfectly fine.


We are getting indexed.


We are getting indexed in Google News as well.


But somehow, we are not appearing in the top box.


So we have raised this query multiple times


on Google support and to the Google team.


But we have never got a satisfactory response.


We have never received any message in our search console


about what is happening and what is the issue.


So can you please enlighten me something about it,


what is happening here, and what is the exact issue?


Yeah.


So the what is it called?


Kind of the box on top there.


That's something that is, from our point of view,


is an organic search feature.


And it's not something where news publishers


automatically show up.


It's not something that's limited to news publishers.


That is the thing.


But the thing is that we used to appear in the top box.


But it's been one and a half years


that somehow we got disappeared from there


without getting any notice in the search


console or something.


We never, never, ever appear for even for a single query


in the top search.


We get appeared in the search result,


but never in the top box.


Yeah.


So we don't have anything.


As far as I know, we don't have any kind of manual action


that would block a site from appearing in the box.


I forgot what the official name is.


But from that point of view, it's


not that there is kind of a setting for your site


or anything like that where we would say


this site should never show up.


But rather, this is some top stories.


Yeah, top stories.


That's what I meant.


So from that point of view, it's not


that there is any setting or anything like that,


which would be blocking it.


But rather, we see this as an organic search feature.


And over time, our algorithms evolve.


And it can happen that a site is visible in Google News


and does not appear in the top stories section.


that I'd say this is particularly


kind of a bug on Google's side or something


that you're doing incorrectly.


But rather, we try to show the appropriate results


in the top stories section.


And sometimes that includes some sites.


Sometimes that doesn't include other sites.


But it's not the case that we kind of block sites


from appearing there.


So from that point of view, it's kind of tricky to say.


I understand your point.


But then we have other websites as well, other English websites,


Hindi website, and regional websites.


They do appear for search queries in the top box.


But for a business website, there's


not even a single query, not even a single query


that appears in the top box.


So I can understand that it happens that sometimes one


website shows and the other one doesn't show.


But it cannot happen in everyday basis


that not even a single news is getting


picked in the top story.


It can happen.


I don't know your website.


So it's really hard to say if that's something that I


would expect for your website.


But it can certainly happen that a website is.


It's been one and a half to two years.


This is not happening.


It's not a one month or one week old thing.


It's been quite two years.


And despite of writing multiple queries on this webmaster


search forum and the Google people


and asking them in the seminars, I


have not got any response that could actually


relate to the issue or what I can do on my part


to solve this problem because I have seen everything.


Everything is getting followed, all the guidelines,


all the schemas, feeds.


Everything is working fine.


It still does not appear.


That can happen.


I mean, on the one hand, the technical things


are things that are always useful to follow.


But it's not that there is a technical requirement where


we say, you need to do this, and then we will show you there.


But rather, there are lots of also quality issues


that are involved there with regards


to trying to figure out what type of website to show there.


And it can certainly happen that a website exists


for a longer period of time and is even indexed in Google News.


But we still think, well, we're not


convinced that it makes sense to show this website in the Top


Search, Top Stories section.


So that's something where I wouldn't say offhand it's


impossible that any new website is able to show up in there


because new websites show there all the time.


But it's also not the case that it's


a sign that there's something specifically wrong


with a website when we don't show it in a Top Stories


section because it's an organic search


feature and the criteria to be shown there


from an algorithmic point of view.


We have a lot of things that play into that.


And it's not guaranteed that we would always show a site there


or that we would ever show a site there.


Does BERT affect the ranking of websites


which have content in other languages than English?


So I struggle a lot with all of these BERT questions


when they come up because from our point of view,


BERT is a way of understanding text better.


And we use that, on the one hand,


to understand queries better, so to better understand


when someone writes something complicated into the search


box, and on the other hand, to understand pages


a little bit better.


So from that point of view, it's something, even in English,


when we use things like BERT to understand the pages


and the queries a little bit better,


it's not that you'd have to do anything particularly different


with regards to your pages.


But rather, we're just trying to understand your pages better.


We're trying to understand which queries better


fit your pages, especially things where the queries,


for example, might be kind of complicated,


where people are asking things that we can't map one-to-one


on a keyword basis, where we have to understand what it


is that they actually mean.


So from that point of view, it's something


where when we do use BERT, it's not


that you need to change anything on your website


when we don't use BERT.


It's also not something that you need to do anything


different on your website.


So when we kind of try to expand our understanding of queries


in English, often that's kind of one of the places where


we start, because we have a lot of people that are active


in English.


And also, there are lots of research


that's being done in English.


Maybe we'd start there.


But we do try to apply this technology to other languages.


And for other languages, for other language content,


it essentially means that we have to do a lot of research


to try to make sure that we're understanding things correctly.


We have to do a lot of testing to make sure


that we're actually improving things overall


and that we're not causing more confusion in the search


results.


That's something that in English, we


did spend a lot of time to really test these algorithms,


to try to fine-tune them.


So that's something where I could imagine it just


takes a lot longer for us to get that done in a way that


is easily transferable to other languages.


And then even there, we'd still have


to test that significantly.


So my guess is over time, we will


learn to understand a lot more content, a lot better,


understand queries a lot better.


But there is nothing specific that you


need to do on your pages to make them compatible with BERT


or to make them work better with BERT.


But rather, if you're creating good content,


if you're creating content in a way that


is easily understandable to users,


then that's what we're looking for.


We're not looking for new meta tags


to try to find things a little bit better,


but rather, we're trying to understand your content


a lot better.


Let's see.


We're kind of out of time.


There's still a few questions left,


but it feels like these are kind of tricky ones that


need a lot of time to kind of go through.


Maybe I'll just switch over to more questions from you


all if there's anything specifically on your mind


that you'd like to talk about.


I've got a question, John, if you have the time.


So last time we met, I mentioned about a website


where certain commercial pages were doing really bad,


average position 70 to 80.


So worse than it would be just competition there.


Looks like there might be something else going on.


You mentioned, let's just try switching the URL,


see what happens, see if it kind of wiggles it around.


So we did that.


But the issue is the old URL is now 404,


and we did a new URL, which is the exact same content


for that page.


It's just a different URL.


But when I go to Search Console and do a URL inspection


on the old URL, it doesn't show it as a 404.


It shows it as Google selected canonical to the new URL.


So I think that kind of defeats the whole purpose of switching


it and 404ing the old one, because Google doesn't


consider it 404.


It just migrates all of the ranking signals


and possible issues to the new URL as well.


So it doesn't really validate the experiment, I guess.


I totally don't remember what the context was.


So it's hard for me to say.


But so you're not seeing any positive effects in short?


No.


But one of the reasons that I think I'm not


is because Google doesn't see the old URL as a 404,


and the new URL as a brand URL.


It just sees the old URL as canonical to the new URL.


And I don't know how to make it kind of see


that that one is a 404.


Using the test live URL, it does show a 404.