Let’s talk sitemaps

Did it have the word tool or console?


Let’s talk sitemaps

I think it was called Google Sitemaps.


Let’s talk sitemaps

And people just knew that this was a thing that you could use?


Let’s talk sitemaps

You didn't need the word tool in the name?


Let’s talk sitemaps

You know how we are really good at picking


Let’s talk sitemaps

terms that are ambiguous?


Let’s talk sitemaps

Oh, yeah.


Let’s talk sitemaps

OK.


Let’s talk sitemaps

So Google Sitemaps.


Let’s talk sitemaps

Excellent.


Let’s talk sitemaps

Excellent name for many things.


Let’s talk sitemaps

The tool, probably also the docs, and the help group.


Let’s talk sitemaps

Yeah, the help group was the other thing


Let’s talk sitemaps

that came out at the time because it was positioned


Let’s talk sitemaps

kind of as a beta for site owners.


Let’s talk sitemaps

And they wanted to get their feedback, I guess.


Let’s talk sitemaps

So they created a help group for site owners,


Let’s talk sitemaps

specifically around sitemaps.


Let’s talk sitemaps

And I got involved in that at the time as well.


Let’s talk sitemaps

Kind of helping people with their.


Let’s talk sitemaps

Like pre-Google?


Let’s talk sitemaps

Yeah, pre-Google helping people to figure things out.


Let’s talk sitemaps

Oh, you were a Bionic poster, right?


Let’s talk sitemaps

That was, I think, before that.


Let’s talk sitemaps

And at some point, it migrated from being


Let’s talk sitemaps

a group about sitemaps to being the Webmaster Help


Let’s talk sitemaps

Group or something like that.


Let’s talk sitemaps

That was pretty fun.


Let’s talk sitemaps

And I guess in the early days, there


Let’s talk sitemaps

wasn't a ton of documentation from Google's side


Let’s talk sitemaps

about how to make websites.


Let’s talk sitemaps

So there's lots of guessing and people trying to make tests.


Let’s talk sitemaps

It was interesting.


Let’s talk sitemaps

Did it just start out with how to use sitemaps


Let’s talk sitemaps

and then kind of grow from there?


Let’s talk sitemaps

So I think the main problem there


Let’s talk sitemaps

was because there was no other Google official discussion


Let’s talk sitemaps

forum for these kind of SEO topics,


Let’s talk sitemaps

everyone went to the sitemaps group and was like,


Let’s talk sitemaps

why is my website not being indexed?


Let’s talk sitemaps

And luckily, we solved that problem.


Let’s talk sitemaps

Right, Gary?


Let’s talk sitemaps

I don't want to talk about it.


Let’s talk sitemaps

It's still a trauma for me.


Let’s talk sitemaps

I still have PTSD.


Let’s talk sitemaps

I mean, other parts of Google or other search engines of Google,


Let’s talk sitemaps

like Google News, they had similar problems.


Let’s talk sitemaps

They didn't have documentation, or the documentation


Let’s talk sitemaps

was not great.


Let’s talk sitemaps

And that's how I got involved in new sitemaps as well.


Let’s talk sitemaps

Because I don't know if you know,


Let’s talk sitemaps

but sitemaps can have extensions because it's


Let’s talk sitemaps

an XML file and it's extensible.


Let’s talk sitemaps

Oh, wait.


Let’s talk sitemaps

So new sitemap is different from a normal sitemap?


Let’s talk sitemaps

I thought it was just smaller.


Let’s talk sitemaps

What?


Let’s talk sitemaps

No?


Let’s talk sitemaps

I thought there was just a limit of the number of pages


Let’s talk sitemaps

that you could include.


Let’s talk sitemaps

Wait, you know the answer to this?


Let’s talk sitemaps

We've been trying to track down why there's a discrepancy.


Let’s talk sitemaps

We thought maybe there's a discrepancy.


Let’s talk sitemaps

There is a discrepancy or not?


Let’s talk sitemaps

We don't know.


Let’s talk sitemaps

Now we're trying to find out.


Let’s talk sitemaps

No, I'm asking John.


Let’s talk sitemaps

Yeah, John.


Let’s talk sitemaps

I was never involved with the new side of things.


Let’s talk sitemaps

You would know more.


Let’s talk sitemaps

Well, you seem to know something.


Let’s talk sitemaps

Yeah.


Let’s talk sitemaps

I just know it's like smaller file.


Let’s talk sitemaps

Maybe I knew more about this in the past,


Let’s talk sitemaps

and you're kind of making me worry


Let’s talk sitemaps

that I'm forgetting things.


Let’s talk sitemaps

But I don't really know the details


Let’s talk sitemaps

of what otherwise is kind of special around new sitemaps.


Let’s talk sitemaps

So let's go back to sitemap extensions


Let’s talk sitemaps

because those are one of the exciting things


Let’s talk sitemaps

that you can do with sitemaps.


Let’s talk sitemaps

Basically, you have the base sitemap,


Let’s talk sitemaps

and then you can extend it with a new namespace, like XML


Let’s talk sitemaps

namespace.


Let’s talk sitemaps

And then it becomes an image sitemap or a video sitemap


Let’s talk sitemaps

or a new sitemap.


Let’s talk sitemaps

And I'm very certain that there are a bunch of more different


Let’s talk sitemaps

sitemaps as well that we didn't talk about.


Let’s talk sitemaps

But those seem to have been very popular also in the earlier


Let’s talk sitemaps

days of sitemaps.


Let’s talk sitemaps

I think video sitemaps, for example,


Let’s talk sitemaps

came to be around 2008, 2009 when Universal Search was


Let’s talk sitemaps

launched, and then video became more prominent on search


Let’s talk sitemaps

result pages.


Let’s talk sitemaps

And then we started adding, because that


Let’s talk sitemaps

was a Google thing, like it was a Google sitemap extension,


Let’s talk sitemaps

we could just add tags to it whenever we wanted,


Let’s talk sitemaps

which I can't decide if it was a good thing or a bad thing.


Let’s talk sitemaps

Definitely good.


Let’s talk sitemaps

OK, it was a good thing.


Let’s talk sitemaps

Now it's a bad thing.


Let’s talk sitemaps

Why is it a bad thing?


Let’s talk sitemaps

Well, you would know because you maintain our documentation.


Let’s talk sitemaps

And some of these sitemap extensions


Let’s talk sitemaps

have these kilometer-long, oh, I'm sorry,


Let’s talk sitemaps

half-mile-long tables with tags and attributes.


Let’s talk sitemaps

Yes, I do know about that.


Let’s talk sitemaps

Do you think that they need to be that long?


Let’s talk sitemaps

I'm fairly certain that they shouldn't be that long.


Let’s talk sitemaps

What makes you say that?


Let’s talk sitemaps

My assumptions.


Let’s talk sitemaps

You just have a gut feeling things should be shorter,


Let’s talk sitemaps

more succinct.


Let’s talk sitemaps

And if they are too long?


Let’s talk sitemaps

I think it would be worth looking


Let’s talk sitemaps

into those tags and attributes and see


Let’s talk sitemaps

if they are still useful.


Let’s talk sitemaps

Because some of them have been replaced, or not replaced,


Let’s talk sitemaps

complemented, let's say, with schema organizations,


Let’s talk sitemaps

like Schema, what's it called?


Let’s talk sitemaps

What's the name?


Let’s talk sitemaps

Structured data.


Let’s talk sitemaps

Structured data, markup, schema, these


Let’s talk sitemaps

are all fair words to be using.


Let’s talk sitemaps

So some of those things, some of those tags and attributes


Let’s talk sitemaps

have a structured data counterpart.


Let’s talk sitemaps

And if they have a structured data counterpart,


Let’s talk sitemaps

then maybe it's better to supply them in the structured data.


Let’s talk sitemaps

Because then all parsing is in one place instead


Let’s talk sitemaps

of two different places.


Let’s talk sitemaps

Because usually when you have it,


Let’s talk sitemaps

well, usually when you have it in two different places,


Let’s talk sitemaps

then sometimes you might end up with conflicts.


Let’s talk sitemaps

Like, for example, the sitemap is generated offline,


Let’s talk sitemaps

not when the page is rendered.


Let’s talk sitemaps

So it might have a different value for a tag.


Let’s talk sitemaps

But technically, the structured data on the page


Let’s talk sitemaps

should always be the up-to-date version, I guess.


Let’s talk sitemaps

That makes sense.


Let’s talk sitemaps

Because that happens when you actually render the page


Let’s talk sitemaps

or pull the data from your database for the page.


Let’s talk sitemaps

So maybe some of those tags could go.


Let’s talk sitemaps

So maybe we should go check with the video team


Let’s talk sitemaps

and see if all they need is the structured data


Let’s talk sitemaps

and see if there's some tidying that we could


Let’s talk sitemaps

do in the sitemap extensions.


Let’s talk sitemaps

I mean, video team, the Google Images team, and probably also


Let’s talk sitemaps

the sitemaps team, because we also


Let’s talk sitemaps

have to figure out how, if at all is possible,


Let’s talk sitemaps

to their code to deprecate these tags.


Let’s talk sitemaps

I imagine that's somewhat of a longer process anyway.


Let’s talk sitemaps

Especially if I imagine it won't be that the sitemap file will


Let’s talk sitemaps

stop working.


Let’s talk sitemaps

It's just like we will primarily pull the data


Let’s talk sitemaps

from the markup on the page.


Let’s talk sitemaps

And then it's more like we work together with the team,


Let’s talk sitemaps

and then we work together with the ecosystem


Let’s talk sitemaps

to let them know about the change


Let’s talk sitemaps

early on so that they can update if they want to.


Let’s talk sitemaps

Because I imagine a lot of the sitemap generators out there,


Let’s talk sitemaps

they haven't been touched in many years


Let’s talk sitemaps

because they just work.


Let’s talk sitemaps

And it's like, why would you change it if it's working?


Let’s talk sitemaps

What do you think about the future of sitemaps?


Let’s talk sitemaps

I don't know how Jason feels about that.


Let’s talk sitemaps

Not that Jason.


Let’s talk sitemaps

Oh.


Let’s talk sitemaps

Wrong Jason.


Let’s talk sitemaps

I don't know.


Let’s talk sitemaps

I kind of feel, on the one hand, XML


Let’s talk sitemaps

is a really ancient format.


Let’s talk sitemaps

So it's kind of weird to keep using,


Let’s talk sitemaps

but it kind of just works for this purpose.


Let’s talk sitemaps

So especially about informing search engines or anyone


Let’s talk sitemaps

who's interested in a website what


Let’s talk sitemaps

the pages are on the website, I don't


Let’s talk sitemaps

know what the future will be.


Let’s talk sitemaps

There's the initiative, I think, from Bing


Let’s talk sitemaps

and some other search engines about index now,


Let’s talk sitemaps

where you submit individual pages.


Let’s talk sitemaps

There's the indexing API from our side, where you also


Let’s talk sitemaps

submit individual pages.


Let’s talk sitemaps

Maybe at some point, things will transition in that direction.


Let’s talk sitemaps

But I don't know.


Let’s talk sitemaps

I still kind of find the process of crawling websites


Let’s talk sitemaps

useful to understand the websites a little bit better.


Let’s talk sitemaps

So I don't want to move to a model of people


Let’s talk sitemaps

don't understand what their website is actually


Let’s talk sitemaps

like when it's crawled.


Let’s talk sitemaps

And they just submit pages whenever they think,


Let’s talk sitemaps

like, this page is interesting.


Let’s talk sitemaps

It should still be something that is crawlable.


Let’s talk sitemaps

And that kind of maps to what users see as well,


Let’s talk sitemaps

because if a website is crawlable,


Let’s talk sitemaps

then users can also click around and find the content.


Let’s talk sitemaps

And that's ultimately kind of the important part.


Let’s talk sitemaps

You guide people to part of a website,


Let’s talk sitemaps

and they should be able to dig deeper from there


Let’s talk sitemaps

and find out more.


Let’s talk sitemaps

So Gary, what do you think about the future of sitemaps?


Let’s talk sitemaps

GARY I'm very fond of sitemaps.


Let’s talk sitemaps

But I also want to see things evolve a little.


Let’s talk sitemaps

But I also don't like JSON, because JSON is weird.


Let’s talk sitemaps

I think there are two possible directions that could happen.


Let’s talk sitemaps

On the one hand, you could just submit


Let’s talk sitemaps

a text file of all of the URLs from your site,


Let’s talk sitemaps

where basically we say, well, all of these attributes


Let’s talk sitemaps

haven't been that super useful.


Let’s talk sitemaps

You should just give us a list of the URLs.


Let’s talk sitemaps

And that might be one approach.


Let’s talk sitemaps

The other approach that I don't know


Let’s talk sitemaps

if Indexed Now uses this or Indexing API


Let’s talk sitemaps

could be where you actually submit the pages themselves


Let’s talk sitemaps

kind of directly.


Let’s talk sitemaps

So it's not that search engines would


Let’s talk sitemaps

have to crawl your web page to find the information there.


Let’s talk sitemaps

But rather, that the information is together


Let’s talk sitemaps

with the submission.


Let’s talk sitemaps

My feeling is that will be trickier, because people,


Let’s talk sitemaps

I don't know, it adds an extra layer of complexity.


Let’s talk sitemaps

And it makes it so that it's easier to get those two


Let’s talk sitemaps

sides out of sync.


Let’s talk sitemaps

You submit something to a search engine,


Let’s talk sitemaps

and you have something different on your website


Let’s talk sitemaps

for accidental reasons or for spammy reasons or whatever.


Let’s talk sitemaps

But kind of that disconnect feels kind of tricky.


Let’s talk sitemaps

Tricky for them or tricky for search engines?


Let’s talk sitemaps

I think both sides, because a search engine or anyone who's


Let’s talk sitemaps

kind of consuming this still has to look at the pages


Let’s talk sitemaps

to confirm that actually this is reasonable.


Let’s talk sitemaps

And at that point, you're crawling the page.


Let’s talk sitemaps

So what is the difference?


Let’s talk sitemaps

We do get a lot of people writing


Let’s talk sitemaps

in that seem to think that this is how it should work,


Let’s talk sitemaps

that they should just be able to send us this URL


Let’s talk sitemaps

that we haven't indexed.


Let’s talk sitemaps

And there should just be a box for them to be like, upload.


Let’s talk sitemaps

Here's this URL.


Let’s talk sitemaps

Google, please know about it.


Let’s talk sitemaps

But it seems like it's more complex than that,


Let’s talk sitemaps

that they might not know that all these other things


Let’s talk sitemaps

that they should be thinking about.


Let’s talk sitemaps

I mean, that could be a good topic for a future episode,


Let’s talk sitemaps

where we talk about what gets into our crawl queues


Let’s talk sitemaps

and what does not, because it is way more complicated than just


Let’s talk sitemaps

submitting a sitemap.


Let’s talk sitemaps

Basically, with a sitemap, you are just telling search engines,


Let’s talk sitemaps

any search engine, that your URLs are here.


Let’s talk sitemaps

You do whatever you want with them.


Let’s talk sitemaps

You're not instructing that you want these crawled.


Let’s talk sitemaps

Or not crawled.


Let’s talk sitemaps

Well, with sitemaps, you cannot say not crawled.


Let’s talk sitemaps

You use robots.txt for that, right?


Let’s talk sitemaps

Oh, look, we made a complete loop.


Let’s talk sitemaps

Full circle.


Let’s talk sitemaps

And that's it for this episode.


Let’s talk sitemaps

Thanks for joining us here, folks.


Let’s talk sitemaps

Next time on Search Off the Record,


Let’s talk sitemaps

we'll be talking about the future of the web


Let’s talk sitemaps

with Alan Kent.


Let’s talk sitemaps

We've been having fun with these episodes,


Let’s talk sitemaps

and I hope you, the listener, have


Let’s talk sitemaps

found them both entertaining and insightful as well.


Let’s talk sitemaps

Feel free to drop me a note on Twitter


Let’s talk sitemaps

or chat with us at one of the next virtual or in-person


Let’s talk sitemaps

events that we go to if you have any thoughts.


Let’s talk sitemaps

And of course, don't forget to like and subscribe.


Let’s talk sitemaps

Thank you, and goodbye.


Let’s talk sitemaps

SPEAKER 2. Bye now.


Let’s talk sitemaps

SPEAKER 3. Goodbye.


Let’s talk sitemaps


How I learned to debug troubled titles

Oh, look at that.


How I learned to debug troubled titles

That's amazing.


How I learned to debug troubled titles

That is awesome.


How I learned to debug troubled titles

Wow.


How I learned to debug troubled titles

Hello, and welcome to the first episode of SEO Fairy Tales.


How I learned to debug troubled titles

With me today is Luke Davis, a technical SEO specialist


How I learned to debug troubled titles

at Impression.


How I learned to debug troubled titles

They also love to play Pokemon in their free time,


How I learned to debug troubled titles

and no pie to the 11th decimal?


How I learned to debug troubled titles

Yes, that's right.


How I learned to debug troubled titles


How I learned to debug troubled titles

That's very specific.


How I learned to debug troubled titles

How did that happen?


How I learned to debug troubled titles

I didn't have a calculator at the time in class,


How I learned to debug troubled titles

so I had to kind of know the number to that length


How I learned to debug troubled titles

to get an accurate result.


How I learned to debug troubled titles

But 11, that's a very specific number.


How I learned to debug troubled titles

It's like probably 10th, and then you learn the 11th because?


How I learned to debug troubled titles

Yeah, rounding your sheet out.


How I learned to debug troubled titles

Same for me.


How I learned to debug troubled titles

I think I can do four, and then I learned the fifth eventually,


How I learned to debug troubled titles

just accidentally.


How I learned to debug troubled titles

But 11, that's very, very cool.


How I learned to debug troubled titles

You brought something really cool to me today as well, right?


How I learned to debug troubled titles

You have a really interesting case study


How I learned to debug troubled titles

of a technical SEO challenge that you were looking


How I learned to debug troubled titles

into at some point, don't you?


How I learned to debug troubled titles

Yes, I do.


How I learned to debug troubled titles

So I had a site where I did a crawl,


How I learned to debug troubled titles

and there were some issues with duplicate titles.


How I learned to debug troubled titles

And then looking in Google search,


How I learned to debug troubled titles

I realized that Google was swapping out


How I learned to debug troubled titles

the titles for the H1s.


How I learned to debug troubled titles

So I realized that there was clearly an issue,


How I learned to debug troubled titles

and then dug a little deeper just


How I learned to debug troubled titles

to find out why that was happening.


How I learned to debug troubled titles

Interesting.


How I learned to debug troubled titles

So you said you ran a crawl.


How I learned to debug troubled titles

Which tool did you use to do that?


How I learned to debug troubled titles

Do you do that regularly?


How I learned to debug troubled titles

How does that work?


How I learned to debug troubled titles

Yeah, so every three months, I used deep crawl to do that.


How I learned to debug troubled titles

And then it kind of triggered some issues,


How I learned to debug troubled titles

and then kind of looked into it.


How I learned to debug troubled titles

And there you noticed that you have duplicate titles.


How I learned to debug troubled titles

Yes.


How I learned to debug troubled titles

And then you said you checked in Google search.


How I learned to debug troubled titles

I guess you just searched for something


How I learned to debug troubled titles

that the site was usually showing on, or how did that go?


How I learned to debug troubled titles

Yes, I kind of looked at some of the pages that were there,


How I learned to debug troubled titles

and kind of did a site search, and looked at some of them.


How I learned to debug troubled titles

And yeah, realized that with an extension


How I learned to debug troubled titles

that some of the titles were being swapped out,


How I learned to debug troubled titles

and they're all the same.


How I learned to debug troubled titles

So it was clearly an issue.


How I learned to debug troubled titles

Yeah, OK.


How I learned to debug troubled titles

So you don't want the same title for all the pages,


How I learned to debug troubled titles

because that just looks bad in search results.


How I learned to debug troubled titles

Exactly, yeah.


How I learned to debug troubled titles


How I learned to debug troubled titles

All right, so how did you go about that?


How I learned to debug troubled titles

Because I think if I were seeing that,


How I learned to debug troubled titles

I can already see a bunch of places


How I learned to debug troubled titles

where things could go wrong.


How I learned to debug troubled titles

So how did you find out what was going on then?


How I learned to debug troubled titles

So I think during a Lighthouse test,


How I learned to debug troubled titles

just kind of checking general pay speed issues, there was.


How I learned to debug troubled titles

Right, Lighthouse.


How I learned to debug troubled titles

So just real quick, Lighthouse is,


How I learned to debug troubled titles

you used the one in Chrome Developer Tools,


How I learned to debug troubled titles

or where did you run Lighthouse?


How I learned to debug troubled titles

Off the web.dev site.


How I learned to debug troubled titles

Ah, OK.


How I learned to debug troubled titles

So cool, yeah.


How I learned to debug troubled titles

Yeah, and then found, while just looking at the page itself,


How I learned to debug troubled titles

that the title was loading with kind of the site name first.


How I learned to debug troubled titles

And then after page load, it was switching to the correct title.


How I learned to debug troubled titles

So I was like, there's clearly something wrong here.


How I learned to debug troubled titles

Ah, that smells like JavaScript was in the mix, wasn't it?


How I learned to debug troubled titles

Yes.


How I learned to debug troubled titles

All right, OK.


How I learned to debug troubled titles

So you crawled the site with deep crawl,


How I learned to debug troubled titles

found that all the titles were the same,


How I learned to debug troubled titles

then looked at Lighthouse using the web.dev version.


How I learned to debug troubled titles

But you would have to do that for a bunch of pages.


How I learned to debug troubled titles

So was it the same problem for all the pages, I guess?


How I learned to debug troubled titles

Or did you just sample a few?


How I learned to debug troubled titles

Just sampled a few that attempt to call a pull out.


How I learned to debug troubled titles

It all showed the same behavior?


How I learned to debug troubled titles

OK, cool.


How I learned to debug troubled titles

So now we know that all the pages probably


How I learned to debug troubled titles

show the same behavior, and that it happens,


How I learned to debug troubled titles

like the switch of the titles happens after page load.


How I learned to debug troubled titles

How did you then go further?


How I learned to debug troubled titles

Did you have to talk to developers at this point,


How I learned to debug troubled titles

or was there something else you could do before?


How I learned to debug troubled titles

So I asked a colleague just for some advice,


How I learned to debug troubled titles

because it's something I hadn't seen before,


How I learned to debug troubled titles

where a title could change in page load.


How I learned to debug troubled titles

Because it wasn't there in the original HTML?


How I learned to debug troubled titles

Or what tipped you off to a job, or your coworker off


How I learned to debug troubled titles

to a JavaScript issue?


How I learned to debug troubled titles

Did they explain why they think it's JavaScript?


How I learned to debug troubled titles

So I think it was something between experience and, yeah,


How I learned to debug troubled titles

kind of understanding that the page was doing.


How I learned to debug troubled titles

Right.


How I learned to debug troubled titles

So to make sure that I understand correctly,


How I learned to debug troubled titles

basically the problem was that the initial HTML that


How I learned to debug troubled titles

was sent by the server did not contain the proper title


How I learned to debug troubled titles

at all.


How I learned to debug troubled titles

It had a default title left in there.


How I learned to debug troubled titles

Yeah, that's right.


How I learned to debug troubled titles

Oh, god.


How I learned to debug troubled titles

OK, so this is something that I am really


How I learned to debug troubled titles

happy that you bring this to our show, because this is something


How I learned to debug troubled titles

that is probably the first JavaScript issue that I


How I learned to debug troubled titles

encountered as a Googler in 2018 when I was becoming more


How I learned to debug troubled titles

or less like the JavaScript person in the team of Search


How I learned to debug troubled titles

Relations.


How I learned to debug troubled titles

The reason is probably because the developers did not


How I learned to debug troubled titles

think this through.


How I learned to debug troubled titles

A lot of the tutorials out there that teaches developers


How I learned to debug troubled titles

how to build applications using JavaScript or websites using


How I learned to debug troubled titles

JavaScript, for that matter, is basically


How I learned to debug troubled titles

ignoring it, neglecting the initial HTML.


How I learned to debug troubled titles

It's just like, yeah, you need a head.


How I learned to debug troubled titles

And if you have a head, you might as well


How I learned to debug troubled titles

throw a title in, and then you have a body,


How I learned to debug troubled titles

and that's where a placeholder is.


How I learned to debug troubled titles

And then only then the JavaScript eventually


How I learned to debug troubled titles

populates, which, to be honest, works perfectly fine


How I learned to debug troubled titles

in the browser on your machine, on my machine.


How I learned to debug troubled titles

Doesn't work so well for Google Search.


How I learned to debug troubled titles

So having established that it is probably JavaScript,


How I learned to debug troubled titles

what did you do to confirm that theory?


How I learned to debug troubled titles

I mean, at this point, it's just a hypothesis, right?


How I learned to debug troubled titles

JINGLE BELLS


How I learned to debug troubled titles

Yeah.


How I learned to debug troubled titles

Speaking to the dev, we had a kind of hypothesis


How I learned to debug troubled titles

of it being related to rendering.


How I learned to debug troubled titles

They confirmed that it was using client-side rather than


How I learned to debug troubled titles

server-side.


How I learned to debug troubled titles

And then we made our recommendations


How I learned to debug troubled titles

after some further consultation with colleagues.


How I learned to debug troubled titles

So how did that happen, the conversation


How I learned to debug troubled titles

with the developers, because I know that developers are


How I learned to debug troubled titles

sometimes not enthusiastic about SEOs coming


How I learned to debug troubled titles

with technical recommendations.


How I learned to debug troubled titles

How did you make sure that they were listening


How I learned to debug troubled titles

to what you were saying and how you were making sure


How I learned to debug troubled titles

that they were not defensive immediately?


How I learned to debug troubled titles

Or were they defensive?


How I learned to debug troubled titles

No, I was fortunate that that wasn't the case.


How I learned to debug troubled titles

I kind of outlined as much as I could


How I learned to debug troubled titles

without kind of waffling in the email,


How I learned to debug troubled titles

explaining what the issue was and what was happening,


How I learned to debug troubled titles

offering recommendations and the pros and cons of each


How I learned to debug troubled titles

just to make sure that everyone, including the client


How I learned to debug troubled titles

who was in the email, was on the same page.


How I learned to debug troubled titles

Interesting.


How I learned to debug troubled titles

What were your recommendations, if you recall?


How I learned to debug troubled titles

So I think the default or fallback


How I learned to debug troubled titles

was to recommend server-side and then explaining why


How I learned to debug troubled titles

and why client-side was an issue for SEO specifically.


How I learned to debug troubled titles

And then offering kind of a if in an ideal world


How I learned to debug troubled titles

you could use dynamic, and then explaining


How I learned to debug troubled titles

all the different options there.


How I learned to debug troubled titles

But server-side was at least the default we wanted to go for.


How I learned to debug troubled titles

Right.


How I learned to debug troubled titles

Coming from a developer's background,


How I learned to debug troubled titles

I know that server-side is also the most tricky to implement.


How I learned to debug troubled titles

So I'm guessing that was not accepted


How I learned to debug troubled titles

without any resistance.


How I learned to debug troubled titles

There must have been some resistance.


How I learned to debug troubled titles

Or was it like, all right, we can do that?


How I learned to debug troubled titles

Yeah, more or less.


How I learned to debug troubled titles

I mean, I basically suggested that and made


How I learned to debug troubled titles

aware that there was some financial cost and time


How I learned to debug troubled titles

building.


How I learned to debug troubled titles

So I was aware so they knew that was understood.


How I learned to debug troubled titles

And then I was like, oh, I'm going to do that.


How I learned to debug troubled titles

You said you explained the different options


How I learned to debug troubled titles

and you explained dynamic rendering and server-side


How I learned to debug troubled titles

rendering.


How I learned to debug troubled titles

What were these options?


How I learned to debug troubled titles

And how did you come up with server-side rendering


How I learned to debug troubled titles

as the default?


How I learned to debug troubled titles

What made you choose that?


How I learned to debug troubled titles

Make sure of speaking to colleagues who had


How I learned to debug troubled titles

experience with the two.


How I learned to debug troubled titles

And what did they tell you from their experience?


How I learned to debug troubled titles

I'm curious.


How I learned to debug troubled titles

I think they'd had some issues with client-side previously.


How I learned to debug troubled titles

And then were aware that server-side was the best


How I learned to debug troubled titles

case for SEO, at least.


How I learned to debug troubled titles

And after looking up some tutorials and general practice


How I learned to debug troubled titles

and so on, that was the main answer to the question.


How I learned to debug troubled titles

OK, interesting.


How I learned to debug troubled titles

And why did you not take dynamic rendering, for instance,


How I learned to debug troubled titles

as a route?


How I learned to debug troubled titles

I offered it.


How I learned to debug troubled titles

But then I was aware that it could use third party.


How I learned to debug troubled titles

And then the cost of having to use that.


How I learned to debug troubled titles

And I wasn't naive to the fact that more cost is.


How I learned to debug troubled titles

There's Rendertron.


How I learned to debug troubled titles

Rendertron is free.


How I learned to debug troubled titles

Yeah, OK.


How I learned to debug troubled titles

But I just want to make sure that it's used properly.


How I learned to debug troubled titles

Yeah.


How I learned to debug troubled titles

No, I think that's a very, really good point.


How I learned to debug troubled titles

And it's a point I would rather have us reiterate,


How I learned to debug troubled titles

because I think it's important.


How I learned to debug troubled titles

Dynamic rendering has a lot of drawbacks on top of it.


How I learned to debug troubled titles

We considered kind of deprecated at this point


How I learned to debug troubled titles

and a kind of stopgap solution for those


How I learned to debug troubled titles

who can't do server-side rendering,


How I learned to debug troubled titles

because it's the better choice.


How I learned to debug troubled titles

And even if you use a free tool like Rendertron, which


How I learned to debug troubled titles

you can run on your own infrastructure,


How I learned to debug troubled titles

you are then running it on your own infrastructure.


How I learned to debug troubled titles

Your own infrastructure is not free either.


How I learned to debug troubled titles

So you always are spending some amount of money and maintenance


How I learned to debug troubled titles

on this additional infrastructure


How I learned to debug troubled titles

that you have to operate.


How I learned to debug troubled titles

And also on top of that, it adds complexity,


How I learned to debug troubled titles

because then how you test these things,


How I learned to debug troubled titles

you then have to test specifically for Googlebot


How I learned to debug troubled titles

and any other crawlers that you set up in Rendertron.


How I learned to debug troubled titles

And yeah.


How I learned to debug troubled titles

When was this issue?


How I learned to debug troubled titles

How long ago was that?


How I learned to debug troubled titles

Because you said they had problems with client-side.


How I learned to debug troubled titles

And I know that we have increased our client-side


How I learned to debug troubled titles

rendering abilities a lot in the last couple of years.


How I learned to debug troubled titles

This is the end of last year, I believe.


How I learned to debug troubled titles

Whoa.


How I learned to debug troubled titles

OK, that comes as a surprise.


How I learned to debug troubled titles

That's interesting.


How I learned to debug troubled titles

I did not anticipate that.


How I learned to debug troubled titles

So last year, end of last year, you were running into it.


How I learned to debug troubled titles

How long has it been?


How I learned to debug troubled titles

Everything depends on the stack, on the setup,


How I learned to debug troubled titles

on the infrastructure.


How I learned to debug troubled titles

If Google picks it up or not is a tricky thing.


How I learned to debug troubled titles

I think generally server-side rendering


How I learned to debug troubled titles

has a big advantage.


How I learned to debug troubled titles

And it's faster for the user, usually, if it's done right.


How I learned to debug troubled titles

But it's the same thing if it's done right.


How I learned to debug troubled titles

So did your coworkers give you any guidance


How I learned to debug troubled titles

on how to do it right?


How I learned to debug troubled titles

Because I guess developers might not necessarily


How I learned to debug troubled titles

be familiar with how to set up server-side rendering properly.


How I learned to debug troubled titles

So they didn't.


How I learned to debug troubled titles

But fortunately, the dev was kind of familiar


How I learned to debug troubled titles

with that kind of switch.


How I learned to debug troubled titles

So they managed to put that in place.


How I learned to debug troubled titles

And yeah, that worked.


How I learned to debug troubled titles

Out of curiosity, because the stack and the technology


How I learned to debug troubled titles

that you use specifically makes a lot of difference


How I learned to debug troubled titles

in how easy or hard it is to set up server-side rendering,


How I learned to debug troubled titles

what technology do you recall them using?


How I learned to debug troubled titles

On Nuxt.


How I learned to debug troubled titles

Nuxt.


How I learned to debug troubled titles

So that's Vue.js plus server-side rendering


How I learned to debug troubled titles

kind of built in.


How I learned to debug troubled titles

You just have to use it.


How I learned to debug troubled titles

So that makes a lot of sense then.


How I learned to debug troubled titles

Because I know that especially with client-side rendered


How I learned to debug troubled titles

applications that you have, like existing client-side


How I learned to debug troubled titles

applications, it's not easy to move them


How I learned to debug troubled titles

to server-side rendering.


How I learned to debug troubled titles

But in this case, it might have been easier


How I learned to debug troubled titles

because the technology kind of features it built in.


How I learned to debug troubled titles

You just need to use the right bits and pieces.


How I learned to debug troubled titles

Cool.


How I learned to debug troubled titles

All right, so did that solve the problem?


How I learned to debug troubled titles

I would say mostly, yeah.


How I learned to debug troubled titles

Mostly?


How I learned to debug troubled titles

Yeah, so there was a very small window where it switched.


How I learned to debug troubled titles

But it was so small, it was almost instant


How I learned to debug troubled titles

if you didn't see it.


How I learned to debug troubled titles

So I believe it's quick enough that Google


How I learned to debug troubled titles

picked that up without it being an issue again.


How I learned to debug troubled titles

Oh, that's amazing.


How I learned to debug troubled titles

Yeah, I mean, to be honest, hypothetically speaking,


How I learned to debug troubled titles

another person running a Nuxt application


How I learned to debug troubled titles

might not even have seen the problem.


How I learned to debug troubled titles

Because even though it does take time until the title switches,


How I learned to debug troubled titles

we might have picked it up in the rendering process anyway.


How I learned to debug troubled titles

But in this case, we didn't.


How I learned to debug troubled titles

So that's an interesting one where that didn't work.


How I learned to debug troubled titles

So how did you check if that worked?


How I learned to debug troubled titles

Did you do just another crawl?


How I learned to debug troubled titles

Or how did you monitor the fix?


How I learned to debug troubled titles

Just loaded a page, refreshed it, and checked it.


How I learned to debug troubled titles

And it works?


How I learned to debug troubled titles

Yeah, yeah.


How I learned to debug troubled titles

It was much quicker then, yeah.


How I learned to debug troubled titles

Excellent.


How I learned to debug troubled titles

That's fantastic.


How I learned to debug troubled titles

And how long did it take for the developers


How I learned to debug troubled titles

to actually implement that change?


How I learned to debug troubled titles

Did it take weeks, months?


How I learned to debug troubled titles

Weeks, I'd say.


How I learned to debug troubled titles

Weeks.


How I learned to debug troubled titles

There were other things in the background


How I learned to debug troubled titles

that needed to be doing outside.


How I learned to debug troubled titles

Did any of the metrics improve?


How I learned to debug troubled titles

Or did you see any outcome from the change,


How I learned to debug troubled titles

besides the titles actually changing?


How I learned to debug troubled titles

Not as yet.


How I learned to debug troubled titles

But I was just glad to see that changed.


How I learned to debug troubled titles

One thing off the list.


How I learned to debug troubled titles

One thing off the list, absolutely.


How I learned to debug troubled titles

Interesting.


How I learned to debug troubled titles

So what did you learn throughout the process there?


How I learned to debug troubled titles

Just going through the general process


How I learned to debug troubled titles

with finding an issue, seeking assistance from colleagues,


How I learned to debug troubled titles

getting more information on what was happening,


How I learned to debug troubled titles

speaking to a dev, and letting them know that I understood


How I learned to debug troubled titles

from their side.


How I learned to debug troubled titles

And then kind of...


How I learned to debug troubled titles

That makes a huge difference.


How I learned to debug troubled titles

Yeah, give the person comments.


How I learned to debug troubled titles

That makes all the difference, absolutely.


How I learned to debug troubled titles

Yeah.


How I learned to debug troubled titles

So toolbox-wise, so you started noticing the issue


How I learned to debug troubled titles

because you crawled the pages regularly.


How I learned to debug troubled titles

Okay, so that's how you detected the problem.


How I learned to debug troubled titles

Why did you choose Lighthouse as your next stop?


How I learned to debug troubled titles

It was just part of the general technical audit that we were doing.


How I learned to debug troubled titles

Right, so you had to do it anyway.


How I learned to debug troubled titles

It's like, ah, clever.


How I learned to debug troubled titles

Okay, that's nice.


How I learned to debug troubled titles

Yeah, I did that.


How I learned to debug troubled titles

And it just happened that while looking that up,


How I learned to debug troubled titles

there was some page speed issues.


How I learned to debug troubled titles

And while looking at the page,


How I learned to debug troubled titles

I caught a glimpse of what was happening in the tab.


How I learned to debug troubled titles

Interesting.


How I learned to debug troubled titles

Regarding Lighthouse, where in Lighthouse,


How I learned to debug troubled titles

because Lighthouse is a big tool and has lots of things,


How I learned to debug troubled titles

especially the web.dev version does a bunch of stuff.


How I learned to debug troubled titles

Where in Lighthouse did you go to see the problem?


How I learned to debug troubled titles

Where did you look in Lighthouse?


How I learned to debug troubled titles

Because it's like a lot to look at.


How I learned to debug troubled titles

I'm trying to remember.


How I learned to debug troubled titles

Want to take a look?


How I learned to debug troubled titles

Yeah.


How I learned to debug troubled titles

All right, let's take a look.


How I learned to debug troubled titles

Ha, that's probably the bunny, huh?


How I learned to debug troubled titles

I didn't know it does that.


How I learned to debug troubled titles

Because I was like, oh, he's leaving our HTTPS and stuff.


How I learned to debug troubled titles

We'll see how that goes.


How I learned to debug troubled titles

And it does that automatically.


How I learned to debug troubled titles

That's nice.


How I learned to debug troubled titles

It's a small thing, but it makes a huge difference.


How I learned to debug troubled titles

Yeah, for sure.


How I learned to debug troubled titles

All right, so I know.


How I learned to debug troubled titles

Yeah, OK, so Lighthouse gives us all the scores.


How I learned to debug troubled titles

But from the scores, I don't know.


How I learned to debug troubled titles

Or was it on the screen?


How I learned to debug troubled titles

No, it's not.


How I learned to debug troubled titles

The title is not visible on the screen.


How I learned to debug troubled titles

Yeah, I think I was looking at this


How I learned to debug troubled titles

and noticing that the page speed was off.


How I learned to debug troubled titles

And then, yeah, something like that.


How I learned to debug troubled titles

So you used the page speed as a marker for a problem


How I learned to debug troubled titles

to be present with the loading itself.


How I learned to debug troubled titles

Yeah, and then testing a page live


How I learned to debug troubled titles

to see that it was loading slowly, looking.


How I learned to debug troubled titles

It's just a visual thing, really,


How I learned to debug troubled titles

because I kind of had it there.


How I learned to debug troubled titles

The tab was kind of loading the title and then shifting.


How I learned to debug troubled titles

And I realized that there was an issue there.


How I learned to debug troubled titles

Hmm, interesting.


How I learned to debug troubled titles

So that's how you debug it in Lighthouse.


How I learned to debug troubled titles

I would not have thought of Lighthouse


How I learned to debug troubled titles

as a way to go through that.


How I learned to debug troubled titles

That's pretty cool.


How I learned to debug troubled titles

All right.


How I learned to debug troubled titles

And then you talked to your peers, basically, about this.


How I learned to debug troubled titles

And what did they bring to you as new insights or evidence?


How I learned to debug troubled titles

What was the thing that you learned from your peers talking


How I learned to debug troubled titles

to them?


How I learned to debug troubled titles

That initially would likely have been a JavaScript issue.


How I learned to debug troubled titles

And then kind of pinning that to rendering


How I learned to debug troubled titles

and then asking me whether it was server side or client side.


How I learned to debug troubled titles

I wasn't immediately sure.


How I learned to debug troubled titles

So I went to the dev and asked, and they confirmed it's client


How I learned to debug troubled titles

side.


How I learned to debug troubled titles

And then they confirmed it's client side.


How I learned to debug troubled titles

Interesting.


How I learned to debug troubled titles

Did you have access to Google Search Console for that site?


How I learned to debug troubled titles

Yes, I did, yeah.


How I learned to debug troubled titles

Would it have been helpful to see what the rendering


How I learned to debug troubled titles

phase was seeing at the time?


How I learned to debug troubled titles

Perhaps, yeah.


How I learned to debug troubled titles

I'm not sort of immediately that knowledgeable with that sort


How I learned to debug troubled titles

of rendering.


How I learned to debug troubled titles

And that's fine.


How I learned to debug troubled titles

That's 99% out there.


How I learned to debug troubled titles

It's just something that I find interesting


How I learned to debug troubled titles

because you went to Lighthouse, which is fine.


How I learned to debug troubled titles

I mean, that's a reasonable thing because you


How I learned to debug troubled titles

had to do it anyway.


How I learned to debug troubled titles

And it gives you all the scores.


How I learned to debug troubled titles

And the screenshots as well are quite nice


How I learned to debug troubled titles

to see how the website renders over time.


How I learned to debug troubled titles

But there's also the URL inspection tool


How I learned to debug troubled titles

that not many people know about can do that.


How I learned to debug troubled titles

Exactly what you just described as like,


How I learned to debug troubled titles

I needed to see what Google was seeing in rendering


How I learned to debug troubled titles

and it was a rendering problem.


How I learned to debug troubled titles

Yeah.


How I learned to debug troubled titles

And that's what I wanted to see.


How I learned to debug troubled titles

If you type in a URL into Google Search Console,


How I learned to debug troubled titles

into the URL inspection tool, you


How I learned to debug troubled titles

can see what we saw when we crawled the page.


How I learned to debug troubled titles

So there you have an overview of all the source code


How I learned to debug troubled titles

that we saw.


How I learned to debug troubled titles

And there we would probably, at this point,


How I learned to debug troubled titles

have seen the title being the default title, which


How I learned to debug troubled titles

is unfortunate because that's what you don't want.


How I learned to debug troubled titles

And then when you load that in your browser


How I learned to debug troubled titles

and you go to the inspect element


How I learned to debug troubled titles

and then you go into the title, you would see the proper title.


How I learned to debug troubled titles

And that would have been your moment where you're like, oh,


How I learned to debug troubled titles

right.


How I learned to debug troubled titles

So the server sends us the default title.


How I learned to debug troubled titles

And unfortunately, that is what Google sees


How I learned to debug troubled titles

because anything that happens after the server has sent


How I learned to debug troubled titles

its HTML, that's JavaScript land.


How I learned to debug troubled titles

Yeah.


How I learned to debug troubled titles

I feel like the fear and loathing in Las Vegas movie,


How I learned to debug troubled titles

this is JavaScript land.


How I learned to debug troubled titles

We can't stop here.


How I learned to debug troubled titles

And then from there, you can also jump into,


How I learned to debug troubled titles

like, why is this not happening?


How I learned to debug troubled titles

And then also try a live test, for instance,


How I learned to debug troubled titles

where we then live crawl again and then render live again.


How I learned to debug troubled titles

And you see if that still is a problem.


How I learned to debug troubled titles

If that still is a problem, then that's the exact steps


How I learned to debug troubled titles

that you want to take.


How I learned to debug troubled titles

Cool.


How I learned to debug troubled titles

So server-side rendering seems to be the way out


How I learned to debug troubled titles

of these kind of problems.


How I learned to debug troubled titles

Excellent.


How I learned to debug troubled titles

Did you have similar problems where


How I learned to debug troubled titles

you applied the same tools and the same process as well?


How I learned to debug troubled titles

No.


How I learned to debug troubled titles

I think this is one of the first cases where this has come up.


How I learned to debug troubled titles

Has been.


How I learned to debug troubled titles

Yeah.


How I learned to debug troubled titles

I think everyone else has been speaking to server-side.


How I learned to debug troubled titles

That's pretty cool.


How I learned to debug troubled titles

So would you say that the experience


How I learned to debug troubled titles

of going through this issue and basically just taking it


How I learned to debug troubled titles

step by step and using different tools to uncover what


How I learned to debug troubled titles

was the root cause of this was an interesting journey that


How I learned to debug troubled titles

helps you debug other problems in the future quicker or better?


How I learned to debug troubled titles

So I think something like this, which I think


How I learned to debug troubled titles

is probably fundamental to our work,


How I learned to debug troubled titles

was really good to kind of understand that and then


How I learned to debug troubled titles

know that if anything even remotely related comes up,


How I learned to debug troubled titles

I know that's the process to go with.


How I learned to debug troubled titles

That's amazing.


How I learned to debug troubled titles

Thank you so, so much for showing me


How I learned to debug troubled titles

how you approached this and how you got through


How I learned to debug troubled titles

to the source of all of this title mess.


How I learned to debug troubled titles

It's really, really interesting to hear that we are clearly


How I learned to debug troubled titles

still not 100% on top of JavaScript.


How I learned to debug troubled titles

But as I said, it depends on a case-by-case basis.


How I learned to debug troubled titles

And I know that having a tool set that you


How I learned to debug troubled titles

can rely on to test these things and then debug these things


How I learned to debug troubled titles

is very, very useful and helpful.


How I learned to debug troubled titles

And also really, really cool that you


How I learned to debug troubled titles

chose server-side rendering as the more sustainable,


How I learned to debug troubled titles

robust solution over other kind of stopgap solutions


How I learned to debug troubled titles

such as dynamic rendering.


How I learned to debug troubled titles

Yeah, for sure.


How I learned to debug troubled titles

Excellent.


How I learned to debug troubled titles

Thank you so much for being here with me today, Luke.


How I learned to debug troubled titles

It has been a huge honor having you.


How I learned to debug troubled titles

Thanks.


How I learned to debug troubled titles

It's my pleasure to be here.


How I learned to debug troubled titles

And for all of you out there, I hope


How I learned to debug troubled titles

this was useful and interesting.


How I learned to debug troubled titles

Let us know in the comments what tools you are using


How I learned to debug troubled titles

or if you had similar problems in the past.


How I learned to debug troubled titles

And if you like the episode, leave us a like


How I learned to debug troubled titles

and maybe subscribe for the upcoming episodes in the future.


How I learned to debug troubled titles

In that case, stay safe, take care, and bye-bye.


How I learned to debug troubled titles

And that is the solution.


How I learned to debug troubled titles

It has to be like the more you have to do the motions.


How I learned to debug troubled titles


English Google Webmaster Central office-hours from Sept 20, 2019

All right, welcome everyone to today's Webmaster Central Office


Hours Hangouts.


My name is John Mueller.


I am a Webmaster Trends Analyst here at Google in Switzerland.


And part of what we do are these hour Hangouts


where master publishers can come in and ask us


questions around their websites and web search,


anything like that.


A bunch of questions were submitted on YouTube already.


So we can go through some of those.


But if any of you have anything on your mind


that you'd like to talk about beforehand,


feel free to jump in.


Hi, John.


Hi.


I would like to ask some questions.


So one of our clients, they are a non-profitable organization.


So they arrange a lot of events, a lot of time.


So when they arrange an event, they


publish it on their blog post, on their blog about the event.


But when the event is finished, they take off


the page from their blog.


So in this case, do you need to create any 301


redirections for those event page?


I think the part that's kind of where they've been missing


something where it would be useful for their website


in general is if there were some kind of persistent landing


page for these kind of events.


So for example, if they do a monthly event,


then having one page for that monthly event,


just kind of as a placeholder, we do this event every month


and like some information about it


so that people can link to some persistent place


and that page can gain value over time.


Whereas if you have just individual event


pages for each date and those turn into 404,


then that's something where all of the links, all of the value


that they collect for those pages,


they kind of disappears again.


So something kind of like a persistent page that says,


our next event is then.


And if you want to keep an archive of the old events,


maybe have a separate archive section,


but kind of have one persistent place for the events.


SPEAKER 2


Then that's something that you might just


want to keep and remove.


But if it's something that happens regularly, then


having a persistent page is fine.


So if you have something like, I don't know, Black Friday


offers, and you have a page that you always


use for Black Friday, then that's


something that might make sense.


Whereas if you have just like one page that says Black Friday


2017, Black Friday 2018, then all of those pages


will kind of have to stand on their own,


and it'll be hard for a new page to kind of gain value there.


SPEAKER 1 The last question.


We have a client.


They build garden sheds, and at the same time,


they build playgrounds.


So they have two type of products.


So what they have decided recently,


they build a new website for the playground product.


And they want to move all the products from the website


to the new playground website.


So they want to redirect all the playground products


to the new domain.


Now, the question is, the rank they have on the playground


related keywords in the old domain,


will it transfer to the new domain with this 301


reduction, or will it take time?


JOHN MUELLER If they're creating a new website that


is a mix of multiple other websites,


then that's something that is not


the same as just moving from one domain to another.


So if, for example, they have all of their content


on one domain, they just want to move to another one,


that's something where a site move, a setup,


or the redirect works really well.


But on the other hand, if they have playgrounds and garden


furniture, and they want to create one new website that


is a mix of both, or they want to take one mixed website


and split it into two separate websites,


then that's something where we essentially have


to re-evaluate the final state.


We can't just say, this plus this equals the new one.


We have to figure out what the right balance is.


SPEAKER 2 Thank you, John.


JOHN MUELLER Sure.


Any other questions before we jump in with the rest?


That's fine, too.


I mean, you're welcome to jump in in between


if anything pops up, or if there's anything that I can


help clarify along the way.


Let's see.


What do we have?


Our news article template features two bylines,


one at the top below the title, which includes only the author


name, and another one on the bottom, which


includes more detailed information about the author.


Is it OK to have two bylines on the page?


Does it create any confusion to the crawler in understanding


who the author is?


So from our point of view, it's fine to have


these bylines on the page.


It's not something where we're explicitly looking for that.


We don't use the authorship markup or anything specific


like that.


It's essentially just a way for you


to tell your users what this page, kind of the background


of that page.


If you're just saying, this is written by these authors,


then that's a good place to put it.


Some people put it on the top.


Some people put it at the bottom.


Some people have both.


I think that's totally up to you.


What's the best place for the schema markup with item prop


author?


Shall we add it to both bylines or just one?


So from my point of view, where you add it


is essentially up to you.


What I would not recommend doing is adding it


in two places on the same page, because that just


makes it a lot harder to maintain properly


and to understand what it is that you're trying


to tell with the markup.


So from that point of view, I would pick one place


and use the markup there and not just put it in two places.


Or if you use JSON-LD for the structure data,


then you could put that somewhere in the head,


for example.


It doesn't need to be exactly in the same place


where the name, for example, would be.


We're an online tour operator and have implemented aggregate


rating for offered accommodations


via schema.org slash hotels and our offered destinations


via schema.org slash web page.


And I think it goes into the change


that we recently had with regards to the review-rich


snippets that we would show in search, where we essentially


made three changes.


On the one hand, we limited the review-rich snippets


to a certain set of types.


And on the other hand, we made it


so that if you're reviewing your own entity


or your own business, then you can't host that


on your own site.


Or rather, if you host it on your own site,


we just wouldn't show that.


So those are kind of the main changes that we did there.


And I think what you're probably running into


is you used to review essentially the web page.


And that's not something that we support anymore.


So from that point of view, that setup


that you have where users can review the page


is not something that, from our point of view,


makes a lot of sense.


We expect that the reviews are about a specific product


or a specific thing that is actually being reviewed,


something that anyone can look at and review


in a comparable way.


So for example, if you, I don't know,


have a car that people are able to review,


then everyone is able to buy that car


and able to review exactly the same product.


Whereas if you have something like the destination page


that you mentioned there, I think


it would be hard for multiple people


to review the same geographic region that someone


could go to and review based on the same criteria.


So probably that wouldn't make that much sense there.


Of course, it's totally fine to show these kind of reviews


on your pages if you think that it makes sense for users.


It's just something that we wouldn't


show in the search results.


Our site got hacked.


The hack got through.


Oh, is there a question?


Sorry.


SPEAKER 2


OK.


Yeah, I recently started with an indexing API for one of my blog


sites or search websites.


But I have an issue with the get metadata


using whatever the API.


And I'm not sure as to what API key I'm supposed to use.


During the app installation, I did create the service account


over to client ID, and all the three authorization


pieces have gone through.


So but still, I'm not able to figure out what API


I'm supposed to use in that.


And also regarding the batch, how


I'm going to process in a batch.


I would check in with one of the Google API's help forums,


because that's something which is kind of hard to explain


just live in a Hangout.


It's almost helpful more to kind of look at the code


that you're actually trying to use.


So for the indexing.


Yeah, go ahead.


Yeah, I did post in the forum, but there


is no specific forum for Google indexing API.


So it's whatever, even if I search,


it is redirecting the web-based API there.


And if I posted there, I didn't get a perfect one.


Also, it was flagged as out of topic in there.


So can you just any reference in exactly where I can post


in that one?


Another idea might be to just post on Stack Overflow,


since this is kind of a generic Google API that


seems like something that someone from Stack Overflow


would also probably be able to help with.


OK.


Yeah.


John.


John, hi.


Hi.


Samos here.


In the last Hangouts, I was asking about the Google


for Jobs indexation.


We are having problems here with our visibility.


We have done a migration, and we sort of drop off


in basically our Google for Jobs traffic.


Now I was wondering, you said, yeah, ping you,


the website, et cetera.


I did that in the comments.


I was wondering if you had any updates.


I don't think I heard back from the team,


but I passed it on to them to look at.


But it might be good to check in with them


again if you aren't seeing any changes.


Usually, they're pretty good at picking these kind of things


up.


But if you're still seeing problems


in that it's not performing the way that you would expect,


then I am happy to update that again.


If you can just send me a quick note, then I won't forget.


OK.


I'll add this in the comments.


Thank you.


Awesome.


Our site got hacked.


The hack got removed, but it left virtual pages


indexed in Google.


How can I de-index those pages?


So sorry to hear that your site got hacked.


That's something that happens to lots of sites.


So it is always a bit of a hassle.


With regards to kind of the hack got removed


and some virtual pages are still indexed in Google,


the first thing that I would double check


is that these pages are actually no longer existing


on your website specifically for Googlebot.


So what we sometimes see is that hackers


create this kind of virtual pages or fake pages


on a website that when you try to go to them manually,


they don't work, or they look like they don't work.


And when you go there with Googlebot


to look at the page itself, then they do show some content.


So for a webmaster, it looks like it's completely cleaned up.


And for Googlebot, we look at it and say, well,


all of this content is still here.


We'll continue to index it.


So that's kind of the first step that I would do.


Check with the testing tools to make sure


that it's really, really removed.


And then the second step with regards


to getting these out of Google, there


are essentially two approaches that you can take.


One is if you have individual pages that are very visible


in your normal search results, then that's


something you could remove with the URL removal


tools in Search Console.


So with those, you can specify a subdirectory.


If these are tied to a specific subdirectory, often they are,


then you can specify a subdirectory


and say, all of the things in the subdirectory I don't


want to have shown in search.


And then for a certain time, I think


it's something like 180 days, we will filter all of those out


from the search results.


For individual URLs, if you can't isolate a subdirectory,


then you can submit them there as well.


It's just a very manual process, and there are some limits.


I think it's something like 1,000 URLs a day


that you can submit there.


So because of those limits, I would


focus on the URLs that are actually


visible for your normal search results.


So if you search for your company name


and you find five hacked pages from your website,


that's something definitely to clean up.


On the other hand, if you search for specifically that hacked


content, where you search for your company name,


and then free download, and whatever hacked content


they had on your pages, and then you find those hacked pages,


then probably that's not as critical,


because normal users wouldn't be searching like that.


They'd be searching for your company name or for your products.


So from that point of view, usually there


is a set of very visible pages that you can remove,


and those are the ones that I would remove.


For pretty much everything else, I


would recommend just making sure that those pages return


a normal 404 status code.


That tells us this page no longer exists.


And the next time we go off and try to crawl those pages,


we'll see these are gone, and we can just drop them


from the search results.


So that means there's no extra work involved for you.


It's kind of cleaned up automatically over time.


It can take a bit of time.


So often it'll take a couple of months, maybe half a year,


for all of these pages to be updated and dropped out.


So it's not something that happens right away.


But again, unless you're explicitly


looking for the hacked content, oftentimes you


don't see those pages.


So those are kind of the two approaches I would make.


And definitely make sure before you go down that path


that these pages really don't exist anymore for Google.


And if they still exist for Google,


then obviously there is still something


on your site that's kind of affected from this old hack.


How do crawlers deal with GDPR cookie bars?


Obviously, no content may be loaded.


Before a user has set an opt-in, we


see many data which are not loaded in Search Console,


and they seem blocked.


Yes, that can be problematic.


So in general, Googlebot does not click through on anything.


So Googlebot will not go to an interstitial and say, I accept.


Instead, Googlebot will try to crawl the page


as it comes to Googlebot.


And it'll try to render the page as it comes to Googlebot.


So in particular, if you have a kind of a banner that


is shown on part of the page, or if you


have a banner that is shown on top of the actual HTML content,


then that's something where we can still


see the actual content.


We can still index the actual content.


So that's less of an issue.


On the other hand, if no content at all is loaded


and it's just this interstitial page,


then we would only see the interstitial page


and think, well, this is what you want to have indexed.


So we'll try to index it for you.


And that's probably not really what you want to have indexed.


The other variation that I sometimes see


is that you redirect to an interstitial URL.


And then from there, when the user has accepted,


you redirect back.


That's equally problematic because the content,


on the one hand, is not there when we look


at that interstitial URL.


And oftentimes, the URL of the page


itself is included in a kind of, I don't know,


a cookie or some kind of a snippet that we can't pick up.


And what happens then is we see all of the pages


on your website redirect to one page.


And we think, well, you're redirecting your whole website


to one single page.


And this single page has some legal information on it.


So maybe you want to replace your whole website


with this one page.


And probably you don't want to do that.


So from that point of view, I would


recommend against doing this kind of redirect thing,


instead showing a banner on top of the actual content


so that users get the banner that you need to have shown


for legal reasons, but that the actual content is still


loaded behind so that search engines can actually


process that.


Of course, the other thing that you might be able to do


is to think about which users actually


need to have this interstitial shown.


And depending on what it is that you need to do,


it might be that you discover that perhaps users in the US


don't need to have this interstitial shown.


I don't know the guidelines in this specific case,


but that's something that you can double check.


Googlebot primarily crawls from the US.


So if there's something that you don't


need to show users in the US, then you


don't need to show that to Googlebot either.


So that might make it a little bit easier.


In some legal cases, it can make it a little bit harder,


where if there are policy reasons or legal reasons


why you can't show any of your content in the US,


then obviously Googlebot can't see that content either.


So that's kind of depending on which case


applies to your specific website,


there are different options there.


The way to kind of test this is ideally


to use the inspect URL tool, where you can


see how we would render a page.


And you can also look at the HTML


that we pick up and use for indexing.


So if you check your pages and you


see the interstitial is being shown in the rendered version,


as long as within the HTML version


the actual content is still shown, that's kind of OK.


Because then we can still pick up the actual content.


On the other hand, if you look at the HTML version


and there is no content at all other than the interstitial,


then essentially we only have the interstitial content


to index.


And that's probably not what you want.


SPEAKER 3 Hey, John.


I have a question.


This is related to the image sitemap.


We have posted our images in Amazon AWS.


So the URL comes as the S3 bucket.


Then as a subdirectory, we have our domain name.


So can I just use these URLs in our image sitemap?


Sure.


That's perfectly fine.


Images don't need to be on the same domain.


So that's like if you're using a CDN like Amazon,


if you're using any other kind of CDN,


you're welcome to host your images there.


The one thing I would kind of caution about, though,


is indexing images takes a lot longer than indexing web pages.


So if the noise, da-da-da.


So the thing with images is images take a lot longer


to be indexed.


And they take a lot longer for redirects to be processed.


So if at any point in the future you


decide to move to a different CDN setup,


you would have to set up redirects from the old URLs


to the new ones.


And for images, processing those takes a lot longer.


So if your website really depends on images,


on image search in particular, then that's


something where I would recommend going down


the path of using a subdomain from your own domain, which


you can then kind of route independently depending


on the CDN that you're using.


So if you use a subdomain, then those image URLs


can remain the same for a longer period of time.


And you can swap out the CDN in the back


without as much problems.


I don't know if that's possible with kind of the AWS S3 bucket


setup.


But that's kind of what I would recommend doing with images,


especially if you're unsure of the kind of the long term setup


that you have with your infrastructure, which CDN


do you want to use, like which one is cheaper,


which one is faster, those kind of things.


So the closer you have your URLs on your own domain,


the more likely you'll be able to just reuse those URLs


for the long run.


And that makes it a lot easier for when you have


to change the infrastructure.


SPEAKER 2


Hi, John.


I've got a question about the new rel link for linking.


I don't know if now it's int.


Until now, Google respect the nofollow directive.


But when you are on e-commerce website,


you have a link for categories and the link for attributes.


Don't know the name in English, Facet Navigation.


SPEAKER 1


With attribute and in e-commerce, for example,


for each attribute, there's a link


to a page that have no interesting content for Google,


but is useful for the user.


And sometimes I just put nofollow


to this sort of link.


So Google, until now, didn't call this link.


And now, with this change, what Google will do?


So it's not 100% defined.


But the plan is to make it so that you


don't have to make any changes.


So that we will continue to use these internal nofollow links


as a sign that you're telling us these pages are not


as interesting.


Google doesn't need to crawl them.


They don't need to be used for ranking, for indexing.


So it's not a 100% directive like robots.txt,


where you say these are never going to be crawled.


But it does tell us that we don't


need to focus on them as much.


So for us, the main change with nofollow and these new


attributes is for outbound links.


So from your website to another website.


Within the website, for these kind of faceted navigation,


for categories and sorting and things like that,


that continues to work.


OK, and for those attributes, UGC and sponsor,


so we said, what is the way, as a developer,


we must implement it?


You don't have to implement it.


We think it's a good idea to make it easier for search


engines to understand which of these are, sorry?


There's a word for this to add for Google


to understand more the content.


It should just hint for Google, and there's


no incidents on the ranking on the page rank, just


to the website.


Yeah, it's essentially the same handling as with nofollow,


except that you're telling us a little bit more detail of why


you want to nofollow this link.


So is this an advertisement, or is this blog comments?


So those kind of differences can make it a little bit easier


for our algorithms afterwards to think about, well, maybe these


are good blog comments, for example,


and maybe we should focus on them a little bit more.


But at least we have the ability to understand


the difference of what you're trying to say.


With the nofollow in the past, you're basically saying,


I just don't want you to look at this link.


And with the new attributes, you can tell us a little bit more


why you don't want us to look at it.


And again, it's for outbound links.


So it's not something that, like, within a website,


you really need to worry about.


And there's no way to use HTML sync to have a sidebar with only


a section for comments and to be better understand by Google.


So you have a section with a side for all the links.


We have no impact with the content.


OK, so something where you say everything in this section


should be nofollowed?


Yes.


Yeah.


I don't think there are any plans


to have kind of a section or an HTML element level nofollow.


So you would either have to do that


on the page level, which usually doesn't make much sense,


or on the per-link level so that you mark up


each individual link.


OK.


Thanks.


Sure.


Hi, John.


Hi.


So two questions, please, if I may.


First question will be, so a few days ago, three of my website,


which are all domains, were rolled to mobile-first index.


And I was wondering if you just ran


like another batch of indexing for a mobile-first index,


or this is like, have you heard from other webmasters


lately being rolled to mobile-first index,


or this is just my case?


I don't think they pick just your websites.


So this is something that we want


to roll out to all websites.


So we're doing that step by step as we can figure out


that sites are more or less ready.


We shift them over to the mobile-first indexing.


So that's like, the web is pretty big.


And I think we have half of the search results


moved to mobile-first indexing now, probably


a little bit more.


So the team is still moving forward on that.


Great, thanks for confirming.


Second question, I don't know if it relates to this discussion,


but I've seen the video Bartos did with you and Martin,


which was just blow-minding hearing those things.


And my question was, so if I have a full JavaScript


website, single-page application,


should I expect Google to be able to fully crawl


my website in the future?


And also taking into consideration that lately,


Google will move to the evergreen version.


So should we expect in the next few months


to have Google able to crawl the website fully,


also the JavaScript in a manner sense?


So we're getting better.


So from that point of view, I don't


think we can say we will absolutely


be able to index everything with JavaScript.


And we don't have any specific timeline where we say,


by then, we will be able to do that.


I think there will always be specific edge cases that


make it hard sometimes.


So a really basic one is a lot of JavaScript-based web apps


use one single URL for all of their content.


You click around, and it stays one single URL.


And if you just use one single URL,


then we will be able to only index that one single URL.


And we won't see all of the other content


that you could load if you click on different things.


So that's one kind of basic example


where we can do a lot of things with JavaScript,


but we can't make up URLs for your website that don't exist


and index content like that.


So from that point of view, I think


we can get a lot better.


We can definitely get into a situation


where a normal JavaScript-based website that


uses different URLs, that uses normal links,


then that we should be able to crawl and render that


and index it normally.


I kind of see that as a reasonable expectation


for the team.


I don't think that will happen in the next few months,


but we're working on it.


But we definitely won't be able to just take


any kind of JavaScript application or website


and automatically make sure it will be completely


workable in search.


So that's kind of one thing there.


Let's see.


Something else I wanted to kind of add there.


Oh, yeah, of course.


The other thing is if you currently


have a JavaScript-based website like that,


you probably don't want to be in a situation


where you're waiting for Google to figure it out.


So if you're currently thinking about search


and you already have a JavaScript-based website,


then there are ways that you can make sure


that it works where you don't have to wait.


So if you have a business, if you have something


that is important to be found in search,


then don't just kind of say, well, Google


will figure it out someday.


Instead, just kind of follow the guidelines


that Martin has been putting together at the JavaScript


videos that he's been putting up and make sure


that your site works in search, because it's


possible to make JavaScript-based sites work


well in search.


And it's also possible to make them work well


in other search engines.


So Google is able to do a lot with JavaScript sites,


but not all other services and search engines are.


And there are ways that you can kind of work around that.


So those are, yeah.


If I may share, so the technical solution we have


is dynamic rendering.


We built our own internal caller.


And the internal caller goes on the sitemap XML


and creates snapshots for all the URLs it finds in the sitemap.


And then save those pre-rendered HTML version in the server


to serve Google this version.


But I do remember in the previous IO,


you mentioned that hybrid rendering will be the future.


So I wonder if it's still the case,


because we haven't learned to improve our rendering


solution, like moving from dynamic rendering


to either server-side rendering or hybrid rendering.


Is this still the recommended solution?


I think you can do it with dynamic rendering or the setup


that you have now.


So there are different ways that you can do it.


And they also have different effects on search engines


as well as on users, kind of from a speed point of view.


So I wouldn't just blindly switch from one technology


to another just because you think it might make sense.


But I would test it out and make sure


that it works well for your users


from a speed point of view.


Sometimes with server-side rendering,


you can do really fancy things from speed.


And similarly with dynamic rendering,


it's something that works a little bit different.


But there are trade-offs in the different approaches.


So I would try it out with something small


to see what are the effects on your specific use case,


and does it make sense?


OK, thank you very much, John.


It was very helpful.


I appreciate it.


Hey, John.


Sorry for interrupting again.


No problem.


So first of all, I have to say, I


hope there will be more Hangouts in the future


so we have a little bit more time


to get all the questions answered.


You're doing a great job.


So the question I have is a follow-up to the schema part.


Would be the Google Note is making


with you rich results more helpful.


To make it short, maybe you can make a yes or no.


Will this list be extended in the future?


Is it final?


I don't know.


I would assume it's final for the moment,


because these kind of changes get discussed a really long


time internally.


And it's not something where they will kind of like say,


oh, we will just make a small list


and then change it a few days later.


They kind of want to have a stable state.


Why I'm asking, earlier in a Hangout last week,


we were just asking if we may change the schema type


from product to hotel for reviewing our products


in a way.


And yeah, actually, we are worried about investing more


in schema in general.


So can we make a change here?


Should we make a change?


And the question about reviewing the regions


or the destinations, which was been asked earlier


this Hangout, we see that Google collects


stars for skiing areas.


Knowledge panel, for example, you can see the stars are there.


So we think we were collecting them in the right way.


And now, I'm wondering if we can maybe implement a ski resort


or something like that, because yeah, we're


not sure if this will be for the future.


Yeah.


So one of the things with regards to the blog post


is that I think the types that we listed


include their respective subtypes.


So something like organization, I imagine,


includes things like hotels.


And maybe it even includes things


like ski resorts, which are kind of like one clean thing where


it's run by one organization.


I don't know for sure.


So I would double check that.


But in the schema.org documentation,


you can see the higher level types and the subtypes.


And that's something that might make it possible for you


to kind of pick a type that matches both what you're


trying to do and that aligns with what


we're trying to do.


All right.


Yeah.


Cool.


You also had, I think, a question about the spam network.


I think that was from you.


Exactly.


Can you tell me a little bit more


about what you're seeing there?


Yeah, each day, we are seeing links


that pop up from that network.


So a lot of domains appear.


And yeah, we're wondering, do they hurt?


Don't they hurt?


And we are uploading the disavow file,


not on a daily basis, of course, but sometimes we


think, do we have to?


And we have to do it for a lot of properties.


That makes it even more hard.


So yeah, and we have an agency that says,


that hurts your rankings, or two agencies which


were doing an audit.


So everyone is thinking we should re-upload the disavow


file, but I can't do it on a daily basis.


I don't want to do it.


So yeah, usually with these kind of things,


that's something that we can catch fairly well on our site.


So you don't have to do that on a daily basis,


definitely not on a daily basis.


So that's not something that should be affecting


your site negatively.


It sounds like you're really worried


about this particular network, and maybe it's affecting


a bunch of your sites.


So what might be useful there is if you


can send me some information about what you're seeing there.


You can send it to me by email, or on Twitter, or something.


Let me just drop my email address here.


And then I can pass that on to the web spam team


to double check.


But in general, these kind of weird link spam networks,


they come and go all the time, and they


don't have any negative effect on the websites


that get linked to.


Sometimes it will be that they link to five spammy sites


that they want to promote, and then 10 good sites


that they want to make it look like this kind of spammy site


is linking to some good sites as well.


And that sometimes includes good sites


where people don't know what is happening.


But we have a lot of practice with these kind of networks,


so we should be able to just catch that


so that you don't have to do anything there.


And again, I'm happy to double check with the web spam


team on this particular case.


Just to, yeah.


Yeah, we're really sure this must be spam


and must be annoying for other SEOs too.


So it would also be great if we could just remove them maybe


from the disavow file, because it's already


hundreds of domains which make it a little bit more


hard to check the disavow file maybe.


OK.


Thank you very much for the email, and we'll send it to you.


All right, thank you.


Cool.


Then there's a question about how.


How do we prevent Google from being crawling or indexing


our staging server?


How to prevent Google from crawling the staging server?


There are multiple ways.


So people do it in multiple ways.


I think the most important part is that you don't link to it,


because if you don't find it, then we can't crawl it.


But sometimes it still happens.


Ideally, what you would want to do


is provide some kind of server-side authentication


on the server so that normal users, when they go there,


they would get blocked from being able to see the content.


So that would include Googlebot.


And you can do that, for example,


on an IP address basis.


You can do it with a cookie.


You can do it with normal authentication on the server.


Anything where you have to prove that you're the right person


so that you can actually look at that content.


I think that's generally the best


approach for staging servers, because it means that you


can use overtext to prevent it.


You can also do that.


That's the next part I was getting to.


I think using authentication is the cleanest,


because it's something that means


you don't have to change the normal settings on the site


itself.


So in particular, robots.txt, but also noindex meta tags,


for example, on these pages.


Because it's a very, very common thing


that you set up your staging site with a new design,


and you have a robots.txt saying,


don't crawl any of this Google, because I'm still


testing things out.


And then you push that staging site to production


to make it live, and you accidentally


include that robots.txt file.


Or you accidentally include all of these noindex meta tags,


and then suddenly your normal website


is blocked from crawling, or your normal website


is blocked from indexing.


There's no examples of that.


Yeah, I think it happens to pretty much everyone.


So it just happens.


But ideally, I would just use authentication somehow


with staging sites so that you don't


have to think about the robots.txt and the meta tags.


If you can't do authentication, then robots.txt


is a good way to help with that as well.


What will happen with robots.txt is


if people are explicitly linking to your staging site,


then we might index the URL of the staging site.


So if you do a site query for staging.yourdomain.com,


then maybe you'll see a bunch of URLs that are shown there,


and they all have this snippet saying,


we can't tell you what this page is about


because we can't crawl it.


So that's something that is a little bit


confusing in the beginning, but it's not problematic


because your normal website will rank for your normal queries


in the staging website.


We don't know what is there, so we don't show it


in the search results.


We can show it if someone explicitly


knows that the staging site is there,


but we wouldn't show it in the normal results.


So I'd say ideally authentication,


and then robots.txt if you can't do it any other way,


and noindex, I would try to avoid doing that on the staging


site because it's really easy to accidentally push


a set of changes with the noindex meta tag on your pages.


SPEAKER 2


How will it be shown in Search Console?


So in Search Console, we start at one


when it comes to ranking.


So if your page is being shown as a featured snippet, which


is what sometimes people call the zero position in the search


results, then we would call that position one.


And if it's additionally shown in other places in the search


results page, then what happens with Search Console


is we decided to count the average topmost position.


So that means for a search results page


where you're ranking number one and number four,


we would count that as ranking number one.


In particular for your website, that kind of


makes sense because you're visible


and in the first position.


Similarly, if you're visible with multiple URLs


from the same website in the same search result page,


so perhaps your product page is ranking number five


and then your home page is ranking number six,


then what would happen there is on a site level,


when you look at the queries, we would count that


as ranking number five.


So that's the topmost position there.


If you look at it on a per URL basis,


then we will show your ranking once, one URL at number five


and one URL at number six.


So it depends a little bit on how


you look at it in Search Console,


if you look at it on a query level or on a URL level.


But in general, when we count the rankings,


we count the topmost position in that search results page.


And we average that across all of the search results


that are shown.


Also, it's worth keeping in mind this is not


a theoretical ranking, but based on what was actually


shown to users.


So what can sometimes happen is that your page


is shown in the featured snippet for a short while,


and then afterwards is shown in the normal position


for a longer period of time.


And then you have kind of that number one position and then


the number four position.


And if you look at it yourself manually afterwards,


it might be that you look at it and you see,


only ranking at number four, why did Google ever


say I was number one?


And that ranking that we show there, the position,


is really based on what we showed people at that time.


And sometimes it can happen that very few people see it


at position one.


But because that's where we showed it,


that's how we would count it.


Let's see.


Another one that is kind of an interesting question,


where probably we will go on forever.


I have a bit more time.


So if you all want to stick around a little bit longer,


that's fine too.


In your Webmaster Guidelines, the first bullet


of things to avoid is automatically generated content.


And yet, lots of sites do that.


So it kind of goes on with an example.


And you can find examples like this in finance.


You can even find it in Google News.


What's up with this guideline in the real world?


So I didn't double check that specific example


that you have there.


But I think this is a really interesting question.


And I think it's something that will probably evolve over time.


In particular, there are two aspects


that I see that are kind of common.


I don't know, common.


Let's see.


Maybe three.


Let's talk about three things.


So on the one hand, this guideline


was primarily put up because a lot of spam


is completely auto-generated.


And we need some kind of a way to take action on that.


And this is kind of the guideline


that we chose to do with that.


So a lot of spammy sites will take a list of keywords,


and they will automatically generate fake sentences that


make absolutely no sense, where when a user looks


at those pages, they're like, there's nothing useful here,


but there are ads, so they click on the ads.


And the Webmaster Guidelines here


give us a little bit more room to say,


well, this is a completely auto-generated site that


makes no sense to keep.


Therefore, we will remove it from search.


Another variation of this that we sometimes see from spammers


is that they take existing content,


and they run it through an automatic translating machine.


And they use the automatically generated translation


and try to rank with that, where essentially the translation,


again, is something that when a user looks at it,


they would say, well, this doesn't make any sense.


It's in the right language, but the content


is not comprehensible.


So that's, again, something that we


would treat as automatically generated content


and say, we need to take action on this.


The examples that you're kind of pointing at


are things where I think it is almost


going in the direction of, well, you're just


providing a table of data in a little bit more


understandable way.


So you could imagine things like a weather report in a sidebar,


where you could just show the weather is this many degrees,


and it's cloudy kind of in a table.


Or you could take that information


and just make a sentence out of it, which might


be easier for people to read.


And this kind of reformatting some amount of data


that you essentially have and providing that


in a way that is more readable, I


don't see that much of a problem with that.


I prefer to make sure that the rest of the page


has sufficient value so that when people go to that page,


they actually find value there, that they actually


think this is something that is useful to them.


After they search in the search results,


they click on something, and they found


something that was useful.


And that's the kind of thing where I kind of say,


well, if it's useful to users, it doesn't really


matter completely where it actually came from.


So that's, I think, one aspect where


things are a little bit different compared


to a spammy site that just auto-generates gibberish


content.


The other more, I guess, future-looking aspect


is there are various machine learning algorithms


that try to generate text as well based


on some amount of starter information.


And more and more, the examples I'm seeing there


are that these algorithms are able to generate something


that is actually pretty understandable and actually


pretty useful, where if you feed it


the right amount of information in the beginning,


it'll be able to take that content and write it up


in a way that is actually really easy to understand


and provides a lot of value.


I don't know if that's that far now,


that I would say that this kind of auto-generated content


is completely fine, and nobody will notice


that it was generated by a script rather than a human.


But maybe a few years down the time I found the road,


I could imagine that that happens.