RenderTron is a wrapper around Puppeteer, in fact.


JavaScript SEO office hours April 29th, 2020

And it has two components.


It has one component that is universally useful


if you want to use RenderTron, and one that is specifically


made for Express.js web service.


So the way that RenderTron works is actually, again, I'll


just show you because we have documentation


on the hats diagrams, which is nicer to explain them,


just like wave in the air in front of me.


Because I do that, I notice that I wave my hands quite a bit.


Here we go.


So in our search documentation, in the guides,


in the implement dynamic rendering guide.


So, oh, god.


Basically, it is one way to implement dynamic rendering.


Dynamic rendering means your server


looks at the user agent that comes in.


If it's a user browser, like Firefox, Edge, Opera,


or regular Chrome, then you would just run your JavaScript


and produce the JavaScript.


That's this part of the diagram here.


So the first part here where your server sees,


OK, a regular user requests the information.


So I give it the single page application or whatever


JavaScript it needs to actually fetch the content.


The JavaScript to load more content in,


or to populate search results on this category page,


or whatever.


Whereas if it detects a browser that is, sorry,


a user agent that is from a crawler, like Google


bot, Twitter, Facebook, any of the social media bots,


then it would, instead of just sending the response back,


it would proxy back to a rendering solution.


And then the rendering solution takes that URL,


opens it in a browser, waits until the page has finished


rendering, and then makes a snapshot of the HTML


and sends the HTML, the static HTML, back to the web server.


And the web server then sends that back


to the bot or crawler that requested it originally.


Rendertron is a thin wrapper around Puppeteer.


It basically creates two endpoints,


one for a screenshot, one for rendering.


We have a demo instance as well.


So there even is a web interface just for the fun of it.


So I can go to a website that does require JavaScript


to do a bunch of stuff.


Then I can, for instance, take a screenshot.


And OK, I can't take a screenshot.


The demo instance does not do screenshots.


OK, fine.


Wow, OK, the demo instance doesn't do anything.


Ah, no, I have a, I know what's the problem.


My certificate went down.


But that's OK, because it's an experiment page anyways.


So here we go.


So this uses a bunch of JavaScript


to actually do all these tests.


But if I check the page here for any script tags,


then we will see that we don't have any JavaScript


on this page.


So this is just static HTML rendered by Headless Chromium


through Puppeteer.


And that's the main component of it.


So you have a server which gets a URL


and then produces static HTML that you can send back.


The second component is the Express.js middleware.


If you are using Express.js as your web server


or as your application server, then you


can use Rendertron middleware to actually get


a way to integrate Rendertron into your web server.


If you are using Apache or Nginx or any of the other servers,


IIS, then you would basically configure it


like a reverse proxy.


That's Rendertron.


Does it only execute script tags, or does it execute, for example,


would it be a problem if it did the Google Analytics reference,


that sort of thing?


What do you mean?


So you get a script tag to fire a page view in Google Analytics.


It's got an employee doing that.


Rendertron would then execute that JavaScript,


generate the page view, but would not.


It strips the JavaScript entirely out of the HTML


that it serves as a suspect.


So you would not get Google Analytics on the HTML


that Rendertron generates, which is not great


if that's what you want.


A few people asked for an option to keep scripts in,


and I'm like, dynamic rendering's entire idea


is to remove every bit of JavaScript from a page.


So I don't think Rendertron is the right tool for that.


If what you're saying sounds more like you


want server-side rendering.


All right.


We have time for two, maybe three more questions.


Maybe one if it's one that makes me go talk for a longer time.


Oh, there's something coming in via chat.


Last question on that problem with infinite scroll,


if it's OK.


Yeah, sure.


We are afraid that when we'll fix it,


we should expect serious fluctuations in search results.


When we are searching description


of an article that loads in the infinite scroll,


Google says that there are seven or more articles with this


word.


Does that mean that the Googlebot interpreted


our articles incorrectly all this time?


I mean Googlebot saw twice bigger text


when it was in original.


I'm not 100% sure if I understand


where you're going with this.


But basically, if we, what's that,


where's that link taking me?


Let's click on that link.


OK, well, that's the, I think what we're seeing here


are different, yeah, all of these are, oh wait,


690C, 94E.


So we seem to be seeing different pages


with this text in it.


But I'm not 100% sure if that's a problem or not, to be honest.


If you change the way that your content looks like


and the site structure looks like,


then yes, that will cause ranking fluctuations


because we are effectively seeing changes in all the pages


that we have in the index.


We might see new pages if you change your URL structure


in the same time.


That will cause fluctuations, yes.


If what you are seeing right now in terms of search performance


is what you expect to see, then I


don't think that's a problem that needs to be fixed.


If you see problems with the way that search


works for your site, then that's something that you want to fix.


Don't fix what isn't broken.


As far as I can tell, and correct me


if I'm wrong, as far as what you have submitted


through the chat so far, we are seeing all your content.


We are ranking your content.


Your content shows up in search results.


Sounds to me like that's what you want.


And I wouldn't fix this if it wasn't a problem.


If you are seeing that we are not


seeing some of your content, or if you


notice that your content doesn't perform very well,


then you can consider reconfiguring your site.


But I wouldn't change anything without being 100% sure


that what you are seeing right now in terms of search


performance is not what you expect to see.


And just the fact that we see the same content


on multiple search, the same words in multiple URLs,


that does not mean that it's a problem.


It can only be that reporting is a little harder,


because your content might be spread across multiple URLs


and multiple different pages.


But if it works the way that you would want this to work,


and you get the clicks and impressions that you look for,


don't fix it, because it doesn't seem broken.


You're welcome.


OK, one more question.


Anyone?


I think maybe we have submissions through YouTube as well.


Is there some tool that I can use


to track the page speed with JavaScript automatically


in a dashboard?


I believe there's a bunch of tools that do things


like performance budgets.


You can run Lighthouse in an automated fashion,


automated fashion, blah, in an automatic fashion.


That's what I wanted to say.


There's a Lighthouse CI, I think it's called,


is the tool that you can set up to run, I don't know,


like twice a day.


And then you can snapshot the scores


and then see how the score changes over time.


I know that that has been built. I'm


pretty sure there's also paid services that


do something similar, but I don't have a list of them


right now.


I would just search for web performance budget tracking


or Lighthouse over time or something


like that, because I'm pretty sure it has been done before


and it's probably well documented at this point


as well.


So Lighthouse is a good tool to track performance over time.


I think you can probably also somehow


do that with web page test, but I'm not so sure about that.


And there's definitely paid solutions


out there that do that as well.


All right, now one last question from the audience.


That was an easy one.


Can I do one that might be, which JavaScript's involved.


Sure.


So I'm using Puppeteer to audit clients' websites.


So it's rendering the page and things like that.


And as I mentioned before, I noticed


that it runs Google Analytics scripts, which


I found quite powerful, because I can actually find out


if a client's got Google Analytics tagged three


or four times, things like that.


Whereas Googlebot uses robots.txt


to stop it doing that.


I'm wondering if, I don't know if it's you guys,


should we be using Puppeteer because it's a bot, really?


Should we be blocking, triggering


things like Google Analytics and things like that?


I think that is up to you.


On one end, if the client's marketing or analytics or data


science departments complain that they get visits


on analytics that look weird, then they


can either filter it by filtering,


or you can probably give them something


that they can filter by in Google Analytics.


You don't have to block Google Analytics,


because you are a bot.


That is your decision in the end.


If you don't want to accidentally skew,


I mean, if you are requesting the entire website


like once for an audit, I don't think


that's going to make a big difference,


unless the website has like five visits anyways per month.


And they're going to be like, we have 10 visits.


That's a 100% increase.


And then they found out that's actually


you botting the website.


But normally, it doesn't really matter that much.


And I remember that at least the last time I


worked with a data science department,


they're like, oh, yeah, we're removing everyone who works


here from Google Analytics, and we also remove


all the bots that look weird.


You can maybe set the user agent to something


that is easily spotable.


Yeah, I do that.


Yeah, and then that's an easy filtering question.


Maybe Google Analytics knows about it anyhow.


Probably.


If you are having a really weird user agent,


I'm pretty sure that Google Analytics will be like, hmm,


interesting.


Let's put that here.


All right.


Yeah, but in the end, it's your decision


if you want to run Analytics or if you don't.


OK.


Excellent.


In which case, thank you all so much for your questions


and for joining.


Also, thanks to everyone who submitted


a question through YouTube.


And the next JavaScript SEO Office Hours


is in a week's time.


It is also already posted on youtube.com


slash googlewebmaster slash community.


If you search for the post that says JavaScript SEO Office


Hours May 6, I believe, is the next one,


then you can submit your questions there.


I'll make sure that that one's also recorded, as well as


this one.


And I wish you all a fantastic time.


Stay safe, stay healthy, and see you soon again.


Bye.


Cheers.


Ciao.


The recording.


Building a better website for Steve

Welcome back, everyone, to the next episode of the Search


Building a better website for Steve

Off the Record podcast.


Building a better website for Steve

Our plan is to talk a bit about what's


Building a better website for Steve

happening at Google Search, how things work behind the scenes,


Building a better website for Steve

and who knows, maybe have some fun along the way.


Building a better website for Steve

My name is John Mueller.


Building a better website for Steve

I'm a search advocate on the Search Relations team


Building a better website for Steve

here at Google in Switzerland.


Building a better website for Steve

Today, I'm joined by Gary, also on the team.


Building a better website for Steve

GARY ILLYES-SELLERSEN.


Building a better website for Steve

And Martin.


Building a better website for Steve

MARTIN SPLITT SANDBERG.


Building a better website for Steve

Both are on the Search Relations team, actually.


Building a better website for Steve

In this episode, we're back at Steve,


Building a better website for Steve

working on our theoretical search engine.


Building a better website for Steve

If you don't know what Steve is, then check out


Building a better website for Steve

episode 17 of our podcast.


Building a better website for Steve

Just to be clear, this is not Google.


Building a better website for Steve

A theoretical search engine makes


Building a better website for Steve

it a bit easier to talk about what might


Building a better website for Steve

be done if we started fresh.


Building a better website for Steve

So welcome back, fellow Stevians.


Building a better website for Steve

This time, we're going to discuss guidelines


Building a better website for Steve

for people who make websites.


Building a better website for Steve

What kind of guidelines?


Building a better website for Steve

Well, you'll see.


Building a better website for Steve

Gary, what should we include?


Building a better website for Steve

Well, I definitely think we should have quality guidelines,


Building a better website for Steve

for example, because quality guidelines are great.


Building a better website for Steve

And we can use them in very imaginative ways.


Building a better website for Steve

Can they be a little less vague than saying


Building a better website for Steve

you should make a good website?


Building a better website for Steve

Yes, please.


Building a better website for Steve

What would be good quality in terms of making a website?


Building a better website for Steve

What would you say?


Building a better website for Steve

I mean, there are lots of things to that, right?


Building a better website for Steve

Like write good content, for example.


Building a better website for Steve

But what does that mean?


Building a better website for Steve

Like certain word length or?


Building a better website for Steve

Well, no.


Building a better website for Steve

So this is my pickle with certain search engines


Building a better website for Steve

guidelines as well.


Building a better website for Steve

Like write good content, for example, that's not precise.


Building a better website for Steve

It's like, what does that mean, as you said?


Building a better website for Steve

And I think we can definitely say more about it.


Building a better website for Steve

Like structure your content better


Building a better website for Steve

or use words instead of images to express your thoughts.


Building a better website for Steve

Write naturally.


Building a better website for Steve

Like if you are writing as a robot,


Building a better website for Steve

then probably it's not going to do any good for your website.


Building a better website for Steve

There are things that we can say.


Building a better website for Steve

But when I'm saying quality guidelines,


Building a better website for Steve

I'm thinking more about the imaginative ways


Building a better website for Steve

that people try to spam, perhaps, search engines.


Building a better website for Steve

Basically trick a search engine into believing this


Building a better website for Steve

is useful when it isn't.


Building a better website for Steve

Yeah, and then rank the content more or higher, let's say.


Building a better website for Steve

And for those practices, we definitely need guidelines.


Building a better website for Steve

Because some things can be borderline.


Building a better website for Steve

Some things are just plain spam.


Building a better website for Steve

Like, for example, if you think of keyword stuffing,


Building a better website for Steve

that was very prominent in the earlier days of the internet.


Building a better website for Steve

Those are clear cut spam.


Building a better website for Steve

And they are not useful for users.


Building a better website for Steve

So why would they be useful for search engines?


Building a better website for Steve

OK, so no keyword stuffing.


Building a better website for Steve

Fair enough.


Building a better website for Steve

So is this more like policies, like what


Building a better website for Steve

people should be doing in order to appear in Steve?


Building a better website for Steve

Or is it like a recommendation?


Building a better website for Steve

I would say it's more like policies.


Building a better website for Steve

Steve should definitely have policies for content or spam


Building a better website for Steve

quality.


Building a better website for Steve

Content quality?


Building a better website for Steve

I don't know.


Building a better website for Steve

I don't know what would be the right word for that.


Building a better website for Steve

Policies about what people can try


Building a better website for Steve

to rank in the search engine.


Building a better website for Steve

So for example, don't try keyword stuffing


Building a better website for Steve

because we might do or we might manually adjust


Building a better website for Steve

your rank or the website's rank.


Building a better website for Steve

What about like wide text on wide background?


Building a better website for Steve

Yeah, hidden text.


Building a better website for Steve

That's also good.


Building a better website for Steve

That's also very 90s, I guess.


Building a better website for Steve

But isn't that one a little tricky?


Building a better website for Steve

Because I mean, wide text on wide background


Building a better website for Steve

is very clearly a spammy thing to do


Building a better website for Steve

because I don't see any value in doing that for like,


Building a better website for Steve

there's no good reason to do that.


Building a better website for Steve

But what if you're hiding text behind, I don't know,


Building a better website for Steve

a carousel or a tab box or something like that?


Building a better website for Steve

Then it's not visible, but it's not a spammy thing to do.


Building a better website for Steve

I guess that's where the nuances come in.


Building a better website for Steve

Like you are doing this for UX purposes


Building a better website for Steve

or you are doing this for the purpose of manipulating search


Building a better website for Steve

results or search rankings.


Building a better website for Steve

But then that also means that someone will


Building a better website for Steve

have to assess the situation.


Building a better website for Steve

Basically, someone will have to take a look


Building a better website for Steve

and see what you are doing on your website


Building a better website for Steve

and why, which is kind of like gray area, I think.


Building a better website for Steve

Because sometimes it might not be


Building a better website for Steve

that clear what's happening.


Building a better website for Steve

And then you need more guidance, perhaps.


Building a better website for Steve

So is this a bit like telling people what not to do?


Building a better website for Steve

Because Steve isn't that good at recognizing a page


Building a better website for Steve

like a user would recognize it?


Building a better website for Steve

Like a user, if you see a page with hidden text,


Building a better website for Steve

they'd be like, oh, there's no text here.


Building a better website for Steve

I mean, in this case, I don't think it's a policy.


Building a better website for Steve

I think now we're talking a lot about best practices


Building a better website for Steve

and guidelines in the terms of this


Building a better website for Steve

is what you should do to make sure


Building a better website for Steve

that Steve understands that this is good for a user,


Building a better website for Steve

rather than, hey, if you're doing this,


Building a better website for Steve

you are very clearly in the bad book.


Building a better website for Steve

You're definitely doing this to manipulate Steve


Building a better website for Steve

into believing something that isn't actually there


Building a better website for Steve

or isn't actually good.


Building a better website for Steve

It is, in a way, also about the limitations of Steve, probably.


Building a better website for Steve

But it's also a thing where if you accidentally


Building a better website for Steve

had white text and white background,


Building a better website for Steve

I mean, that can happen, right?


Building a better website for Steve

Style sheets are a thing.


Building a better website for Steve

And if you do it wrong, then text might not show up for users.


Building a better website for Steve

And then users won't see the text either.


Building a better website for Steve

So even you might not have intended to manipulate Steve,


Building a better website for Steve

you ended up manipulating Steve plus hurting your users.


Building a better website for Steve

So there's a bit of overlap, I guess.


Building a better website for Steve

GARY I think it comes down to intent.


Building a better website for Steve

What are you intending to do with that?


Building a better website for Steve

Because as Martin said, it can happen


Building a better website for Steve

that the style sheet doesn't load.


Building a better website for Steve

For example, you are hosting your style sheet on a CDN,


Building a better website for Steve

and the CDN's TLS certificate expires.


Building a better website for Steve

And then the browser will just not load that style sheet.


Building a better website for Steve

And then suddenly you have weird UX on your site,


Building a better website for Steve

and some text got hidden.


Building a better website for Steve

Like, you didn't intend to mislead anyone with that.


Building a better website for Steve

It's a technical problem.


Building a better website for Steve

But I think that's more clear cut.


Building a better website for Steve

I think the problem is where someone is purposefully


Building a better website for Steve

placing objects on the real estate of the page


Building a better website for Steve

such that they are hiding stuff for the purpose of manipulating


Building a better website for Steve

search results.


Building a better website for Steve

That's very hard to assess, I think.


Building a better website for Steve

It's not that search engines cannot recognize that there is


Building a better website for Steve

hidden text on the page, but more like there's


Building a better website for Steve

an intent of manipulation.


Building a better website for Steve

OK.


Building a better website for Steve

So I guess I would also cover things like text


Building a better website for Steve

that is not actually hidden.


Building a better website for Steve

Like the old school technique of one pixel high fonts.


Building a better website for Steve

Oh, yeah.


Building a better website for Steve

I mean, that's also hidden text, right?


Building a better website for Steve

Well, I mean, theoretically you can see it, right?


Building a better website for Steve

It's just very hard.


Building a better website for Steve

I mean, one pixel height, that's like Morse code.


Building a better website for Steve

Yeah, that's fair.


Building a better website for Steve

You are not even going to have shapes.


Building a better website for Steve

OK, three pixels.


Building a better website for Steve

Three pixels.


Building a better website for Steve

It's like for mobile phones, they have a small screen.


Building a better website for Steve

You have to use a small font.


Building a better website for Steve

I mean, I guess the aspect is also


Building a better website for Steve

like what you're trying to include


Building a better website for Steve

in those hidden elements.


Building a better website for Steve

Like if you're just elaborating on the text


Building a better website for Steve

that you already have on the page,


Building a better website for Steve

it's like that doesn't really change much for the page.


Building a better website for Steve

But if you're including aspects that


Building a better website for Steve

are totally tangential to what you have visible on the page,


Building a better website for Steve

then suddenly that page shows up in search results


Building a better website for Steve

where it doesn't really make sense to show.


Building a better website for Steve

And that seems like a problem.


Building a better website for Steve

Do you think these, I don't know,


Building a better website for Steve

Steve Master guidelines, or I don't


Building a better website for Steve

know what we would call them, these guidelines would end up


Building a better website for Steve

being framed in a way that you could see them


Building a better website for Steve

like as a law book or like some legal advice, which


Building a better website for Steve

is like clearly formulated.


Building a better website for Steve

Don't do this.


Building a better website for Steve

I mean, knowing people, they'll take it that way either way,


Building a better website for Steve

even if it's not intended to be that.


Building a better website for Steve

So I think phrasing there is very, very tricky


Building a better website for Steve

to get right in order to manage the expectations from people


Building a better website for Steve

reading them.


Building a better website for Steve

Also, if people think that they are legal advice, that's fine.


Building a better website for Steve

But we are not, and I repeat, we are not giving legal advice


Building a better website for Steve

because we are not lawyers.


Building a better website for Steve

And another thing that I think goes forgotten sometimes


Building a better website for Steve

is it's kind of like a playbook for both people making websites


Building a better website for Steve

but also for people making Steve.


Building a better website for Steve

Because if we develop something new in Steve,


Building a better website for Steve

we need to see how that fits with what we tell people


Building a better website for Steve

is a good practice or is not a good practice.


Building a better website for Steve

If we are, for example, saying like, oh, we


Building a better website for Steve

will do something about people who have invisible text,


Building a better website for Steve

then we need to be very, very careful on what


Building a better website for Steve

we define as invisible text for us to be caught in that way.


Building a better website for Steve

And again, then it comes back to intentions and clarifying


Building a better website for Steve

the intention, which might be hard to do


Building a better website for Steve

on a technical level.


Building a better website for Steve

But it's important that we do this work.


Building a better website for Steve

So I guess some of these things are


Building a better website for Steve

very grayscale, which make them really hard to enforce


Building a better website for Steve

as well.


Building a better website for Steve

Do you think there are any guidelines or policies that


Building a better website for Steve

are completely clear cut and obvious,


Building a better website for Steve

like you should not do this?


Building a better website for Steve

Otherwise, we should not show you?


Building a better website for Steve

For quality.


Building a better website for Steve

I mean gibberish.


Building a better website for Steve

Gibberish.


Building a better website for Steve

We see not that much anymore, but we still


Building a better website for Steve

see some gibberish pages on the internet,


Building a better website for Steve

not so much in search results anymore,


Building a better website for Steve

but on the internet definitely, where people just come up


Building a better website for Steve

with lots of random words that they want to rank for.


Building a better website for Steve

And then they put it on a page, and they


Building a better website for Steve

hope that they are going to rank for it,


Building a better website for Steve

and then use affiliate links or something


Building a better website for Steve

to earn money from those pages.


Building a better website for Steve

And that's pretty clear cut, I think.


Building a better website for Steve

But it might just be Swiss German.


Building a better website for Steve

OK.


Building a better website for Steve

Like random words on a page.


Building a better website for Steve

I mean, that also feels like something


Building a better website for Steve

that Steve should just be able to recognize.


Building a better website for Steve

It's like these words do not make sense together.


Building a better website for Steve

We should just skip over it.


Building a better website for Steve

But then you can also combine it with different things.


Building a better website for Steve

The other thing of these guidelines


Building a better website for Steve

is that you might be able to combine.


Building a better website for Steve

Like, for example, if we go back to hidden text, which perhaps


Building a better website for Steve

sometimes you can't make up your mind about whether it's bad


Building a better website for Steve

or not because you can't decide if it's for UX


Building a better website for Steve

or for manipulation.


Building a better website for Steve

But then you combine it with gibberish,


Building a better website for Steve

and then suddenly you have much better grip on what's


Building a better website for Steve

happening on the site.


Building a better website for Steve

Basically, the owner of the site or whoever


Building a better website for Steve

is producing the content is actually


Building a better website for Steve

trying to manipulate something.


Building a better website for Steve

So it's probably good to have, not so prominent anymore,


Building a better website for Steve

but it's still probably good to have something about gibberish,


Building a better website for Steve

for example.


Building a better website for Steve

OK.


Building a better website for Steve

Then we have links, for example.


Building a better website for Steve

Yeah.


Building a better website for Steve

I say, for example, a lot.


Building a better website for Steve

Interesting.


Building a better website for Steve

We have links, and we need guidelines for that as well.


Building a better website for Steve

Because as far as I remember, when


Building a better website for Steve

Martin was coding up in JavaScript the ranking algorithm


Building a better website for Steve

for Steve, he was using links as a ranking factor.


Building a better website for Steve

Can't we just move away from links?


Building a better website for Steve

But I read this really cool paper.


Building a better website for Steve

Sure, it's a bit dated, but it kind of


Building a better website for Steve

has a point on the fact that if good things link to your thing


Building a better website for Steve

and lots of good things link to your thing,


Building a better website for Steve

then there might be a correlation


Building a better website for Steve

in that thing being linked to also being quite good.


Building a better website for Steve

But obviously, we can't just use that one ranking factor.


Building a better website for Steve

We have to use other things as well.


Building a better website for Steve

I mean, couldn't we just get rid of links as a ranking factor?


Building a better website for Steve

We can use them to discover new content.


Building a better website for Steve

But it feels like at this point, the links on the web


Building a better website for Steve

are completely unnatural.


Building a better website for Steve

I think there's still a good deal of natural useful things


Building a better website for Steve

out there in terms of linking.


Building a better website for Steve

But I don't think the links are any really, really


Building a better website for Steve

important ranking factor.


Building a better website for Steve

I mean, Gary, you looked at the ranking factor weighting.


Building a better website for Steve

What do you think?


Building a better website for Steve

Is links or are links still a very, very important thing,


Building a better website for Steve

or is it just one thing out of many?


Building a better website for Steve

I mean, it's always just one thing, right?


Building a better website for Steve

Yeah, fair enough.


Building a better website for Steve

But is it a really important thing or not so much?


Building a better website for Steve

As far as I remember, you can rank without links.


Building a better website for Steve

But since we do have in the ranking algorithm,


Building a better website for Steve

we do need guidelines for it or policies about what people


Building a better website for Steve

should do about links or can do about links,


Building a better website for Steve

may do about links.


Building a better website for Steve

What's the right word?


Building a better website for Steve

I don't know.


Building a better website for Steve

So the guidelines or policy should


Building a better website for Steve

be about things that we use for ranking and probably also


Building a better website for Steve

for serving, like how we show things in the search results,


Building a better website for Steve

as well as best practices for users


Building a better website for Steve

so that users are happy when they see something.


Building a better website for Steve

So basically, if you see something in the guidelines,


Building a better website for Steve

you can assume that either it's used by ranking,


Building a better website for Steve

it's used in serving, or users might see it.


Building a better website for Steve

Well, I guess that covers pretty much all of the web.


Building a better website for Steve

MARTIN SPLITTENBERG


Building a better website for Steve

OK.


Building a better website for Steve

I feel like we're going to need a whole website for all


Building a better website for Steve

of these guidelines to cover all of these details,


Building a better website for Steve

like so much stuff.


Building a better website for Steve

Wow.


Building a better website for Steve

MARTIN SPLITTENBERG Does feel like it.


Building a better website for Steve

We should definitely publish them


Building a better website for Steve

so that people can look at them and use them to figure out


Building a better website for Steve

how to make better decisions on how to make their websites.


Building a better website for Steve

Just let's not call it Webmaster Guidelines,


Building a better website for Steve

because Webmaster is such an archaic term.


Building a better website for Steve

It is.


Building a better website for Steve

Yeah.


Building a better website for Steve

Front-end developer guidelines.


Building a better website for Steve

No.


Building a better website for Steve

It's not just developers.


Building a better website for Steve

It could just be like small business owners or something.


Building a better website for Steve

Web creators guidelines, website owner guidelines.


Building a better website for Steve

But I mean, they're specific to Steve's search, right?


Building a better website for Steve

So it's fair enough.


Building a better website for Steve

Wait, wait, wait.


Building a better website for Steve

We talked about policies.


Building a better website for Steve

We talked about guidelines, best practices.


Building a better website for Steve

Oh, I like best practices because IETF,


Building a better website for Steve

the organization that that's publishing


Building a better website for Steve

the standards for the internet, is also


Building a better website for Steve

using best practices as a word.


Building a better website for Steve

So we can borrow that.


Building a better website for Steve

And policies, that's really nice because it has a different


Building a better website for Steve

meaning than guidelines.


Building a better website for Steve

Basically, it's like we might take manual action


Building a better website for Steve

on your site if you are violating these policies,


Building a better website for Steve

for example.


Building a better website for Steve

So policies is also good.


Building a better website for Steve

And they are related to content.


Building a better website for Steve

So could be content policies.


Building a better website for Steve

And then we have best practices.


Building a better website for Steve

We have content policies.


Building a better website for Steve

And probably we are missing the technical guidelines.


Building a better website for Steve

Oh, yeah, technical.


Building a better website for Steve

We should have something like all websites should include


Building a better website for Steve

five lines of JavaScript minimum, right?


Building a better website for Steve

To have them banned from Steve.


Building a better website for Steve

I mean, for the beginning, I wonder how much of the web


Building a better website for Steve

we can still represent through our search engine


Building a better website for Steve

or show to users through our search engine


Building a better website for Steve

if we don't support JavaScript.


Building a better website for Steve

Because supporting JavaScript is costly, after all.


Building a better website for Steve

It turns out, surprise.


Building a better website for Steve

I know, Gary looks at me like, yes, of course it is.


Building a better website for Steve

Yeah, I mean, it is an integral part of the web of today.


Building a better website for Steve

So we need to support it.


Building a better website for Steve

We can't just ban it.


Building a better website for Steve

But I really want to.


Building a better website for Steve

Of course you want to.


Building a better website for Steve

But you can't.


Building a better website for Steve

Move on.


Building a better website for Steve

So what kind of guidelines or technical guidelines


Building a better website for Steve

should we have?


Building a better website for Steve

What should we include?


Building a better website for Steve

Oh, my.


Building a better website for Steve

So with the technical guidelines,


Building a better website for Steve

in this specific case, I don't like the word guidelines here.


Building a better website for Steve

Because I think when we come to technical guidance,


Building a better website for Steve

I believe it's really, it definitely


Building a better website for Steve

is firmly in best practice land.


Building a better website for Steve

Because for instance, it does make our life a lot easier.


Building a better website for Steve

And it makes it a lot more reliable and reproducible


Building a better website for Steve

if people are using valid HTML, if their HTML is valid


Building a better website for Steve

according to the specification and they write


Building a better website for Steve

HTML the way it was intended.


Building a better website for Steve

Because then we have a very clear cut


Building a better website for Steve

semantic and a very clear cut structure of every document


Building a better website for Steve

that you put on the web.


Building a better website for Steve

So if my website is not valid HTML, it won't be indexed?


Building a better website for Steve

That's exactly why I say best practice and not


Building a better website for Steve

a guideline or a policy.


Building a better website for Steve

Because of course, there are billions of websites out there,


Building a better website for Steve

if not more, that are not written in valid HTML.


Building a better website for Steve

They have some sort of HTML validation errors.


Building a better website for Steve

If you would put it into a validator,


Building a better website for Steve

it would have a problem.


Building a better website for Steve

We can't just skip over that.


Building a better website for Steve

So if you are writing an HTML page that is not valid HTML,


Building a better website for Steve

we still want to index it.


Building a better website for Steve

Because the content might be really, really good and useful


Building a better website for Steve

for users.


Building a better website for Steve

So we have to deal with that.


Building a better website for Steve

But obviously, the moment we are dealing with something


Building a better website for Steve

where the semantics aren't clear, we might make mistakes


Building a better website for Steve

or we might misunderstand something.


Building a better website for Steve

We might misunderstand something is a title when it isn't


Building a better website for Steve

or is like a headline when it isn't


Building a better website for Steve

or is a link when it isn't, depending


Building a better website for Steve

on what's the problem with the HTML that you wrote.


Building a better website for Steve

But we have to be lenient, and we have to understand it.


Building a better website for Steve

But you, as the website's creator,


Building a better website for Steve

might get mixed results.


Building a better website for Steve

You might not necessarily see the page perform


Building a better website for Steve

the way you would expect it to perform on the search engine,


Building a better website for Steve

just because Steve had to make a bunch of guesses.


Building a better website for Steve

But again, if we were to say, this is what you have to do,


Building a better website for Steve

you have to write valid HTML, people


Building a better website for Steve

would get the idea that, yes, if my HTML is not valid,


Building a better website for Steve

my page won't show up in Steve, which is not true.


Building a better website for Steve

That's why I would like to call the technical side of things


Building a better website for Steve

best practices rather than guidelines or even policies.


Building a better website for Steve

OK.


Building a better website for Steve

That works for me.


Building a better website for Steve

Yay.


Building a better website for Steve

What would we put in the guidelines bucket then?


Building a better website for Steve

So my other one.


Building a better website for Steve

Or we don't need that bucket at all.


Building a better website for Steve

I think we need that bucket with technical guidelines.


Building a better website for Steve

I think one is, as I said, valid HTML does help us better


Building a better website for Steve

understand the page and have more ease in parsing


Building a better website for Steve

and understanding what you're trying to put out there.


Building a better website for Steve

A pet peeve of mine is, because that still


Building a better website for Steve

is an ongoing problem, use links.


Building a better website for Steve

Like, there's an HTML tag.


Building a better website for Steve

It's the anchor tag.


Building a better website for Steve

It has an href attribute that should


Building a better website for Steve

go to a URL that is resolvable and actually


Building a better website for Steve

fetchable from the network.


Building a better website for Steve

If you're doing something fancy, like buttons or spans


Building a better website for Steve

with on-click handlers and lots of JavaScript,


Building a better website for Steve

just to mimic what a link would do, then you're doing it wrong.


Building a better website for Steve

So that should probably be in there.


Building a better website for Steve

Use image tags.


Building a better website for Steve

When you have images, don't use like CSS background images,


Building a better website for Steve

because we might find them to be just decorative and not


Building a better website for Steve

like semantically important for the page.


Building a better website for Steve

Have captions on your figures.


Building a better website for Steve

Have alt text on your images so that people


Building a better website for Steve

who are using assistive technology


Building a better website for Steve

understand what's in the image.


Building a better website for Steve

Also, we save money there if we don't


Building a better website for Steve

have to do like computer vision, because computer vision, again,


Building a better website for Steve

is expensive if we want to do that.


Building a better website for Steve

No?


Building a better website for Steve

GARY I think we should still do computer vision on the images,


Building a better website for Steve

because as you may have noticed, I'm not putting too much trust


Building a better website for Steve

in things.


Building a better website for Steve

MARTIN SPLITTENBERG really?


Building a better website for Steve

I didn't notice.


Building a better website for Steve

GARY If there are two sources for the same thing,


Building a better website for Steve

then that's probably better.


Building a better website for Steve

MARTIN SPLITTENBERG I mean, I understand, of course,


Building a better website for Steve

we want to do image recognition or image understanding


Building a better website for Steve

and computer vision for doing that.


Building a better website for Steve

But if we get input from that, which also


Building a better website for Steve

is usually fuzzy, right?


Building a better website for Steve

You don't necessarily, a computer


Building a better website for Steve

gives you a confidence of some percentage.


Building a better website for Steve

I think to 90% likelihood, this is an apple.


Building a better website for Steve

And when we then also have an alt text that


Building a better website for Steve

says picture of a red apple, then I


Building a better website for Steve

think that would reinforce our confidence


Building a better website for Steve

in believing what it is.


Building a better website for Steve

So alt attributes should still be there,


Building a better website for Steve

no matter how much computer vision we do.


Building a better website for Steve

Yeah, I'm saying I agree.


Building a better website for Steve

So the guideline should still be there


Building a better website for Steve

needs to be semantic enriched information, like an image tag


Building a better website for Steve

with alt text that should match the content of the image.


Building a better website for Steve

Actually, the mismatch between an alt text


Building a better website for Steve

and what we think is on the picture


Building a better website for Steve

after doing computer vision is probably a website quality


Building a better website for Steve

guideline situation, or like a spammy issue thing.


Building a better website for Steve

JOHN MUELLER I don't know.


Building a better website for Steve

I struggle with the images sometimes


Building a better website for Steve

because you put images for lots of different reasons on a page.


Building a better website for Steve

You have an image of a beach on a website.


Building a better website for Steve

It could be, I don't know, a vacation photo.


Building a better website for Steve

It could be a hotel.


Building a better website for Steve

It could be a recipe.


Building a better website for Steve

It could be water quality measurements


Building a better website for Steve

or some random scientific thing.


Building a better website for Steve

And the topic of the website could be completely different,


Building a better website for Steve

but it's exactly the same image.


Building a better website for Steve

And we recognize it's like, oh, it's a photo of a beach.


Building a better website for Steve

And it's like, OK, so what do we do with this information


Building a better website for Steve

that there's a photo of a beach on here?


Building a better website for Steve

I mean, to be honest, that's exactly why I


Building a better website for Steve

want to call it best practices.


Building a better website for Steve

So for any image that adds meaningful information,


Building a better website for Steve

for instance, if it's a website about water quality


Building a better website for Steve

in Switzerland and there's a map of different water


Building a better website for Steve

qualities across Switzerland, that


Building a better website for Steve

is meaningful to the content versus if it's


Building a better website for Steve

a glass of water, like a stock photo of a glass of water.


Building a better website for Steve

And that you might add for decorative purposes.


Building a better website for Steve

And then you could leave out the alt text


Building a better website for Steve

or have an empty alt text.


Building a better website for Steve

But if we were to say images have


Building a better website for Steve

to have alt text, then we are exactly in the area


Building a better website for Steve

where people would then be like image of a glass of water,


Building a better website for Steve

which is not very helpful.


Building a better website for Steve

But if it's best practice and we say


Building a better website for Steve

for images that add to the content


Building a better website for Steve

are meaningful to the content, then I think we can say that.


Building a better website for Steve

GABRIEL KREIMAN-MARTINEZ-HENRYO Yeah, this is my pet peeve,


Building a better website for Steve

actually, with some other search engines,


Building a better website for Steve

Webmaster Guidelines, because they're related to images.


Building a better website for Steve

We actually have something like you must have,


Building a better website for Steve

or worded as you must have the alt attributes filled out.


Building a better website for Steve

And then we end up with lots of really weird alt text,


Building a better website for Steve

like the file name of the image, for example.


Building a better website for Steve

And then for whom is that useful?


Building a better website for Steve

No one.


Building a better website for Steve

And the other thing is that we don't,


Building a better website for Steve

or we will have to differentiate between decorative images,


Building a better website for Steve

functional images, and informative images, as you said.


Building a better website for Steve

Because two of those might not be needed for search engines,


Building a better website for Steve

like we don't necessarily need them in search results.


Building a better website for Steve

Although functional images should still


Building a better website for Steve

have alt attributes for accessibility reasons.


Building a better website for Steve

But otherwise, for decorative images, for example,


Building a better website for Steve

you don't need alt attributes filled out, because why?


Building a better website for Steve

JOHN MUELLER


Building a better website for Steve

Are there any aspects where you'd say,


Building a better website for Steve

this is very clear cut and obvious?


Building a better website for Steve

For example, if a page has a noindex, should we be like,


Building a better website for Steve

well, you say noindex, but it looks


Building a better website for Steve

like there's something useful here.


Building a better website for Steve

We'll just show it anyway.


Building a better website for Steve

MARTIN SPLITT


Building a better website for Steve

I think with noindex or also robots tags,


Building a better website for Steve

we should be quite clear that we follow those.


Building a better website for Steve

So if you say noindex, then the page just


Building a better website for Steve

doesn't go in the index.


Building a better website for Steve

Which with robots.txt and noindex in between


Building a better website for Steve

is this weird gray area where people are like,


Building a better website for Steve

I don't want it crawled and I don't want it indexed.


Building a better website for Steve

But if we can't crawl it, then we don't see the noindex.


Building a better website for Steve

But if we crawl it, then we violate the robots.txt,


Building a better website for Steve

but we do see the noindex.


Building a better website for Steve

So it's like, hm, it's an interesting one.


Building a better website for Steve

JOHN MUELLER


Building a better website for Steve

It feels like Steve should have a directive


Building a better website for Steve

for noindex and nocrawl.


Building a better website for Steve

What do you think, Gary?


Building a better website for Steve

GARY ILLYES


Building a better website for Steve

No.


Building a better website for Steve

Oh, yeah, we could just call it no.


Building a better website for Steve

We are going to do the same thing that the industry does.


Building a better website for Steve

And we are going to have disallow, allow,


Building a better website for Steve

and user agent plus the sitemap in robots.txt, and that's all.


Building a better website for Steve

And then we are going to have the same robots metatexts


Building a better website for Steve

that other search engines also follow.


Building a better website for Steve

But we could be a trendsetter.


Building a better website for Steve

We could drive some new standards across the internet.


Building a better website for Steve

We don't need new standards.


Building a better website for Steve

MARTIN SPLITT


Building a better website for Steve

Hold on, hold on.


Building a better website for Steve

Actually, the draft has us covered


Building a better website for Steve

because it allows vendor-specific additions.


Building a better website for Steve

Yes, that's not to say that it's a good idea.


Building a better website for Steve

No, I didn't say that.


Building a better website for Steve

I'm just saying we can follow the industry standard


Building a better website for Steve

recommendation and still do bad things.


Building a better website for Steve

GARY ILLYES


Building a better website for Steve

Exactly.


Building a better website for Steve

That's what we are going to avoid, doing bad things.


Building a better website for Steve

OK, fine.


Building a better website for Steve

But if you want a very clear-cut thing


Building a better website for Steve

in the technical guidelines, HTTPS, I think,


Building a better website for Steve

is a no-brainer.


Building a better website for Steve

I think a website having HTTPS is a good thing.


Building a better website for Steve

It's a good thing, but it's also not something that,


Building a better website for Steve

like if you are on HTTP, Steve will still try to index you.


Building a better website for Steve

So again, it's a best practice, but it's not a guideline.


Building a better website for Steve

MARTIN SPLITT


Building a better website for Steve

But it's a very clear-cut best practice,


Building a better website for Steve

like if you can have HTTPS.


Building a better website for Steve

But what would be a guideline that is clear-cut?


Building a better website for Steve

Oof, a guideline.


Building a better website for Steve

Well, as I probably said, don't do links with JavaScript only.


Building a better website for Steve

GARY ILLYES


Building a better website for Steve

Yeah, OK.


Building a better website for Steve

That's a good one because we are not going to see it.


Building a better website for Steve

Or another really good one is don't rely on user input events


Building a better website for Steve

to extend the content on the page, like, for instance,


Building a better website for Steve

lazy loading on scrolling because people might resize


Building a better website for Steve

the window, or they might do something that


Building a better website for Steve

doesn't trigger a scroll event.


Building a better website for Steve

Or if you do on click of a button,


Building a better website for Steve

and then it loads more content, then don't expect us


Building a better website for Steve

to know that.


Building a better website for Steve

So what you're saying is that we are


Building a better website for Steve

going to need JavaScript-specific guidelines, not


Building a better website for Steve

technical guidelines.


Building a better website for Steve

MARTIN SPLITT


Building a better website for Steve

Fine.


Building a better website for Steve

I mean, we could just simplify it to don't use JavaScript.


Building a better website for Steve

GARY ILLYES


Building a better website for Steve

No.


Building a better website for Steve

Yes.


Building a better website for Steve

How do we always end up in this place?


Building a better website for Steve

Let us there, Martin.


Building a better website for Steve

Quite literally.


Building a better website for Steve

JavaScript is great.


Building a better website for Steve

Come on.


Building a better website for Steve

For instance, you want to use JavaScript to make a PWA.


Building a better website for Steve

And we do want to index PWAs as well.


Building a better website for Steve

So that's fine.


Building a better website for Steve

Do we, though?


Building a better website for Steve

I think so.


Building a better website for Steve

I think from the technical side, that's all I can come up with right now.


Building a better website for Steve

Do you have anything else that you would say you would want


Building a better website for Steve

from the technical side of things?


Building a better website for Steve

I mean, don't use plugins, for example,


Building a better website for Steve

because we probably don't want to support plug-ins like Flash.


Building a better website for Steve

MARTIN SPLITT


Building a better website for Steve

Yeah.


Building a better website for Steve

GARY ILLYES


Building a better website for Steve

Because.


Building a better website for Steve

I forgot, because I thought they're dead.


Building a better website for Steve

Even Adobe isn't supporting Flash anymore.


Building a better website for Steve

So why would Steve?


Building a better website for Steve

JOHN MUELLER


Building a better website for Steve

But there could be a new plug-in kind.


Building a better website for Steve

Like, I don't know.


Building a better website for Steve

Like, Flash 2 or Iron Man or.


Building a better website for Steve

I think it's quite unlikely.


Building a better website for Steve

But yeah, that's a good point.


Building a better website for Steve

Maybe that might happen at some point.


Building a better website for Steve

But still, I would say that at this moment,


Building a better website for Steve

we don't want people to use plug-ins to present content.


Building a better website for Steve

So for now, plug-ins should be a no-no.


Building a better website for Steve

So is that like our opinion of what users should see on the web?


Building a better website for Steve

Or is it just a reflection of our technical capabilities


Building a better website for Steve

at the moment?


Building a better website for Steve

It's like, we don't want people to ever use plug-ins.


Building a better website for Steve

Or we can't process plug-ins.


Building a better website for Steve

Therefore, if you want things processed,


Building a better website for Steve

you shouldn't use plug-ins.


Building a better website for Steve

I mean, we are a reflection of what users might want to see.


Building a better website for Steve

Because users are going to sites through Steve.


Building a better website for Steve

So basically, whatever guidelines and best practices


Building a better website for Steve

and policies we come up with, they


Building a better website for Steve

are, in a sense, reflection of our users' ideas of how content


Building a better website for Steve

and sites should be.


Building a better website for Steve

OK.


Building a better website for Steve

So in a kind of like a future world


Building a better website for Steve

where everyone has their 3D headset on


Building a better website for Steve

and the internet is a 3D interactive model,


Building a better website for Steve

then we would try to just index the 3D content like that,


Building a better website for Steve

like users are seeing it.


Building a better website for Steve

That's correct.


Building a better website for Steve

Also, one important thing is that the guidelines are not


Building a better website for Steve

set in stone.


Building a better website for Steve

They are supposed to evolve.


Building a better website for Steve

As the web evolves, we will have to keep up with the web.


Building a better website for Steve

And going back to your example about plug-ins,


Building a better website for Steve

if there is a new plug-in coming out,


Building a better website for Steve

like gold darkness plug-in, then we


Building a better website for Steve

might want to support that because so many people's


Building a better website for Steve

browsers support it that we also need to support it.


Building a better website for Steve

And then we will evolve our guideline


Building a better website for Steve

and change it that, OK, no plug-ins except whatever


Building a better website for Steve

I called it, gold darkness.


Building a better website for Steve

OK.


Building a better website for Steve

But I guess, I mean, from a practical point of view,


Building a better website for Steve

with the way JavaScript has evolved,


Building a better website for Steve

it would basically just be a JavaScript library nowadays,


Building a better website for Steve

not a plug-in for a browser anymore, right?


Building a better website for Steve

Or how do you see that, Martin?


Building a better website for Steve

I mean, for XR, for instance, there's the WebXR API.


Building a better website for Steve

So you can create things.


Building a better website for Steve

If you mix that together with web components,


Building a better website for Steve

like for instance, Mozilla's A-Frame does,


Building a better website for Steve

then you basically call it components, web components.


Building a better website for Steve

Yeah.


Building a better website for Steve

Then you end up with HTML that is somewhat semantic.


Building a better website for Steve

And we might be able to parse that without having


Building a better website for Steve

to run any plug-ins.


Building a better website for Steve

And I think plug-ins historically


Building a better website for Steve

had a hard time on the web.


Building a better website for Steve

And since the invention of the iPhone


Building a better website for Steve

are basically dead in the water, I


Building a better website for Steve

don't see anyone coming up with a plug-in anymore


Building a better website for Steve

that can't be done in a JavaScript API


Building a better website for Steve

or in some sort of web standard.


Building a better website for Steve

Cool.


Building a better website for Steve

OK.


Building a better website for Steve

No more plug-ins.


Building a better website for Steve

That makes things a little bit easier.


Building a better website for Steve

Another thing that I just remembered,


Building a better website for Steve

if we think about gibberish, what about languages


Building a better website for Steve

that we don't understand?


Building a better website for Steve

Like Swiss German, again?


Building a better website for Steve

Like Swiss German, yeah.


Building a better website for Steve

Or like languages that have a completely different alphabet.


Building a better website for Steve

I mean, that goes down to indexing.


Building a better website for Steve

If we can tokenize it and we can, oh, we


Building a better website for Steve

don't understand content, right?


Building a better website for Steve

As in Steve doesn't understand content because it's not


Building a better website for Steve

sentient for now.


Building a better website for Steve

But don't we try to find out what's


Building a better website for Steve

the main topic of something?


Building a better website for Steve

Well, yeah.


Building a better website for Steve

But for that, you don't necessarily


Building a better website for Steve

need to understand the language.


Building a better website for Steve

Fair enough.


Building a better website for Steve

Like if you vectorize the content,


Building a better website for Steve

then you can create a relationship


Building a better website for Steve

between the segments.


Building a better website for Steve

And you don't actually need to understand the different words.


Building a better website for Steve

Because computers don't understand the words anyway.


Building a better website for Steve

It's symbols, basically.


Building a better website for Steve

It's symbols, yes.


Building a better website for Steve

And then the relationship between the symbols.


Building a better website for Steve

So technically, we can support any language whatsoever


Building a better website for Steve

except Klingon.


Building a better website for Steve

I don't want to.


Building a better website for Steve

It would be kind of fun to take some of these science fiction


Building a better website for Steve

languages and see what our systems can learn from them.


Building a better website for Steve

I mean, there are plenty of elvish pages


Building a better website for Steve

out there on the internet.


Building a better website for Steve

And yeah.


Building a better website for Steve

And like Enya, for example, there


Building a better website for Steve

are hundreds of thousands of pages in Enya.


Building a better website for Steve

And you can find them.


Building a better website for Steve

If you search for the words on the page, you can find them.


Building a better website for Steve

Even like approximations, like you


Building a better website for Steve

are using a slightly different word for something.


Building a better website for Steve

Or Google's synonymization is working properly, even


Building a better website for Steve

on elvish.


Building a better website for Steve

So yeah.


Building a better website for Steve

I don't see why we couldn't do the same thing.


Building a better website for Steve

JOHN MUELLER, PhD, MPH,


Building a better website for Steve

I think we should have a brainstorming session at one


Building a better website for Steve

point and just put everything on a whiteboard


Building a better website for Steve

and then shuffle around and put them in the right buckets


Building a better website for Steve

before we actually launch Steve at some point.


Building a better website for Steve

Because at that point, we are going


Building a better website for Steve

to know exactly what kind of ranking algorithms we have


Building a better website for Steve

and what we need to protect and not.


Building a better website for Steve

The simplest ones and the more obvious ones that we mentioned


Building a better website for Steve

in this episode.


Building a better website for Steve

But I'm sure there are many, many, many more.


Building a better website for Steve

This sounds like it's going to be a big rule


Building a better website for Steve

book or best practice book.


Building a better website for Steve

Luckily, space on the internet is almost free, right?


Building a better website for Steve

We will call it the Book of Steve.


Building a better website for Steve

Yeah, the Book of Steve.


Building a better website for Steve

And it's only available in hardcover.


Building a better website for Steve

And then if someone on the internet is misbehaving,


Building a better website for Steve

we are going to throw the Book of Steve after them.


Building a better website for Steve

That seems very threatening.


Building a better website for Steve

I don't know.


Building a better website for Steve

I know.


Building a better website for Steve

That's why I said it.


Building a better website for Steve

Wow.


Building a better website for Steve

OK.


Building a better website for Steve

We had flex, but OK.


Building a better website for Steve

Cool.


Building a better website for Steve

Well, I will not give you my address.


Building a better website for Steve

So because.


Building a better website for Steve

I know where you live, John.


Building a better website for Steve

Oh my gosh.


Building a better website for Steve

Steve knows everything.


Building a better website for Steve

Thank you all for joining us here, folks.


Building a better website for Steve

It was fun talking about our theoretical search engine,


Building a better website for Steve

Steve.


Building a better website for Steve

We've been having fun with these podcast episodes.


Building a better website for Steve

I hope you, the listener, have been


Building a better website for Steve

finding them both entertaining and insightful as well.


Building a better website for Steve

Feel free to drop me a note on Twitter


Building a better website for Steve

or chat with us at one of the virtual events when we go


Building a better website for Steve

if you have any thoughts.


Building a better website for Steve

And of course, don't forget to like and subscribe


Building a better website for Steve

and follow our guidelines.


Building a better website for Steve

Thank you and goodbye.


Building a better website for Steve

Auf Wiedersehn.


Building a better website for Steve

Goodbye.


Building a better website for Steve


English Google Webmaster Central office-hours from January 10, 2020

OK.


Welcome, everyone, to today's Webmaster Central Office Hour


Hangouts.


My name is John Mueller.


I'm a webmaster trends analyst here at Google in Switzerland.


And part of what we do are these Office Hour Hangouts,


where everyone can kind of join in and ask their web search


website-related questions.


And we can try to find some answers, hopefully.


A bunch of things were already submitted.


So we can go through those during the session.


But if any of you want to get started with a question,


you're welcome to jump on in now.


Well, everybody seems silent.


I'm out there.


Go for it.


Great.


I'll turn on my camera so it's a bit more personal.


Good morning.


Thank you for hosting this.


Yeah, we've tried to adhere very strictly to all the structured


data guidelines.


I run a website that offers a comparison of courses,


so education courses, e-learning, things like that.


So we've migrated, basically, from using education event


as our main data type to course, which was newer and felt like,


oh, that's the more correct thing.


We've implemented it all over the site.


And then we lost all our data snippets that would list,


OK, this course has a starting date on this day,


in that city, et cetera.


We used to have them while we used education event.


And now, of course, they have disappeared.


And now it's been about three months,


and they're still not back.


So I've been asking around.


And then one of the replies from some supposed expert


was, well, course isn't really live yet in the EU.


It's launched in the US first.


And I haven't seen it anywhere in the EU yet.


And after doing some research, I concluded the same,


that I don't see any sites here offering it.


So can you offer any guidance on what we could expect


or how we can research it?


Good question.


I don't know if that rich result type is specifically


launched for European sites yet.


In some cases, it is the case that we launch first in the US


or in some other country, and then we


try to figure out what the right approach is,


how much we should show, how we should handle this,


and then we start to expand from there.


I don't know if that's the case with course at the moment.


But I can double check with the team


to see if there's something that we


can do to kind of speed things up so that those who


have been implementing it are shown appropriately.


I tried it out, I think, a couple of days


ago to get some screenshots for a presentation.


And I believe we showed the course results in Europe,


but it might be just that we don't show it


from European sites yet.


Right, yeah, that's the thing.


Yeah, I'd love to see examples, because now I've


been looking at competitors or other ranking for queries


where we are ranking.


And they're all still using event and education event.


So we feel like the best boy in class who do the new thing


and then don't get the snippets that they get.


So I'm really looking for examples as well.


Or we'll just migrate back and wait for reality


to set in or something.


Yeah, sometimes these things take quite a bit of time.


Yeah, yeah.


Something where I like to have it so that we roll these out


as quickly as possible worldwide.


But sometimes it takes several months or even a year or so


for everything to be finalized.


Sometimes there are things around policy


and what we're allowed to show that play into that as well,


which makes it a little bit trickier.


I don't know if that's the case with courses.


Yeah, makes sense.


I can imagine you'd also wait for webmasters


to actually implement it, because otherwise you're also


doing something that nobody does.


So is there a logic there in your experience?


That helps a little bit in the beginning.


But usually with the rollouts, when we roll them out


in other countries as well, it's less of an issue,


because the markup has been documented for quite some time


already.


People have been using it on and off anyway.


So that's something where usually the adoption itself


in specific countries is less of an issue.


OK, cool.


But you said you could check it out at least


to see if there are examples.


And who knows?


I can double check with the team to see if there is anything


that they would find useful to speed things up.


Yeah, great.


Well, I appreciate it very much.


Thank you very much.


I look forward to that.


Which country are you in?


Or which country is your site focusing on?


Main sites, Netherlands.


Secondary is Germany.


And both haven't shown anything.


And all competitors are still using the old stuff.


So it's also very hard to see if we're doing the right thing


here.


That's a problem.


So those are the main markets in UK is third,


but less important.


OK.


Yeah.


Thank you.


Cool.


I'll check with the team on that.


And then you would normally update in the community


on YouTube as well?


Or is there a different?


If there's something specific that I can bring back,


I'll try to put it on Twitter.


We can check there.


But a lot of times, it's just that we give feedback


to the team.


And then they continue working on it.


There's nothing special that we can tell.


If you happen to spot an example of a site using


course in the wild in the EU, and they do get snippets,


I'd love to see it as well.


Then I at least have some proof to my team that it might work.


And it's just a matter of time.


So if you happen to stumble across an example,


that would be really cool.


Or if you know.


So this is just totally random, because I


was looking for screenshots to put into a presentation.


Yeah.


And I think there was something like Android developer courses


or something like that, where from Coursera,


we were showing the kind of the course rich result type.


Makes sense.


Yeah, I've seen US sites get it.


But you've seen it from the EU, because you're in Berlin.


Well, yeah.


I mean, I saw it here in Switzerland


when I was looking for screenshots.


But I don't know if there are any European sites.


OK.


I'll for sure check that to see something in the wild.


Thank you very much.


Great.


All right.


Any other questions before we get started?


I'm sure they'll come over time as well.


So OK.


Let me go through some of the questions that were submitted.


There's a site out there going down the similar road


that another site once did, and is building


a massive amount of backlinks to specific pages on their site


with links in their badge code.


While they're not paying for links, if you link to them


without their code, they start bugging you


for a link, practically demanding


that you link to them.


There's no anchor text in the link,


but they're targeting specific pages on their site.


Would you consider this aggressive link building


tactic to be a link scheme, which could get them penalized?


It's possible that that would go down


the road of being a little bit over the top.


So that's definitely a possibility there.


In general, what's important for us


is that if you create widgets, for example, on your site,


and you give them to other people to put on their site,


and they do contain a link back to your site


as the source of traffic, or the source, not source of traffic,


but source of the data, then we recommend


using nofollow links for that, just


to make sure that when you're giving things out


to other people that it doesn't come across as this trade of,


we give you the widget, and in exchange, you give us


a followed link back to our site.


As soon as it becomes an exchange, where


you're swapping things like that,


then that would be a link scheme from our point of view.


So that's essentially what we would look at there.


And the web spam team does take action on a lot of these,


and it's something that our algorithms also


take into account as much as possible.


So I wouldn't assume that this kind of setup


is something that I wouldn't recommend it


from a practical point of view.


I wouldn't assume that these kind of links


are something that will remain valuable over time.


Instead, I'd really make it so that if you


find people who like your widget and they really like it,


then see if they'd be willing to give a normal natural link


back and not kind of like force it on them through the widget


itself.


News website story.


We automatically make links to tag pages


in the text of news articles.


For example, the anchor Donald Trump


leads to a page with lists of news about the US president.


Is there a risk that Google will consider this to be spam?


No.


I don't see any problem with that.


These kind of tag pages are really common.


That's a setup that a lot of sites have.


It's something where you're essentially


linking within other pages within your own website.


And if you're able to provide more context for the pages


that you're linking to, then that's generally a good thing.


So from that point of view, I don't see any problem with it.


The main thing I would watch out for here


is that we don't generate a massive amount of tag pages.


So essentially, it's just a collection of tag pages,


and the actual content is very minimal.


So use these in a way to create more clear category pages,


if you will, and less in the sense of automatically


generating every combination of words as pages.


So that's the direction I would head there.


Company is preparing for site migration,


where the old website.com is 301 redirected to new website.com,


so root domain change.


However, can you advise if there are any SEO repercussions


if we do not redirect a handful of paid digital URLs?


They want to continue to drive paid traffic


to various old website.com paid landing pages.


While I'll set up one-to-one 301 redirects,


I plan for a catch-all just in case.


I'm worried that Google Search Console will deny the request


for a site move if we continue to drive paid traffic to some


of the old website URLs.


So I think, first of all, the site move request,


the change of address in Search Console,


won't be a problem because we just look at a handful of pages


to see if they're redirecting.


So that kind of submission there should be no issue.


The main thing I would watch out for here


is really that you're moving all of the pages


from the old domain to the new domain.


I'm not completely sure how the setup was


that you're looking at there.


If you're looking to keep some pages on the old domain


and drive traffic to them, or if you're just


driving traffic to the old domain URLs


and then it redirects to the new domain URLs.


If you're keeping pages on the old domain that are not


redirecting, then that's something


that will confuse our systems.


Because when we crawl and index your website


and we run across those pages, then we


start to question whether or not this is actually a site move.


And then it essentially goes from being a move from one


domain to another, where we can just transfer everything


to the new domain.


It becomes a situation where you're splitting a domain.


You're taking one domain, you're keeping some things there,


and moving some things somewhere else.


And from our point of view, that becomes a lot harder


to process.


That essentially means we have to reprocess


both of these domains and see, well,


what is this new website?


And what is on this other website


that we already know about?


And how does a content interact with itself there?


So that's something where if you're


looking to do a site migration and you


want to keep some URLs on the old website,


then that's something where you really need to be aware


that this will take a lot longer to be processed


than if you really moved everything over


to the new domain.


And if it's not the case that you're


keeping URLs on the old domain, but rather you're


just driving traffic to the old URLs


and they're being redirected to the new URLs,


then that's totally non-issue.


People come and visit the old domain from various sources.


If they get redirected to the new one,


then we see that's a part of the site move.


That's absolutely no problem.


It's really just like keeping things on the old domain.


That's something which I would try to discourage


as much as possible.


Does Google still respect the meta robots unavailable after?


Yes.


Yes, we do.


So the unavailable after is a way


of specifying a date when a page is no longer available.


And we do use that.


We even made some, I believe, some changes there fairly


recently with regards to the date formats that we support.


So that's something that we definitely still use.


The thing to keep in mind here is you're essentially


telling us this page will not be available then,


but we need to make sure that we're not just


removing pages from the index that


are actually still available.


So it's very likely that our systems


will go and recrawl that page around that date


that you specify and watch out for no index


or watch out for a 404, which will tell us, well, actually


it is no longer there.


So make sure you're backing up anything


with an unavailable after tag together with actual guidance


to tell us that this page is really not available anymore.


In the HTML source code of an internal page,


canonical tag is mentioned of the home page.


And while doing inspect element on the same page,


I found the canonical is self-referential internal link.


And the URL inspection tools sometimes


shows user declared canonical as self-referential URL


and sometimes the home page canonical.


So which canonical URL will Google consider?


So it sounds like you have a page that is not the home page


and it has a rel canonical pointing to the home page.


And sometimes it doesn't have that rel canonical


on the home page.


That's something which would confuse our systems,


because sometimes you're telling us


to use the home page instead of that page,


and sometimes you're telling us to use that page itself.


So as much as possible, I'd recommend


making sure that the rel canonical stays consistent


and matches the URL that you do want to have indexed.


If this is a case of a JavaScript page


where you're using JavaScript to change the rel canonical,


then that's something where I would recommend making sure


that the static HTML page that you deliver


doesn't have any rel canonical on it,


so that you're not changing the rel canonical,


but rather that you're adding it.


If you use JavaScript to add a rel canonical to a page,


then that's perfectly fine.


We can pick that up after rendering.


We can process that and use that.


However, if you're using JavaScript


to change the rel canonical from one URL to another,


then you're potentially in that situation


where, depending on if we process JavaScript for that


page or not, you're giving us very different information.


So that's something I would try to avoid.


So ideally, if this is a JavaScript site


within the static HTML page, don't


have a rel canonical on there, and then with JavaScript,


add a rel canonical to that.


There are some similar cases where you also


need to watch out for this.


For example, if you have a no index on a page,


on a static HTML page, and then you


use JavaScript to remove that no index,


then depending on what we process,


we might see a no index or you might not see a no index there,


and that can get confusing.


And especially if JavaScript removes the no index,


then it's very possible that we will take the page,


see that there's a no index, and say, oh, we


don't need to do any rendering here, and we'll just drop it.


So if you're using JavaScript with regards to the no index,


then I would only use it to add a no index


so that it drops out and not remove a no index.


Will you tell me about the Google BERT update?


Which types of work can I do on SEO


according to the BERT algorithms?


So this is, I guess, a fairly broad question.


I would primarily recommend taking a look at the blog post


that we did around this particular change.


In particular, what we're trying to do with these changes


is to better understand text, which on the one hand


means better understanding the questions or the queries


that people send us, and on the other hand,


better understanding the text on a page.


And the queries are not really something


that you can influence that much.


As an SEO, the text on a page is something


that you can influence.


And our recommendation there is essentially to write naturally.


So it seems kind of obvious, but a lot of these algorithms


try to understand natural text.


And they try to better understand what topics


is this page about?


What special attributes do we need to watch out for?


And that would allow us to better match the query


that someone is asking us with your specific page.


So if anything, if there's anything


that you can do to kind of optimize for BERT,


it's essentially to make sure that your pages have


natural text on them, and that they're not


written in a way that kind of like a normal human


would be able to understand.


So instead of stuffing keywords as much as possible,


you can kind of write naturally.


In terms of site links, what influences,


which links are picked up to show,


the order of the navigational internal links,


breadcrumb markup.


So site links are the kind of sublinks


that we sometimes show in the search results, where


if you're searching for something, we show one page.


And then you have smaller links to other parts of that website


shown there.


Essentially, site links are normal search results


from our point of view.


But we try to kind of figure out when we can pull those out.


And something that really helps us to understand


when we can pull those out is if you have a really clear site


structure, and if you use clear headings and titles on pages


so that we understand what these pages are really about.


And then with a real clear site structure,


we can kind of understand that if someone


is looking at the home page or looking at one specific page


within your website, then these are


kind of like the four or five related pages on your website


that are really commonly used and that might make sense


for this particular user.


So in terms of that, that's kind of the direction


that we would go there.


And I believe we have that documented in our Help Center


as well, that for site links, things


like the navigation, clear internal structure


do make sense.


What happens for brand sites with products that are only


sold via third parties?


There is no price that we can add as the price varies,


for example.


For example, a brewer, the price of a pint of beer


varies by location.


What would you suggest?


So this one I think is kind of obvious in the sense


that if you have products and you don't have any prices


that you can show, then you can't mark up any prices.


And what practically happens here


is you can use the product structure data


markup for these kinds of pages.


But for us to be able to display the product structure data


markup, I believe we need either reviews


or a price.


And if you have neither reviews or price on your pages,


then we will just not show that as a product rich result site.


So from that point of view, you can


continue to use that structured data markup.


We just won't show it.


And with regards to, well, what can you do,


there's not really a workaround for that in the sense


that if you don't have a price and you don't have reviews


and you don't have those things, so that's something


we just wouldn't show.


From a ranking point of view, this is not really a problem.


It's really just how that search result is shown in the search


results pages.


I see a lot of WordPress sites that


have different schema on their desktop, mobile, and AMP sites.


For example, I'm looking at one that


has websites on the desktop pages, organization,


and blog posting on the mobile pages,


and HCard and news article on their AMP pages.


That sounds complicated.


Is it safe to assume that Google is using the schema that's


on the version of the site that's being crawled


as per Search Console?


For example, if mobile crawling is indicated,


that's the schema that it's using?


Yes, for the most part.


So within the desktop and mobile pages,


we would pick whatever version we're currently crawling.


If we switch to mobile-first indexing for the site,


then we would not use the desktop version


at all for indexing, and that includes the structured app.


So that's kind of the easy part.


For AMP, it's a little bit different in that


for some AMP features, certain structured data types


are required on the AMP pages.


So we take that into account for those AMP pages


when they're kind of eligible to be shown for specific AMP


features.


So that's kind of the complicated part there.


In general, I would strongly recommend making sure


that you're consistent with your structured data,


that you don't have different types on different types


of pages across your website.


It's not so much that your website will be seen


as low quality or be bad if you do this,


but rather it's really just from a practical point of view.


If you have different types, you don't


know what you're getting out of it.


So if there is a way for you to kind of double-check


the structured data types you have


on the different types of pages and to be


consistent across those pages, then it's


something where we can look at any of these pages


and we get all of the information that we need,


and then we're definitely going to be happy.


Whereas if the information depends on the type of page


that we're looking at at the moment,


then you can't really rely on us being


able to take your website into account on its whole.


So it makes it harder for you to diagnose,


why isn't Google showing you a specific structured data type?


And then you're like, well, on this specific variation


of my page, I have it.


Then it's very possible that maybe we're not looking


at that variation of the page.


Or especially if you have different variations


in different places, sometimes the combination


is what makes it important for us.


And if we don't see that combination clear,


then we can't take that into account.


So that's something which, purely


from a practical point of view, I'd


recommend trying to focus on.


It's really hard for us to flag these kind of issues


in Search Console, because when we just look at one type


of page, we don't know what we're missing,


because we don't actually look at the other types of pages.


So that's something where potentially there


are third-party tools that will help you to crawl your website


and pull out the different structured data types.


And maybe there is a trick to flag


these kind of inconsistencies across variations


of the same page.


If anyone knows any tricks to do that,


it'd be cool to hear from you.


Maybe that's something we can share as well.


My client site is still being indexed by regular Googlebot


and not switched to mobile-first indexing.


Also, this site is experiencing great traffic loss now.


What could be the reason?


Or could this be the reason the site


was migrated to the current domain,


and the old domain was switched to mobile-first indexing,


the migration was in August, and the site is mobile-friendly?


So I think there are a few things here


that are kind of combined.


On the one hand, mobile-friendly doesn't


mean that a site is suitable for mobile-first indexing.


Mobile-friendly is kind of that mobile users


are able to use that site without any hindrance.


And mobile-first indexing requires that we actually


are able to index the content in an equivalent way


across desktop and mobile.


So for example, a site could be mobile-friendly if it just


has very minimal content.


But it might not be ready for mobile-first indexing


if that mobile version doesn't match what we


have indexed for the desktop version.


So that's kind of one thing to keep in mind.


Mobile-friendly is different from mobile-first indexing.


In general, with regards to traffic loss,


it's not the case that if a site is still in the desktop index


that it would get less traffic.


So the ideal situation with mobile-first indexing


is we switch sites over to mobile-first indexing


when we believe they're ready.


And when we believe they're ready


is essentially a simplified form of when


we think there is no change in the traffic


to the site when we use the mobile version for indexing.


So in that sense, you wouldn't see a drop in traffic


when we switch to mobile-first indexing


or if we don't switch to mobile-first indexing.


If you are seeing a drop in traffic,


then that would be due to any number of other reasons.


Could be from the site itself.


It could be changes in our algorithms.


It could be changes in user behavior


and that they're looking for different things now.


That would be completely separate from mobile-first


indexing.


When you use a site query for the old domain,


Google still shows a lot of pages.


And there are only 50 pages in Search Console


for the old site, mostly PDFs.


But the weird thing is Google shows


a search result of the new site when you use a site query.


So that part is completely normal.


That's something that confuses especially SEOs, I think.


And it's something that we tend to do more for normal users


in the sense that, so what is practically happening here


is we understand that the site has moved from one domain


to another.


And we've essentially moved all of the URLs


from indexing to the new domain.


However, we still understand that connection


between the old domain and the new one.


So if you explicitly ask us for the old domain,


then we'll say, well, we have some pages


that we know are associated with this old domain.


We'll show them to you in the search results.


So with a site query, you often see that.


Often, you also see that if you just search for the domain name


itself.


And from a practical point of view,


this is not a sign of anything going wrong.


This is essentially just us saying,


well, we know there is this connection here.


And these are some of the old URLs.


And if you look at the cached page for the pages


that we're showing, you'll see that we're actually showing


the pages from the new domain.


So we're essentially showing something


that is no longer indexed like that.


But because we believe the user is explicitly looking for that,


we show it to the user.


So that generally wouldn't be a sign


of anything problematic there.


You mentioned the URLs here as well.


So I can double check that.


I took a quick look before the session,


and I didn't see anything that popped out.


So my guess is this change in traffic that you're seeing


is essentially a natural change in traffic, not something


that is due to the site move or due to anything


around mobile-first indexing.


It's something where I try to focus more


on the actual content itself and try to figure out,


what could you be doing differently


with regards to the content, with regards


to maybe some of the different changes


that we've talked about over the past year or so?


AUDIENCE 1 John, hello.


Thanks.


It was my question.


I just wanted to clarify one thing.


Do sites which were switched to mobile-first indexing


have some benefits in comparison with my site?


Not really.


So the benefit is mostly for us that we're


able to index the version of the site


that users mostly would see.


Because most users use mobile devices nowadays.


So it's more that we're consistent,


but it's not that there is any kind of ranking bonus


for being in mobile-first indexing.


OK, also I wanted to say that this site has more than 80%


of mobile users.


So I don't see the reason why Google is still indexing


with desktop user agent.


I could imagine that if you did a site move late last year,


like you mentioned there, that our systems are just


being on the cautious side.