I'm asking this because I read the How Google News Works


English Google Webmaster Central office-hours from July 21, 2020

document.


And there was a line in there about the audience


and how an audience that turns to the site for one topic, that


could pass off an authority signal for that site's,


I guess, being authoritative or being an expert in that topic.


Because so much of the users who are interested in that topic,


I guess, rely on the site or trust the site.


So that's kind of where I'm getting at.


If the audience is, if I'm publishing


a lot of different topics on one big site


and I have audiences very fragmented,


is that something that could maybe


disadvantage my site overall?


So I don't know about Google News.


I don't know if they do something special there.


But in general, from a web search point of view,


I don't see a big problem there.


The one place where I would watch out for this


is if you have a mix of adult content and general content.


Then that's something where a safe search algorithms might


have trouble understanding where the separation is.


And then putting it on a separate subdomain


makes it a lot easier to say, this


is the content that can be filtered by safe search,


and this is the general content.


That's a lot easier.


But kind of different themes, like sports and politics,


I don't think, for the most part,


it would make sense to separate that out


into separate subdomains.


SPEAKER 2 OK.


Perfect.


Thank you.


SPEAKER 3 John, can I ask another question?


If you have time for it.


So I have a follow up on that about the safe search part.


Let's say, in a case that you have Chrome,


that you get a lot of spammy links using


porn-related anchor tags.


And we are seeing some scenarios that people are actually


coming to our sites and writing down on comment section


adulty words.


And they're also sending links using, you know.


And is there any way that it's going to harm us in a long way?


Because we know, based on documents,


safe search filter is supposed to be on a URL basis.


But are there any sort of, I don't know,


domain-wide algorithms that might affect the whole site?


So in particular, when it comes to things like safe search,


we try to be as fine-grained as possible with the way


that we deal with this.


But it's not always possible.


So it's something where, if we can't isolate


the part of the site fairly easily, automatically,


then we quickly end up in a situation where we say,


well, safe search applies to the whole domain.


So that's something I would definitely


watch out for with regards to people writing things


as comments on your website.


Keep in mind, we do try to understand which parts


of the websites have comments and which parts of the pages


have comments and try to understand that.


But ultimately, the content that you're publishing


includes these comments.


So if you don't want your website


to be found for things like that,


or if you don't think that these comments are providing value


to your website, then that's something


I would take action on.


Because ultimately, it's your website.


And if other people are contributing content


to your website and you're saying, oh, this is fine,


then you will have to live with the situation


that search engines think, well, this


is a part of what you want to stand for on the web.


And maybe we will send you traffic for these comments.


Or maybe we will look at this content


and say, oh, this is kind of gibberish.


We don't know what to do with this website overall.


So that's something where I wouldn't completely


ignore random user-generated content on your website,


kind of the comments that you get like that.


Obviously, it's hard to draw a line and say, well,


I will approve manually every comment


that is made on a website.


If you have tons of comments, then that's


essentially impossible to do.


But you can kind of work to figure out what kind of comments


are useful.


Maybe let users flag comments so that you can manually


review those, or maybe have something like a bad word list


that you try to use in a simple way to figure out,


are these comments actually useful,


or is this maybe something I don't want


to have published on my site?


AUDIENCE 1 The noindex watch, in this case,


the noindex for those pages, the list for?


JOHN MUELLER Sure.


If you noindex those pages, that's kind of a big hammer,


because then the whole page is gone.


But if it's noindex, then we don't use that for indexing.


Thank you.


OK, some more questions.


Most of the important questions answered by Matt


are a decade old, and a lot has changed since then.


Is it possible to have them answered again?


Sure.


I mean, we've been recording lots and lots of videos,


so I'm happy to answer more questions.


But let us know what kind of videos


you think would profit from a modern take,


because we have gone through a lot of those older videos


from Matt, and a lot of the things, they still stand.


There are lots of timeless answers


there with regards to how you can improve things


for your website, particularly around search.


And a lot of these things don't change that much over time.


So just because something is old isn't really a reason for us


to say we need to delete it and do it again,


because it's like having a fresh face on a video


makes things so much better.


I think Matt did a fantastic job with a lot of these videos.


But like I said, if you run across any


that you think definitely need to be replaced


or definitely need to be taken down because they're


no longer relevant, by all means, let us know.


Are rich results from the structured data


markup always on the first page of the search results,


or can they be shown beyond the first page?


They can definitely be shown beyond the first page.


That's something that's not tied to the first page of the search


results.


Some particular kinds of elements in the search results


are sometimes more visible on the first page,


but all the different rich results types, as far as I


know, can be shown on any page in the search results.


Hey, Sean.


Follow up on that.


That is my question.


So a lot of times I see that we go to the GSC, Google Search


Console.


I see something like they have rich results,


but it's in like 50 position or something.


But they have very high impression.


Like, their impression looks like it's on the first page.


So what does that mean in GSC?


I think that's really confusing sometimes.


And I don't know what it means in your particular case.


But the tricky part with the search results page


is it's not just like those 10 links below each other anymore.


We have all of those one boxes and sidebars


and different navigational elements on a page


where it's easy possible that you have maybe 20, 30, maybe


even some paces 50 links that are shown on one search results


page.


And they will all see impressions.


They will have kind of the position number


in Search Console.


And that can sometimes look a little bit confusing.


So sometimes we see it that it looks


like something is on the first page,


even though it's position 20 or 30 or 40.


Sometimes we see things that we say,


oh, this is position number two.


And you look at the page, and it's like, where is my website?


It's not showing up at all.


And it ends up being maybe one of those images


in a thumbnail on top, which for Webmaster is like,


well, technically it's a link to my website.


But practically, it's not as visible as a big text block.


So it's sometimes tricky to, I guess,


to understand what is shown in that search performance report.


And this is something where you also see differences


across different kind of ranking,


checking tools if you use anything like that,


because everyone has a different opinion on how you should


count the position on a page.


And I don't know what the final answer there will be.


But it's also something that we're


looking at from time to time to see,


how can we make that position number a little bit more


useful for people so that they understand what


does this number actually mean?


Is my site visible?


Is it very visible or just slightly visible on a page?


It's sometimes hard to judge.


SPEAKER 2


Or interesting finds or something,


and then there is a load more.


So if you only rank after you hit the load more,


does that still count as average position one


or whatever the actual knowledge panel's position


is in Search Console?


That's only counted when people see your site in the search


results.


So it's not, well, not when they see your site,


but rather when your site is loaded with a link


in that search results page.


So if you have to click the Load More button in order


for the next part of the links to be shown,


then that wouldn't count as the first page.


That would only count when someone actually


clicks on that.


SPEAKER 1 Okay, so the position would be much lower,


or would it not?


How would that appear, I guess?


SPEAKER 2 It would be higher.


It's kind of like going to page two.


SPEAKER 1 Okay.


Okay, thanks.


SPEAKER 2 What are the recommendations


from Google for indexing and ranking audio?


Nowadays, some publishers produce audio, just


have two main ways to index and rank this content, Google


Podcasts or embedded in a news article surrounded by text.


What are some options?


Are there any plans to give more visibility to audio in the


search results?


So I can't say much about the plans.


On the one hand, I'm not aware of anything specific being


planned there.


But we also generally try not to pre-announce things.


With regards to audio in general,


I think the best approach is really just


to work with transcripts.


And especially if you're doing something like a podcast where


people are talking, then if you have a transcript,


then that's something we can look at and see from a web


search point of view.


Here's a bunch of text.


We can rank this page based on this text.


So that's kind of the main approach I would take there.


It's something where I imagine at some point in the future,


we'll be able to go in there and do voice understanding of what


is being said and index the audio content based on that.


But I don't think that will happen


within the next couple of weeks or anything like that.


So making sure you have a transcript, text,


together with the links to your audio files,


that's probably what I would aim for.


The structured data testing tool will be greatly missed.


Are there any plans to expand the scope of the rich results


testing tool?


Yeah, I know lots of people have been


very vocal about wanting the structured data testing


tool to remain.


I think it's something from a technical point of view,


it makes sense for the team to focus on one tool rather than


two tools.


And one of the pieces of feedback we've


received with structured data in general


is that it's sometimes hard for people


to understand which types of structured data


actually have an effect in search.


And that's why we focus on the rich results


tool, which focuses on the things


that we would show as Google in the search results.


So that's kind of the background there.


We are planning on expanding the rich results testing tool.


We've been looking at all the feedback


that we're getting where people are like, oh, this is


like terrible Google because this one use case I have only


works in the structured data testing tool.


And we try to take that kind of feedback,


understand what it is that people are trying to do,


and to make sure that we can implement that


with the rich results testing tool.


Also, the structured data is not going away just now.


So it's not that you urgently have to jump over.


But by all means, make sure to send us feedback


if there's something specific that you


will miss if the structured data testing tool goes away.


And try to make that feedback as actionable and as


understandable as possible.


Sometimes we just get feedback that's like, oh, Google,


you're terrible.


You shouldn't do this and take away our toys.


But you don't tell us what we should change.


And we can't really work with that kind of feedback.


So make sure that the feedback is something


that we can take to the team and say,


look at this smart person here with this smart piece


of feedback about their specific use case.


Makes total sense.


We should make sure that this works.


So that's kind of what we would like.


Hey, follow up on the Google use schema they don't.


So is Google actually use it?


Or is that any benefit?


And also, generally, if there is some schema


mark up in schema.org website, but it's not


listing the Google Developer Guide,


can it be benefited in any way in SEO specifically?


Yeah, so I think one of the trickier questions


with regards to all of the structured data


in that we have a lot of things that we


use to try to understand the page and the content


on the page that we don't necessarily show directly


in the search results.


So in the rich results test, we focus


on the things that actually have direct visible effects


or can have direct visible effects.


But a lot of things help us to better understand


the content and the context of a particular page.


And those are things within kind of like this general schema.org


markup, which you can do various things.


And that's kind of, I'd say, almost a shame


that we don't highlight that in the rich results test.


But it's also something where it's very easy to go overboard.


Because there are just so many different things


you can mark up with schema.org.


And you can spend a lot of time marking up


all of those individual elements.


And there's absolutely zero effect on your search results,


even if we were to process that.


So if there are things where you feel we desperately


need to understand the relationships of the items


on your page is a little bit better, then


go ahead and add that markup to the page.


But if it's just like, oh, there are five different types


of schema.org that could apply to this page,


therefore I'll mark it up, that's


probably not providing any value.


So a really common use case is to mark a page as a web page.


We do that on the Google.com home page as well.


So it's not like us making fun of everyone else.


But it's something where, as a search engine,


you look at that web page, and it's like, it's a web page.


And it says it's a web page.


What else could it be?


It doesn't give us any extra value.


So finding things that are really high level,


important for your pages, and adding the markup for that,


even if it's not visible, I think that's perfectly fine.


Going overboard and just adding all types of markup


that you can find, that's not going to harm your site.


But it's kind of a waste of time.


SPEAKER 2


Hey, John, I have a quick mobile-first indexing-related


question for you.


I've got a moment.


So I manage a lot of sites.


They're in the mobile-first index.


And generally, they get crawled at about 90% mobile, maybe 10%


desktop crawler.


One of the sites, and again, it's


an adaptive platform rather than fully responsive.


So there are some differences.


I have one site in particular.


It's my larger sites.


It's getting crawled about 40% desktop, 60% mobile.


Does that create problems?


Again, it is in the mobile-first index.


Is that a bad thing?


Is that something I should be working to fix?


Can you explain a little bit about that?


SPEAKER 1 I don't think that would


be a sign of a bad thing.


My hunch is that it depends a little bit


on the type of content that you have on the site.


So for example, I'm not sure if for Google Shopping,


we would use the smartphone Googlebot


or if we'd use a desktop Googlebot.


So if it's an e-commerce site that's


visible in Google Shopping and we crawl shopping results


with the desktop Googlebot, then you'll


probably see more crawls with the desktop Googlebot


in a case like that.


I'm not 100% sure if that's the case with shopping.


It might also be similar with the ads bot, the landing page


check, those kind of things.


But depending on the type of content,


we might be using slightly different requests on our side


to crawl the site.


And that could result in a slightly different skew


of desktop versus mobile when it comes to the overall crawling


when you look at your log files.


Cool.


Thank you.


Follow up on e-commerce very quick.


So Martin, a while ago, Martin said


that Google Merchant Center bots don't render JavaScript.


So in that case, if I switch my product page schema markup


from HTML-based to JavaScript, will it


still be able to show up in rich resulting shopping related


term?


I don't know.


I don't know what specifically Martin said there.


I would check with Martin.


Especially when it comes to rendering,


he knows everything there.


I see.


Thanks.


I actually already asked him, but he don't know anything


about Merchant Center.


Oh, OK.


Yeah.


So I think everything around shopping


is something that is kind of sneaking up on us on the web


search side because a lot of the shopping things


are now kind of being merged a little bit


within the organic search results,


and they're no longer kind of this separate paid experience.


So I imagine over time, we'll have a little bit more


information to share around that.


OK, thank you.


Cool.


OK.


We didn't get through a ton of questions,


but if there's anything in the questions


that you really need to have answered,


feel free to drop that into the next Hangout.


I need to jump off to another meeting,


so I can't stick around a little bit longer,


but maybe on Friday, we'll have a bit more time.


Thank you all for joining in.


Thanks for all of the fantastic questions from you all.


I hope you found this useful, and hopefully, we'll


see each other again in one of the future Hangouts.


Bye, everyone.


Thank you, John.


Bye.


Thank you.


How to improve First Input Delay for a better page experience

You never get a second chance to make a first impression.


That's why perfecting your first input delay is key to a great page experience.


Isn't it frustrating when you're trying to browse a site, or use an app, or just


watch a video?


See what I mean?


Delays interrupt us.


They break up our thought process, and make it harder to focus, or just complete our goals.


That's why it's so important to make sure that people using our site have as few delays


as possible.


One of the ways that page experience helps reinforce this idea is through the metric


First Input Delay, or FID.


FID is a new web API that you may have heard of last year, as it was a part of the inaugural


group of Core Web Vitals.


First Input Delay measures how long it takes, in milliseconds, for your site to react after


someone interacts with it.


It's like when they click on a button, or if they tap on a text input.


That means that we're measuring the delay in processing data and interactions, not the


time it actually takes to process that data.


You can think of it as how long it takes for your page to feel like it's interactive.


It's being responsive to the user the very first time that they use it.


And just like every part of page experience, each and every one of the URLs on your site


are scored on First Input Delay independently, directly based off of the actual users using


your site.


Your homepage may have fantastic values, but your product page or articles could be a little


bit far from perfect.


Regardless of how any page is performing, though, neither result impacts how another


page is performing on page experience.


Whatever URL is an island.


However any of your pages are scoring, you can be sure that the numbers that you see


reflect what users are actually experiencing.


That's because the FID scores are coming from the monthly Chrome User Experience Report.


You can read more about this report in the links below.


Now that we have some ideas as far as what FID is, how do we go about finding out how


your pages are actually doing?


Since FID is inherently about how your site reacts to interactions, it requires real users


to get an accurate measurement.


As a result of this, you can only really rely on user measurements that come from real-world


users, either via your own analytics or the ones that you have the ability to access inside


of the Core Web Vitals report section of the Search Console.


If you haven't already, you'll need to create a free account to access the Search Console,


but I highly recommend it.


You will get some invaluable information about how your site is performing with page experience


as well as a lot of other foundational pieces to creating fantastic modern websites.


You can quickly check out how your pages are performing with PageSpeed Insights.


This is a hosted online version of the same Lighthouse tool that's a part of the Chrome


DevTools.


It shows you how your pages are performing in a test environment that's configured to


be around the connection speed and device performance that's going to be average for


a web user today.


Your URL needs to be online for over a month in order to be included in the Chrome User


Experience Report.


If it has been, you can see the actual Core Web Vitals results directly.


So you've got a number.


Now what?


Like all Core Web Vitals inside of page experience, there's no such thing as passing or not passing.


These are all used as guides.


Search is using them to create relative performance differences between URLs to see how they're


performing against one another.


Generally speaking, a good goal is to have about 75% of your site's visitors have a first


input delay of less than 100 milliseconds.


100 milliseconds has long been considered the point at which people perceive user interfaces


as behaving instantaneously.


Anything in the 100 to 300 millisecond range is starting to become problematic.


And anything beyond 300 milliseconds is a noticeable delay and will likely cause a worse


experience for your users.


One thing worth mentioning is you are not guaranteed to have an FID value.


Since it's inherently based off of a user interaction, if your page doesn't have anything


to interact with, or people just don't ever click on buttons or inputs, there won't be


an FID value recorded.


If your pages are new or your URLs haven't been included inside of the Chrome User Experience


Report, you can still figure out what your FID results are.


Just takes a little bit more effort.


Like I mentioned before, all of the field data and page experience comes from your actual


real-world users.


That means that we can figure out the FID ourselves using the same APIs.


In order to do this, we need to add a little bit of JavaScript to our page.


All of the Core Web Vitals are available to us via JavaScript APIs, specifically through


performance observers.


These observers give us an event or list of events that match the type of performance


events that we're interested in.


In this case, we're going to be looking for first input.


Once the browser determines a first input has happened, it will execute the function


that we pass it, passing us the list of events that match what we requested.


Though first input will likely only ever have one event, the performance observer always


passes us a list of events to our callback, so our code iterates over the responses by


calling get entries.


For every entry, we look at the processing start and subtract the start time.


The difference lets us know how long the first input delay was.


You can take these numbers and send them up to your own analytics infrastructure to start


to get a good idea as to what our FID results are going to look like.


If you're having a hard time bringing down some of your results, tracking in your own


analytics can let you do a much more detailed data analysis.


You can look for patterns such as user location, device type, or other bits of information


that may not be as obvious to compare and contrast on the official page experience results.


It is important to note, however, that while these are close and potentially identical


values to the FID results, there are a number of edge cases that could potentially make


it slightly different.


You can read more about the nuances of this at the link below.


Do you have 100% of your users below 100 milliseconds?


That's amazing!


Let us know your secrets in the comments below.


But if you're anything like me, you'll have at least a little bit of work to do.


So from a high level, there are a few things to know about first input delay issues.


First is defer, delay, and delete JavaScript.


By definition, any delay that is occurring is going to be related to JavaScript that


is running on your page.


All of the code on your page runs in the same thread by default.


That means that the browser can only do one single thing at a time.


So if a bit of code takes longer than expected, everything is blocked.


Even actions that people using your site may be trying to do.


This could be that you're just having a large chunk of JavaScript that's being parsed or


loaded when the page is initially loaded up.


An event handler could be blocking something on heavy, or it's just slow for some reason.


Too much code, or it's trying to run it once, or any other things could be happening that


are just making the whole site slower in general.


If you're seeing issues with first input delay, check if you are loading too much JavaScript.


Anything larger than 200 kilobytes is probably pretty large.


If it is, it'd be a great idea to check out code splitting.


This is a feature built into all modern JavaScript bundlers that allows you to split up code


into smaller chunks and make sure that you're not blocking the rest of your page from reacting


to users' input.


You can learn more about code splitting in the links below.


After splitting, any code that isn't essential for page load should be loaded using the async


or defer attributes on their script tag.


By using defer, you're telling the browser that the script is not going to change the


layout of the page, so it shouldn't block the rendering of the page while it's being


loaded or parsed.


Async is a lot like defer, only it takes it a step further.


It says to not wait on any other scripts.


Using the async attribute tells the browser to execute your code without blocking on any


other script beforehand.


While we're already digging into the source code, be sure to be on the lookout for code


that can just simply be deleted.


It can be easy to accidentally include polyfills of workarounds long after they're actually


needed.


Often, the first interaction a user has with a page is going to be something at the top


of your page, like things like a menu button or a search input, things like that.


But if you're having issues with first input delay but aren't really sure why your score


is what it is, a great idea is to start tracking the elements on your page that are having


their delays being measured.


We look at the code snippet we were working on before.


We also have access to APIs off of that same entry object.


Specifically, I like to look at entry.target.


This lets us know the element that we were interacting with whose delay is what's being


measured.


This can be really helpful if you have a large number of links, buttons, or other clickable,


clappable elements on your page but aren't quite sure as to which one it is that's


dragging your scores down.


Once we track what target is often associated with people who are having a slower first


input delay, you can start to dig into your site's code to understand what events are


tied to it and any listeners that may be attached to it.


Another thing to look into is breaking up long tasks.


Even if your code doesn't need to be split up, your tasks may still need to be.


Tasks are just things that your code is doing.


It could be laying out your web app, downloading state, or reacting to user input.


A long task is anything that takes longer than 50 milliseconds to complete.


If you're having issues with FID, then looking at your long tasks is a great starting


place to see where improvements could potentially be made.


Long tasks are actually highlighted in the Chrome DevTools.


If you want to start looking for these on your user devices to build up a correlation


between page experience results and long tasks in the field, you can actually programmatically


access them just like we looked at the first input event before.


We create a performance observer, then observe the long task type.


From there, we can iterate over every instance to look into each attribution.


This will give us a bit of insight into the functions that may be long running.


DevTools is definitely a much better developer experience, but this API allows you to collect


this information from your users in your own analytics, so you can track down correlations


that may be otherwise hard to find in your local dev setup.


Once you've found a long-running task, the next step is going to try to break it up into


smaller asynchronous functions.


This can be as simple as wrapping code into set timeouts or request animation frames,


but always the specifics depend on your specific situation.


You can read more about the techniques you can utilize to break up long tasks in the


links below.


In general, improving first input delay comes down to optimizing the when, where, and why


of the JavaScript on your site.


You want to try to reduce its usage to the minimum needed to maximize your site's potential.


Page experience is new, but the ideas behind it are solid and user-focused.


If you can get great results, then the visitors on your site will be all the happier for it.


There's still quite a bit to cover with page experience, but we're running out of time


for this video, so make sure to share any questions you have on Twitter or in the comments


below, and be sure to subscribe if you haven't already.


We'll be back soon to dive into yet another part of page experience.


Thanks.


Launching Google Search Central (step by step)

Welcome, everyone, to the next episode of the Search Off


the Record podcast.


Our plan is to talk a bit about what's


happening at Google Search, how things work behind the scenes,


and maybe have some fun along the way.


My name is John Mueller.


I am a search advocate in the Search Relations team


here at Google in Switzerland.


And today, we thought we'd do a special episode all


about the Google Search Central launch that we did a while


back.


And for that, we have a part of the core team


here to join us today.


In short, we have Lizzie, Cherry, and Gary here.


So let's do a really short round of introductions,


maybe in reverse alphabetical order,


just to keep things simple.


All right, who's going to start?


I think we'll start with Lizzie.


Wait, how does the alphabet work again?


So what does that mean?


That means you're first.


OK, well, I'm Lizzie, and I work at Google,


and I write documentation.


Where are you located, Lizzie?


Mountain View.


California.


Wow, cool.


In the headquarters, of course.


OK, and then we have Gary.


I don't know if any of you on the podcast


have met Gary before.


Hi, I'm Gary, and I'm doing Gary things.


Meow.


OK, fine.


I'm sure that's very explanatory for everyone.


Cherry, how about you?


Can you introduce yourself briefly?


I'm not going to meow, but yeah.


Hi, I'm Cherry.


I'm a search advocate in John's team.


Cool, and where are you located?


Well, I'm based in Singapore, like my own headquarter,


far, far away from everybody.


So cool.


OK, yeah.


It's like you three were all working


on this fantastic kind of site change


that we did over to the Google Search Central new documentation


center.


I don't know.


It still feels weird to call it that.


But anyway, so how did all of this start?


How did you come up with the idea


that we should just change everything around?


Well, it started at the beginning of this year,


I guess, when I met Cherry when she joined our team in Mountain


View, actually.


She came all the way over here to go to this conference,


and I wanted to work on something with her.


Aw.


So I convinced her that I had a crazy idea,


and I needed a partner on it.


And she listened to me.


I don't know why.


So cool.


And where did you come up with this idea?


Like, what's behind that idea?


Is it basically you're just bored and feel


like changing things around?


Or?


Yes.


Basically.


OK.


I just love work.


I'm going to pass it to Cherry.


Cherry, why don't you talk now?


I'm supposed to talk now?


Well, OK.


So yeah, I met with Lizzie.


And I don't know if you know, but I've been working


in search for a while, too.


So I also know there's some problems.


Well, not really a problem, but some opportunities


that we can tackle.


Yeah, so when I met Lizzie, it was just nice.


Things fall into place.


We started discussion mostly on a paper plate


during one of the lunch.


We ended up skipping all the sessions


and kept talking about this project.


And I think that's how we started these things.


OK.


So the master plan is on a paper plate somewhere.


Yes.


Yes, I framed it.


It's in my office.


Oh, wow.


It's a relic.


The famous paper plate.


We'll have to come and visit that when we can travel again.


I don't know.


So cool.


And what were your goals there?


I mean, you mentioned that there were some problems, Cherry.


What kind of problems did you see with the old setup?


Well, from what I noticed or what people told me,


at least at conferences and all that,


it's a little bit hard to find information.


We have a lot of resources, but there are a few places


that people can find them.


But there isn't one place that I can get everything.


Also, the other thing is that sometimes it's


also difficult to find things that's related


to certain audience groups, too.


Like if you're a developer, then maybe that's easier for you.


But if you're someone who's like hybrid


or have a few other different roles,


then that might not be as easy for you


to understand the resources and easy for you


to find these stuff.


On blog also, I think we have a lot of languages and all that.


And Lizzie came up with a really good idea


to also centralize all these things in one place.


Cool.


Yeah, so the main goal was to combine all of the things.


Would that be a good way to frame it?


Or how else would you, I don't know,


give a brief overview of the project overall?


In one of our initial decks that we made,


there's a slide that says, one ring to rule them all.


So I think that would be how I would summarize it.


OK.


Kind of have fewer copies of the same content out there.


And I guess that makes it easier to maintain as well.


But isn't that also bad for SEO if you


can't fill out the search results with your own content?


I don't know, Gary.


What do you think?


I don't think.


That's not what I'm here for.


I'm just doing Gary things.


Oh, man.


Just Gary things.


Yeah.


I kind of like how we managed to pull


all of these different things into one place


to make it easier also for people


to find the content, easier for us to maintain.


I think it also probably makes it easier for localizations,


right?


Yes, definitely.


I mean, the fewer pieces of content you have,


the easier it is to localize it.


And we are quite an international team.


I don't know.


I think we speak over 13 languages in total


across all of us.


And we do care a lot about localization.


So the easier it is for us to localize content, the better.


And ultimately better for the users.


And we did have major problems with localization


due to the fact that we were spread out


on so many different platforms, like Blogger, Help Center,


and developers.google.com, which I'm going


to call Dev Site for short.


And the fourth one, the Kintaro weird place.


Oh, yeah.


And we had Kintaro as well, which


was the google.com slash webmasters


and everything below, except the tool itself.


So we had lots of things to localize.


And it was all different platforms


with their own quirks.


It was a smallish nightmare for us to localize things.


Like, for example, with Blogger, when


we publish something in English, or even before we publish


something in English, we had to send out the English copy


for localization and then find all the blogs where


we had the localized blog and upload the copy there as well.


That was all manual work.


And it took up quite some time to publish


on those localized blogs, the English content.


And with this move to Dev Site, we


don't have to do that anymore.


Basically, everything happens in just one place.


And we don't have to deal with localization pretty much at all.


And with one of the places, google.com slash webmasters,


we couldn't update the localization at all.


They had moved to a different platform,


and we didn't follow them there.


So it was actually locked, and we


couldn't update that site at all.


Which was kind of crazy.


JOHN MUELLER, MD, PhD, MPH OK.


This sounds like typical big company problems,


like so many different silos and setups separately.


I guess Google has those things as well,


like any other kind of bigger company.


GARY ILLYES, MD, PhD, MPH So I don't know if it's just


a big company problem, because back when I was doing webmaster


things, then we definitely had, for example,


shop, blog, and the main site.


And those were typically different platforms.


For example, you were using WordPress or blogging.


You were using, I forgot what the cart was called,


but a different software for the shop itself.


And then we had probably some HTML custom stuff


for the main site.


And then if you had to localize those,


then you would have to do that in different ways.


And it was not a big site.


It was just a small site.


But it had the same problem that we had.


JOHN MUELLER, MD, PhD, MPH No.


OK.


So we have to deal with the hard part of the web


like everyone else.


No magic cures just because we're at Google.


No machine learning that solves all of our website


hosting problems.


GARY ILLYES, MD, PhD, MPH I want to quit.


But we did use Python, right?


Yes, we did use Python extensively.


And lots of, lots of, lots of, lots of shell scripting.


So one thing that we had to do was pulling in content,


for example, from all the different places.


And my initial idea was to use Googlebot because, well, I


work on search.


I worked on Googlebot.


So why not use Googlebot?


But then, well, external folks wouldn't


be able to use Googlebot.


So it felt kind of unfair to just use Googlebot


and have it solve everything for me.


So I actually wrote a smallish crawler, essentially,


in Python that was using the normal URL fetch library,


or the URL library, and BeautifulSoup,


which is another library, to basically just extract the,


well, crawl the URLs that I specified,


and then extract the parts of the content


that we were interested in.


Which in case of the Help Center,


it was basically just the article body.


We didn't need the navigation, for example.


In case of Blogger, it was different because there we


could just take the feed, basically just create


an export of all the content, and just use that, basically


extract from the feed.


So, yeah, it was fun.


We had to do lots of weird shell magic


to replace certain things, for example.


And for that, I had to go down to the deepest caves of hell


and dig up my regex knowledge.


JOHN MUELLER, MD, PhD, MPH, Addressing Tech,


and I took that lesson from my friend,


I'm not sure why he said that.


Thanks, regex.


What a world.


What?


So for that, we had to create lots of regex patterns with which we could match across


different languages and different files, essentially, and just replace or remove


certain things from the raw HTML. That was kind of fun and painful. And I only thought


a few times about stabbing myself with a spoon.


Well, for me, it was like pretty magical because I didn't understand how you were even doing it.


I would just ask you for stuff like, please fix this. And is it possible to fix it


across like 18 other languages? And then it would happen like the next day.


So for me, learning all about this is very interesting.


Yeah. On this side, it was more like sitting in front of the computer


and taking a spoon and trying to stab my brain with it.


Well, I like being on this side. It's kind of nice. I don't have to do that.


I'm glad to hear that, I guess.


Yeah. So cool. Yeah. I mean, over the years, when you look at a platform like Blogger and


the way that we used it in the past, it just collects a lot of different cruft. So I remember,


I think almost from the beginning, we would copy and paste the blog posts from Google Docs


and copying and pasting from there includes all kinds of weird HTML formatting if you


don't pay attention. So that's got to be. Or like you forget to copy the title of the


blog post into the thing that you upload and then it comes back without a translated title,


which is awesome. And then you have to do another request just for the titles.


Oh, yeah.


Who would do that?


I did that a few times. Yeah. I don't know. Are you talking about me or?


No, I'm talking about me.


Yeah. OK. I mean, we'll miss all of these weird quirks. It's like,


it'll all just be so smooth in the future.


Anyway, talking about Blogger, like how did you set up those redirects? I looked at that once


with my own blog, kind of trying things out on Blogger initially, and I couldn't make it


do redirects like that. What did you do?


So Blogger is special from the perspective of redirects. Like if you want to redirect the whole


site, then that would work relatively well. But then if you need to do one on one mapping with


certain URL changes, so for example, you have to drop the HTML from the end of the URL,


then suddenly that will not work. And you don't have that many options.


One thing that we were suggested was to get in touch with the Blogger team and they can


probably sort out the redirect for us. But then again, that felt kind of weird because


external folks wouldn't be able to do that. They wouldn't be able to get in touch with the


Blogger team and get some special treatment. I don't know why I'm air coating here because no


one can see it, but people wouldn't be able to do that. And so I decided that, well, I had a meeting


with Gary and we decided together that we are just going to initially to do meta redirects because


that would generally work better from indexing perspective because indexing doesn't have to be


interpreted the JavaScript first to understand that there's a redirect going on. But then


Blogger was protesting quite loudly about the meta redirects. It didn't like it at all. So I had


to fall back to, well, JavaScript, which is my favorite language on earth. And I had to write


a short JavaScript script which would redirect based on the URL path to that site, basically


matching up the URL structure. That was one thing that we paid a lot of attention to, basically


maintaining the URL structure for the blog part of that site. And I think it worked out really well.


So cool. And also on the blog, there's the feed, but I guess people don't use RSS anymore, right?


So we probably just dropped that, right? And you're wrong. You're just as wrong as I was.


I ran a quick survey on Twitter and on Reddit asking people whether they use RSS still.


I don't. I only recently got reintroduced to RSS thanks to Lizzie because she's using some weirdo


plugin that she can use to subscribe to feeds or check out feeds or something. I don't actually


know. I just wanted to subscribe to my own feed. So I found this plugin to do it because it's


complicated. Well, it's your fault that now I also have it. So anyway, so the feed, yes.


Initially, I was hoping that we would deprecate the feed and say bye-bye to RSS. But then once I


saw how many people actually use RSS still, I kind of convinced Gary and myself to port over


the RSS as well to Dev site. But it turned out that it's a smallish nightmare.


So no JavaScript?


Why is it a smallish nightmare?


Well, so coding up or scripting up the XML for the RSS is not that much of a problem.


It's fairly easy to create an actual working RSS feed, but that would be static. So you need


something that updates itself. Now the Dev site, due to how it works, due to the infrastructure,


you can't have dynamic files. So every single time we publish something, we have to add


the newly published something to the feed manually. So that's a step back from what we had


on Blogger. But of course, over time, we are going to fix that. It's just going to be rather


complicated from coding perspective. And then, of course, we also had some screw ups with migrating


the feed.


Wait, you messed something up?


No, it was Lizzie.


It was Lizzie? OK. Thanks for that.


Sure. Anytime.


What happened? I didn't even notice.


Oh yeah, you did.


Other people noticed.


Other people. Well, I think that's kind of the nice part about writing for SEOs is they


watch out for every move that you make. And if you get something wrong, they will tell you about it.


You almost don't need to watch out for things yourself.


And they tell you promptly.


So what went wrong?


So I mean, I've been dealing with RSS like a decade or so ago. I completely forgot about


GUID, which is an indicator for feed readers that some specific item in the feed, they


already aggregated or processed. We, well, we, as in Gary and I, wanted to have a simple


feed. So we opted for not having GUID in the new feed, which meant that when we ported


over the most recent items from Blogger and we put it in our new feed and we remapped


feed burner to the new feed URL, then suddenly everyone got notified about the new blog URLs.


But of course, the new blog was not live yet. So everyone accessing those links from the


feed burner feed led to a 404 page, which I should add that not that useful 404 page.


So the experience was kind of, I don't know what the right word is, suboptimal.


Bad.


Very bad.


Okay, we can be direct here.


So why did we think it would be okay to do this? Why did we do it this way?


What do you mean? Because you told me to.


Yeah, okay. That's fair.


I didn't think anything would happen. I thought it would be fine to.


Yeah, me too. Okay. I will take the blame. It was my fault because I should have thought


of the GUID. I forgot about GUID. I'm getting old. I can't be blamed. Don't be an ageist.


The other thing that I thought was kind of cool on the new site is Googlebot's new companion.


We now have a spider. So what's up with that? How did the spider come to be?


So Googlebot has been around for a long time. We think maybe Googlebot could feel a little


lonely all this while. We started talking about having a sidekick or friends or a pet


for Googlebot. From there, I think Lizzie and I spent a lot of time talking about Googlebot.


I spent days of bonding time picking which type of animal. Well, it's more like months, right, Lizzie?


I think, yeah, maybe like two months. I don't know. This was like our pet project,


if you will. No? Is that too far?


It kept us happy. All those months. And the credit goes to Lizzie. She picked the spider.


I had a few more different weird animals in my mind.


Do you want to say what they were?


Maybe not. But there were a few of them. But yeah, after sticking with the spider,


I actually spent days looking at different breeds and types of spider as well, which was like


weirdly very fun, actually.


Wow. OK. Do you think there are going to be other pets in the future?


It's like Googlebot has a spider now. Next week, he'll have a Python.


Or is this going to grow over time?


I wanted more than one, remember? Like you people stopped me from like this bizarre thing.


I always wanted more than one. Is that still not confirmed? Can I still have more than one?


It's not confirmed. It's not a hard no. But it's not an enthusiastic yes either.


OK, well.


I think it would be cool that when we have some bigger


launch, for example, then that launch is joined by a new friend.


Right. So more pets, more friends. Yeah, that's good.


OK. Now, first, we have to do more launches, more big launches. And then it's like if we,


I don't know, have a core documentation update, then maybe we could do it then.


Is this like a reward system? Like we work hard and then we get a pet, a new friend?


But you have to keep it. It's like you can't just like swap out pets.


That's the hard part. It's like life.


We're going to keep them, keep all of them. We're not going to swap them out.


Don't worry. We'll take care of them.


Yeah. Does Googlebot like cheese?


I don't know. It depends. Do you like cheese?


No. You don't?


Sorry. I don't know. Well, what about the name of the spider? It's like you give Googlebot a pet,


but you don't give it a name. Like Googlebot has a name. What's up with that?


No comment.


Wow, that's awkward.


No comment. It's like this is off the record. Come on.


It's too painful. It's too painful for us to talk about because we got so wrapped up into this.


Yes. That's like another month of us. Yeah. About the name.


Yeah. Coming up with a name, like naming our child and getting denied many times by the legal team.


You were very excited with the unborn child and like the name and then yeah, it is hard.


And then Gary didn't like it? Gary didn't like the names or what was the problem?


Hey now.


No, Gary was supportive actually. He liked the names. It was the legal team.


Hey, don't make me sound nice.


Okay, fine. You hated it.


Okay. But it's like there's still an option that the spider will get a name at some point or


is it permanently without a name? The spider?


No, we are still like trying to come up with something. We just couldn't do it in time for


launch. So we are open to suggestions and we are still thinking hard about it.


Okay.


But nothing more to say at this time.


Okay. Cool. Then I guess about the name in general, like the search central name,


what went on to come up with that name? Why didn't we just like pick webmaster central


and go with that? It's like pretty much the same thing, right?


No, no, it is not the same thing.


Yeah. I think like from what I recall along the way there, one of the things that we noticed is


fewer and fewer people associate with the name webmaster. And so it felt kind of archaic almost


to focus on webmaster as an audience when nobody calls themselves webmaster. So I remember I asked


at a conference back when we had normal conferences, what people thought they would


call themselves. If they're like SEOs or developers or webmasters and lots of SEOs in the audience,


lots of developers and webmasters, like very few people raising their hand. So I think at some


point it just kind of made sense to kind of pivot away from the name webmasters and try to find some


new name that we could use. And there are so many people involved with making websites nowadays. So


focusing more on the search side kind of made sense.


Yeah. That was one direction we tried to explore, like when we were coming up with


options, like is there a name that would summarize all the people who work on websites? And we just


couldn't come up with one term. It would be like a very long name, like 10 words long probably.


So we decided not to do that, which I think is good.


Yeah. I think that's also kind of reflected on the new homepage. When you look at the homepage now,


it's not developer focused and not purely kind of like old school webmaster focused. You can


kind of pick which track almost you're interested in. Is that something that was also part of the


goal from the site there? Definitely. That was something that Cherry actually did a lot of work


on. Cherry, do you want to talk about how you discovered the learning paths for the site?


Sure. Well, we looked at the user journey and we started looking at persona as well. So


the simplified version of the persona has like five types of them. And then we also look at


common task things that these persona or people like to do. We know some people want to come in


and explore more. We know some people come in to fix stuff. Some people come in to look for


contact or like trying to get in touch with us. We know that some people come in to fix stuff,


for example, and then we look at how we are currently basically supporting them with all


these tasks. Right? Like for example, if they're looking at fixing certain things, some people


search for things specifically, and they would already land on specific help center articles,


for example. Then we know that we have to cater our homepage, for example, to the audience that's


looking to learn more and more generically, not like learning about certain things because then


they would be landing on a certain page already. Yeah. I felt like that really helped us narrow


down what we were doing with the homepage because when we were first starting the draft for it,


it was like we put so much on there. How do we narrow it down? There's so many things that we


could put here should it summarize everything that we possibly have on the site or should it focus on


a specific type of user. And I think in the end, we decided it should be for brand new people


because we thought that people who know exactly what they're trying to do, like, oh, I want to add


structured data to my site, they will just search for that and go to that page.


And our homepage should be more for like, you don't know what you're doing at all. Okay, here is a


very quick overview and get started link. Yeah. So cool. Have you been able to track how people


are using that so far or is it still the early days for that? Still early days. I think right


now we have two weeks of information. It's much better. Before on the dev site, our homepage was


position number 20 in terms of views. And now it is like the number one page, which feels


appropriate. Okay. So you're less focusing on the rankings of the homepage in search and more


comparing it to other parts of the site? I don't know what the right answer is, but


that is what I was looking at. Okay. Okay. Definitely makes sense. I don't know,


talking about rankings, did you guys end up doing any SEO for the site?


Funny you should ask that because I think John on our team was looking into SEO.


I'm sorry, who? Oh, who could that be? Yeah. So awkward. Ask myself questions. Yeah. It's,


I guess one of those things that I always thought would be useful for us to, on the one hand,


get help with regards to SEO so that we kind of follow the public best practices and also to see


how SEOs tend to work. So one of the kind of unique things we did with this project is to get


some help from the SEO team at Google who works together with some external vendors. And we


decided instead of us kind of using our internal knowledge to do SEO for the site, which I think


wouldn't be fair and wouldn't be in line with all of the other things that we did, where we tried to


essentially do things like any person externally would be able to do, we decided to take their


input into account. And I thought that was pretty insightful. So we got a lot of input on the site


moves, things that we need to watch out for, they're monitoring it with their SEO tools,


all of the usual things. And one of the other things that we got some help with was keyword


research, which I thought was kind of unique because we never did anything like that. How


were your experiences with that, Lizzie? You were involved there a bit.


Yeah, I thought it was pretty interesting just to see how they compiled the words. And for me,


I write and I feel like I have a library of go-to terms. So it was interesting to see,


oh, here are some potential new words that you are not using. What do you think about them?


But there were also some words that were awkward for us to use, or maybe we don't want to use that


kind of phrase. There were some suggestions for SEO workshops. And they suggested the word


workshop and our events aren't workshops. So it felt like this is the wrong meaning,


but it was the keyword that they were suggesting to use. So stuff like that was interesting to


consider. Yeah. Okay. So you didn't always follow all of the advice from the SEO team.


SEO team. I mean, it's kind of a tricky situation for them because they're kind of basing the advice


that they give us based on the advice that we give them. And it's almost kind of circular,


I don't know. But I thought overall, it's been pretty insightful to work together with them. So


that's been pretty cool. What are some of the cool or weird quirks that you ran across while doing


these migrations across the different sites? Gary, anything weird or cool come up on your end?


Yeah, me. Oh, you ran across Gary. Well, I mean, he's been on the team for a while, so


not particularly new. I'm trying to think about weird things, but


there were many things that were weird, but from our perspective, basically because of the


infrastructure that we are using. Like, for example, getting all the content in from all


the different sites that was already kind of weird and probably would have done differently if I had


to do it on a different platform because there you would probably have access to all the databases


and you can just pull from the database directly. So that was definitely weird. My screw ups,


they were weird because it always happens. I can do weird screw ups and I'm proud of that.


Like, for example, Lizzie was telling me for the longest time to import the localized titles from


the Help Center articles and I kept postponing it because, well, it's a small task and I just


didn't feel like it, I guess. I was just pulling up Gary and I always had something more interesting


to do. So yeah, I was just procrastinating. And then at one point we decided to enable the


redirects on the Help Center and then we enabled the redirects and the next day Lizzie was like,


okay, so titles. And I'm like, yeah. I think you need to back up and explain what happened


with the titles. Like, how did we get here? Right. Okay, fine. That was another Gary moment


where I was fixing something in the English version of the content and I did a sweep across


all the languages and basically just swapped out all the localized titles to the English titles.


So basically in Hindi or in German, the title of the article would be in English because,


well, hi, I'm Gary. No, no, no. Like what it was was there was two titles in every page. There was


the English title and then the localized title. And then you removed the localized title for some


reason that I don't know why. So then we were only stuck with the English title.


There you go. Okay. Okay. Again, I'm old, so my memory is not as good as it used to be. Also,


my memory is not as good as it used to be. Yeah. A lot of these things, I guess, just come in


because we have so many different platforms and so many weird quirks across the platforms.


One of the things I noticed is that at some point over the years, a lot of the images in the really


old blog post disappeared. So that was also something that I've been meaning to fix for a


while. And then I put together some Python scripts because of course Python. And I didn't use any


machine learning either because I will teach the machine using Python. And I pulled up the


archive versions of all of those images and embedded them again and rehosted them. And I


think when I sent that away to get reviewed, Gary was like, oh, I was also doing this.


So we almost got the old images back twice. That would have been really weird.


GARY ILLYES-SELLSEN I would be really awkward if we were a communications team and we didn't


communicate with each other. Wouldn't that be weird?


JOHN MUELLER Okay, fine. I will send you a tweet the next time I start doing something.


GARY ILLYES-SELLSEN Oh, that would be greatly appreciated.


JOHN MUELLER Yeah. Anyway, why didn't all of these changes happen at once? It sounds like


a lot of things took place over time. What's the reasoning behind that? Is it just like things got


broken or went wrong or was there a plan? GARY Well, I think a little bit of both. So


there was a plan. Yeah, crazy. I did have a plan. It was intentional for us to do it across the span


of a week. So we wanted to make the announcement on our existing blog that we were changing our


name and that we were going to move the blog. And then we wanted to wait a week to then actually


move it in case something went wrong with our assess feed. We wanted to make sure that people


were notified about what was happening. And then we also wanted to wait with the Twitter name


change. We wanted to announce that we were changing our name and then an hour later,


actually implement the change so that people weren't surprised. But that was one of the


surprising reactions for me because that was the main comment that we were getting on our


announcement tweet that we changed our name. They were like, you dummies, you didn't change the


actual Twitter account name. Why don't you do that? And I thought, oh, it would make sense.


People will be confused if it's changed right away. We should wait an hour and give people


time to read the announcement. And apparently they did not need time.


People are impatient. Yeah. And the split with the blog posts and moving the articles,


that was also planned? Yes. Right. So I wanted to wait a week to announce that we were moving


and then actually move the thing. We planned to do it on a Wednesday and we did part of it.


We were waiting for some help to update the RSS feed link in FeedBurner. And that didn't happen


until too late in the day. So we decided, okay, we'll just move everything and publish the


announcement the next day, which was a mistake because we didn't realize that it would email


everyone that night with this long email with all these 404 links that like, oh, these pages


are not there, which was amazing because then people were tweeting and emailing us, oh, this


is embarrassing. You guys sent this email and none of the links are clickable. Fun. And then we were


racing to fix it the next day to publish the pages. And then many things happened that morning,


but I think it all ended up okay. Yeah. I mean, luckily we have all of these SEOs out there that


know what to watch out for and that let us know as quickly as possible. It's almost like we don't


need to do any monitoring on our side. Well, I would like to do the monitoring so I don't get


embarrassed. I don't like that feeling. Cool. Wow. It's like so much stuff here. Would you do it


again? Was it fun? Definitely. This is like the best project that I've ever worked on. No sarcasm.


OK, but we're going to keep it in this location for a while first, right? OK.


I heard that Sundar requested that we move. No, no, I didn't.


Don't hurt me like that. I think we should keep this location for a while,


at least until everything is ironed out and working well. So one of the things that I notice


is we move the site move guidelines as well with a lot of the documentation. Did you end up


following those guidelines or what was your take on it, Gary? No. No, we make all of these guidelines


so much text and you're like, no, that's correct. And that's because our infrastructure is different


enough that we were just not able to follow it. Some aspects we were like, for example, monitoring


the change, but for example, putting up the site beforehand, that was just not possible for us.


Again, with redirects, basically we had to go with what we had at hand. Like we don't have a magic


HD access where we could, for example, set up a 301 or 307 or 309 redirect. We actually had to do


the JavaScript redirects on Blogger. We didn't have any other choice. And basically we followed


what we could, although inadvertently, I think, I don't think that we actually read it. Well,


I did. Well, you wrote it. I had to rewrite it. That was kind of weird to do that a few weeks


before we were launching, rewriting the site move and then me having a panic attack. Are we


following these guidelines? We're working on this right now. Are we actually doing it by the book?


Is this the book we're writing? Yeah. Pretty weird. Well, the answer was no, we are not. No.


Yeah. Right. So it's fine. But I did freak out about it.


I guess those guidelines are a little bit more the best practice guides. Like ideally,


when everything goes well, you can follow these things. And especially if you're moving from one


domain to another, it kind of makes sense. Right. But yeah, I think one of the things was like,


oh, try to do this on the weekend when things are slower. And it was like, we don't want to do that.


That would be not good. If we did that, we want to be available if things are broken. Yeah. We


generally can't, or we shouldn't launch anything on weekends. That's a Google wide thing, in fact,


except, well, weekend doesn't also local, from local perspective. Like for example, Tel Aviv can


launch on Sundays because that's their Monday essentially. But otherwise we shouldn't really


launch anything or make massive changes on weekends because then you don't have the


supporting teams there to help you out if something goes wrong.


Cool. It's been a really fantastic launch to watch, to see how everything gets ready, and how


most things ended up getting launched properly. And nothing really seriously went wrong. So


I thought that was pretty cool. Thank you all for your fantastic work there.


Anyway, thank you all for joining us here as well, folks. It's been lots of fun doing these podcast


episodes. I hope you, the listener, have found them both entertaining and insightful as well.


And let us know if there's something kind of different that you'd like us to cover here as


well. Feel free to drop me a note on Twitter or chat with us at one of the virtual events along


the way. And of course, don't forget to like, subscribe, and update all the links to our site,


SEO Mythbusting Is Back - Official Trailer (Season 2)

What would you say is actually the most frequent myth or question that people are coming with


to you?


How can I rank on the top, right? That's like number one question everywhere.


But when you tell the community, just build the best possible website, they go back and


start laughing, yeah, build the best possible, of course I think I have the best possible


website, but what can I do to make it better? Should I add HTTPS? Should I make it.5 seconds


faster? Should I, this specific thing, where are my priorities?


What goes on in the Google machine once the site movement domain name change is triggered?


What about word count? SEOs are always asking, is word count a ranking factor?


The first thing they listed was page speed and they were actually quite insistent in


the write-up that it was the most important ranking factor.


People are just trying to group pages topically maybe with canonicalization, but that's not


how it works.


It almost brings to the fact that there's so many different issues that fit into crawl


budgets, whether it's duplication or finding the pages.


I know no one wants to hear that, but it depends really is the answer to a lot of things.


So what's it depend on?


Let's do that again, just one more time.


How do you take your hair and pull it all the way in front of you?


Perfect.


That was almost like a thing about you.


That was great.


Changes coming to the Top Stories Carousel

Way back in 2016, Search introduced the concept of the top stories carousel.


You see, as more and more of the world was, and continues to come online, they were increasingly


doing so not on desktops or laptops that so many sites have been built to expect, but


on mobile phones, often with slow connections.


That's a precarious mixture.


You see, websites with ever-expanding file sizes that are being packed with JavaScript


and CSS don't really result in an ideal experience for folks that are trying to use it from the


phones.


So Search gave users a new way to interact with search results by highlighting exceptional


web content that was exceptionally fast.


That sounds great, but what does it actually mean?


Like, really, how do you actually do that?


Because algorithms are concrete rules.


They're not philosophical ideas of things that are fast.


In collaboration with news publishers, the Google Search team initiated the AMP Project.


This is an open-source project built on the idea that we should only allow the best practices


in order to maintain high user expectations for speed.


By introducing the top-storey's carousel and then requiring all of its entrants to be valid


AMP documents, a lot of those anti-patterns that could negatively impact the user experiences


on mobile devices were able to just be avoided.


AMP served as a sort of signal between a site owner and search.


It was a sort of, I promise I'm not loading gigabytes of JavaScript or CSS and that my


pages are stable, not jumpy.


By requiring this, search was able to know that your page was likely going to fit in


that definition of exceptionally fast.


But that signal, that agreement, is not ideal.


Because yes, AMP pages have consistently been some of the fastest pages on the web, but


anyone that is a web developer will tell you they have only been some of the fastest pages


on the web.


See, AMP can make it a lot easier to build and form a web page, but it's certainly not


a requirement.


It's just a JavaScript library.


Any knowledgeable, skilled web developer can absolutely make web pages that are just as


fast, if not faster, than AMP pages.


What was missing with those other web pages was AMP's validation step.


You see, the enforcement of those rules on web pages that were known to result in a better


user experience.


If you had a valid AMP document, it literally could not do the things developers should


try their best to avoid.


But now, with the recent introduction of Web Vitals, browsers have improved to the point


where search doesn't need to rely on JavaScript frameworks to make those enforcements.


The browsers themselves can check to see how a user is actually experiencing a web page.


Is it jumpy?


Is it slow to respond?


Is it slow to load?


And rather than trying to prevent us from writing code that could have caused that in


the first place, browsers can now actually detect if a user is actually experiencing


those frustrations AMP was trying to prevent.


Because of this, search is evolving the Top Stories carousel to represent all relevant


pages regardless of what is used to make them.


That's right, AMP is no longer a requirement for the Top Stories carousel.


Now, the addition of non-AMP pages is not the only change that is coming to the Top


Stories carousel, however.


Namely, the AMP logo, that lightning bolt badge that was displayed at the bottom of


search results, it's being retired.


Because it's simply no longer true.


Any web content can be in the Top Stories carousel.


Additionally, the AMP viewer is being overhauled.


Today, when a person taps on an article in the Top Stories carousel, it is showed instantly.


This near magic is able to be accomplished because of several underlying parts that previously


required AMP.


That instantaneous loading was able to happen because search was able to download all the


markup used in an AMP document in the background in an iframe before the user ever even tapped


on something.


So as soon as they had tapped on it, it had already been prefetched and pre-rendered and


it was instantly available because it was already available.


But why was AMP needed for all that?


In addition to the AMP JavaScript library, there was this idea of an AMP cache.


See, AMP caches are servers that act as a sort of in-between between the origin server


and the end user.


They allow for AMP hosts, like search, to request and render all of those documents


all while being able to maintain user privacy.


If every time a user searched for something, all those prefetches went to the end servers,


then all those sites would know what each user was searching for, even if they ever


actually even visited that site.


It's a pretty big security violation and certainly not something that we could abide by.


By using an AMP cache to show people the AMP pages, search was able to start requesting


and loading the first few results in the top stories carousel, the ones that people are


the most likely to interact with, all while making sure that only the sites they actually


do interact with are aware of their browsing.


Not only that, but in the top stories carousel today, you can actually swipe from one article


to the next, regardless of what site it was published on.


That was possible because of something called the AMP viewer.


This is another part of the AMP project and it's sort of a wrapper around all the different


content that's being served from an AMP cache, allowing you to jump from one article to the


next in a kind of seamless manner.


But with top stories carousel opening up to more content, search can no longer keep that


same guarantees that are required to make a safe, secure, and swipeable version of the


AMP viewer.


As a result, the swiping interaction is going to be removed as well.


Don't worry, you can still keep that sweet, sweet prefetching that makes the top story


so fast, regardless of what you use to publish your pages.


If you already are publishing an AMP page, then nothing changes.


The AMP project and AMP caches are more active than ever before, and so AMP pages will continue


to be prefetched and pre-ventered from search results.


But if your site can't or just doesn't feel like publishing an AMP version, you can have


prefetching too.


That's because of yet another incredibly exciting browser feature, signed exchanges.


Signed exchanges are, without a question, one of the things I am the most excited for


in the entirety of the web.


There's quite a bit to them, certainly more than I could fit in this entire video, let


alone a section of it.


So I encourage you to check out the link below to learn more about it because it is really,


really awesome stuff.


But the short version that matters here is that signed exchanges are sort of a recording


of a site being requested and loaded.


See, that exchange of information that happens when Googlebot is crawling a server, what


the server responds with, is kind of bundled together and cryptographically signed by that


server.


It's basically saying, these are the exact bits that I created and sent to you.


You can take these exact bits without modification and give it to others if you choose to and


let them know they came from me.


This is really cool.


It means that when someone hits a blue link or a top stories result page on their search


results, search can prefetch all that content just like it does today with AMP pages, but


all while maintaining end user privacy.


And because all that information is cryptographically signed by that server, browsers are sure that


it came from that server and not just search as cache or some other middleman.


That means that end users will see the actual real URLs of the real content from servers,


not the AMP cache ones.


See, in fact, if you are serving AMP pages today and you want to see your real URLs and


not just a banner above it, you can serve AMP content via signed exchanges today.


Now that we understand how our end users experience may change a little bit, let's take a look


at what it's going to take to come up and join the Top Stories carousel in the near


future.


Top stories are not random web pages.


It's highly relevant news stories.


Therefore, just like today, if you want to be in the Top Stories carousel, your content


will need to follow Google News policies.


Additionally, you will really want to dig into your web vitals and page experience scores.


These new metrics join the hundreds of other signals that search evaluates in milliseconds


every single time that someone executes a search.


We've touched on web vitals and page experience a bit, and we'll continue to deep dive into


them throughout this series.


So make sure to subscribe to learn more about how to measure and improve your site to help


your pages ranking.


So what can you do to make sure you're ready for the new Top Stories carousel?


I'm so glad you asked.


As with all things, though, the answer is it depends.


If you haven't already, now would be a great time to investigate how your pages are faring


in page experience.


We'll have a number of links below that you can check out, but most importantly, you should


be checking out your page experience report inside of the search console.


If you haven't already, you may need to sign up for an account, but once you have, you'll


unlock invaluable information about what your actual users are really experiencing when


they load your site.


You'll be able to see specific problematic URLs that need addressing, and we can jump


into just how you're going to address these problems throughout the series.


If you're already featured in the Top Stories carousel, that means you're already publishing


AMP pages.


Up until now, AMP documents were the only ones considered for the Top Stories carousel.


Since search is going to be opening up these results to any web page, those loaded spots


in the carousel will inherently become more competitive.


Make sure you continue to publish high quality, highly relevant content, and fix any page


experience issues as they crop up, and you'll likely be doing just as well when these changes


roll out as you are today.


The AMP team has actually created a great AMP-specific page experience guide that you


can use to automatically check for a large number of issues that may negatively impact


your pages.


Make sure to try it out today.


If you're not publishing AMP, in addition to continuing to make great content and keeping


an eye on your page experience, you'll also need to make sure that you are including article-specific


structured data inside of your articles in order to be included in the Top Stories carousel.


This is just a few lines of metadata that let search know things about the page, like


headlines, bylines, publisher, publish date, things like that.


While there isn't a page experience tool for non-op content yet, you can check out all


the great web vitals tooling to get a really good idea on how your pages are currently


performing.


Don't sweat if you're struggling to meet the numbers that web vitals and page experience


are encouraging you to get to.


You can't really pass or fail these metrics.


These are just tools that search can use to compare relative performance between other


pages.


The better your numbers are, the more likely you are to outperform competitors on that


specific ranking factor.


If you are struggling to improve and you're not interested in creating or maintaining


an AMP version of a page just for the Top Stories carousel, there may be a sort of middle


ground.


The Bento project from AMP allows you to use a lot of the more performant parts of the


AMP library without actually creating an AMP page.


Like I mentioned before, AMP isn't magic.


It's just a JavaScript library.


The Bento project is a new modular version of that library that lets you take individual


components out of an AMP document and load them into a non-AMP page.


Why would you ever want to do that?


Simple.


These are battle-tested web components.


They've been loaded billions, if not trillions of times all across the web.


You have teams of people behind them dedicated to constantly improving their performance.


But perhaps even more importantly, would you really ever want to implement yet another


carousel if you could just pick up one off the shelf that was already performant?


Bento lets you cherry-pick the high-performance part of AMP into any page you want.


It's still early days, but I think it's certainly worth checking out to see if you


can offload some of your performance baggage.


It's a really exciting time for the Top Stories carousel, because it's an exciting time


for the web.


With the introduction of Web Vitals and Page Experience, the original driving idea is behind


the carousel.


A modern, attractive interaction that users are able to trust to get great experiences


from can be realized no matter what you choose to make your site.


If you have any questions, please make sure to leave a comment below or reach out to us


on Twitter.


And don't forget to join us in the next episode, where we'll start digging into what


goes into Page Experience, issues that may come up, and just what exactly you need to


do to take care of them.


JavaScript SEO office hours October 28th, 2020

Please.


All right.


Hello, and welcome to the JavaScript SEO Office Hours


on October 28, if I remember correctly,


or if I can read my numbers on the watch correctly.


In these, you have the opportunity


to post questions onto YouTube.


If you go to youtube.com slash googlewebmasters


slash community, you'll find different threads


for the questions for the different office hours.


There's the general office hours that John Miller runs,


and then there's the JavaScript SEO Office Hours


that you're watching today.


You also have the opportunity to join the recording live


and ask questions live.


I'm really happy to see that I have three guests in this office


hours today.


Hello.


How is it going?


And we have a bunch of questions submitted to YouTube,


so I'll go quickly for those who have submitted questions


on YouTube.


All right.


Is it bad to use JavaScript for pagination?


Oh, that's a lovely question, because the answer is no,


it's fine.


Test it.


Make sure that you're not breaking it,


that you're not doing something that doesn't work


or that we can't pick up.


But generally, JavaScript pagination is not an issue.


Ludwig Larson is asking a longer question.


A web page links to products.


In the raw HTML, the a-link tags are formatted as href


and then double curly braces URL.


With JavaScript enabled, that placeholder


that double curly braces URL gets


replaced with a real link and correctly rendered as href,


for instance, slash category one slash product 123.


These links are followed and indexed by Google.


Good.


However, in Search Console, the 404 error report


contains many links, such as category one slash double


curly braces URL, which obviously


originate from the raw HTML.


Since Google can index the JavaScript rendered links


correctly, would it somehow be a bad practice


to add disallow slash double curly braces URL in robots.txt


to quickly solve this issue?


Well, first things first, if the links get indexed,


it's not really an issue.


So the error report, the 404 error report,


is just to highlight that there is something that is a 404.


That can always happen.


It is definitely possible that someone links


to a page that doesn't exist.


We crawl it, and then you see a 404 error report for that.


That doesn't mean that you need to worry about it.


It's not an issue that you need to address.


It's just something.


If it annoys you, then you can totally


look into your robots.txt and disallow URLs


with that placeholder.


But it's not a problem that you need to address.


It's more like an inconvenience.


Eric asks a very broad question, which is SEO in Create React


App.


Can you talk about that?


Yes.


Generally speaking, Create React App


is mostly fine out of the box.


You do want to add React helmet to get some metadata


in as well.


But if you want to learn more about that, conveniently,


I did talk about that in the past.


If you go to our YouTube channel and search for the JavaScript


SEO video series, there is an episode on React


that I highly recommend you check out.


Then Ricardo asks a question about something


that is actually not JavaScript.


So I think I'll skip that.


Sorry for that.


I should have marked that beforehand.


Then we have Manoj who is asking JavaScript error in AMP.


I'm sorry.


That's too broad to answer.


If you have details, maybe take that question to the Webmaster


forum as well, because that is not


a question that makes sense to answer,


because I don't have any context here.


Joseph is asking about the benefits


of using Jamstack for a website.


He specifically asks about Nuxt or Next.js or Gatsby,


if there's any pitfalls in these approaches


or if I would recommend it.


So generally speaking, Jamstack websites


are great for websites that have some dynamic data


but are not super dynamic.


Next.js, Nuxt.js, both of these allow


you to use service-side rendering plus hydration, which


is generally a very robust and stable approach if done right.


The big pitfall is that people are not testing things.


They're like, ah, it's service-side rendering.


It'll be fine.


If your service-side rendering output is garbage


or if the hydration makes it garbage,


it's not going to help you.


So definitely test this with the testing tools,


be it Search Console, be it Mobile-Friendly Test,


be it Rich Results Test.


Just make sure that it works from our perspective,


that the rendered HTML looks like something


that you would want to see.


But generally, we do recommend service-side rendering just


as a way of having a more robust setup.


But service-side rendering can have interesting consequences


if it goes wrong.


It might require a more complicated server-side


configuration.


It might require a little bit of care


in terms of how you are dealing with data sources.


But there's nothing specific for SEO or Google Search


that you need to be aware of.


Hi, Martin, how can we add JavaScript-powered graphs, asks


Andre, to a large website so that it will not


affect Core Web Vitals?


Well, we'll not waste our crawl budget


and we'll not make Google treat this content as duplicate.


Can you point to some typical pitfalls?


One typical thing, if you add graphs to an existing page,


that's not duplicate content.


That's not an issue.


If the only thing that is on the page


is a JavaScript-powered graph, then that's


probably not really that useful in terms of content,


and we might ignore that.


But if you embed the graph into a page that


has content in this index, that's not a problem.


You want to be careful with cumulative layout shift.


If the graph container basically just


pops in eventually, shifting everything on the page


once it does, that might impact your Core Web Vitals.


You also want to make sure that you're not


loading a bazillion different JavaScript files just


to have a graph, because that way, scroll budget,


try to have one bundle for the graphs


or integrate the graph libraries into your bundle


so that you have one API call maybe,


plus the JavaScript source that needs to be loaded.


And then also use CSS so that the containers have


fixed dimensions so that they don't magically


appear, shifting everything on the site.


That should help a lot.


Also, minimizing the amount of JavaScript


is generally a good idea.


If you can pre-render the graph somehow, do it.


If that is not feasible or is too costly,


then JavaScript graphs is just fine.


And then Dave asked the question,


I have a client that's planning a range


of internationalization features, clothing retailer.


And the first step is to change the price and currency,


shipping and sizing info to match the user's location.


That's very nice.


And they make it swappable.


That's good.


I like to hear that, because I hate it.


But the passion is I'm in the US and everything is like 7 inches.


And I'm like, I don't know what that means.


You might as well say like 20 caterpillars.


That's not helping me.


I want centimeters.


I'm living in the metric world.


So they want to do this on the client side


during hydration, which will differ from the server-side


content that Googlebot might see.


Besides seeing the US version, because Googlebot usually


crawls from the US, would this be an issue?


Generally, oh, read more.


Ah, OK, then there's the second step


that we will get into in a moment as well.


Generally, I would say if these two pieces of content differ,


that's not much of an issue.


Just be careful that if the rendering for some reason


fails, like we can't fetch something, some resource


or something, then we might end up seeing the server-side


rendered content.


And then you might get weird movements or even


canonicalization changes, depending


on if the content looks different enough for us


to say, OK, so this is a canonical version of this one,


and then there's these other canonical


because it's different enough.


Second step is they plan to go for the fuller translation


during hydration, also update the URLs in international one.


Yeah, generally, that is fine.


I don't see a potential problem right away.


Again, testing is key here.


And I know that we have Dave here.


Do we have follow-up questions on that?


Yeah, it was more on the second one.


Because the planning, if I arrive from France,


they plan on updating the URL to a French one


and obviously doing a full translation.


But they also want to use hreflang, et cetera.


My concern was if Googlebot, it's


almost like how you kind of advise against something


where you 301 redirect or 302 redirect people, which


they generally say, no, I kind of worry that basically


or wipe out everything.


Because ultimately, you're updating a URL.


I wasn't sure whether Google would definitely


treat that as a redirect to the French one.


And they'd never ever see anything other than the US


version if that's where they're mostly crawling from.


I mean, they're adamant that they really want to do it.


And I can see it's a nice experience.


Once again, they've got a little switchable thing.


That relies on cookies, which obviously aren't


going to work on Googlebot.


So it is a nice, from a user experience,


I can definitely see their arguments.


It's a nice experience.


It's kind of not having to, are you from the US?


Are you from here in a big banner?


It kind of takes you there.


Does still give you the opportunity


to go where you want to.


But I do worry that for Googlebot,


it's going to be no different to if you just force redirect


and then all to the US version.


So the only sort of solution I could sort of come up with


was try and prevent that happening for Googlebot, which


seems a bit fragile and a bit fraudulent.


I never really liked doing that kind of thing.


But would, ultimately, that cause an issue


if that was the only solution?


MARTIN SPLITTENBERG


I'm not sure how big of a problem this might be.


Try it out if you can.


Try to see what happens when we run into situations like that.


And if that is what you expect, then


don't build additional brittle bits and pieces in.


I think a false redirect for Googlebot


would not be too brittle, depending


on how it's implemented.


Yeah, good question.


No, I can't foresee any obvious problems with this,


to be honest.


But then again.


What I'm trying to get into to is just launch the UK version


and then maybe just roll out a country or two


and maybe see where we go from there.


As ever, they'd love to know it all up front hand,


but I guess we don't really know up front hand.


So OK, thank you.


All right.


Yeah, and I mean, staged rollouts


are generally a good idea.


These all or nothing things have a tendency


to break in interesting ways.


Now I've got three questions from Asaf


to finish up the YouTube backlog, so to say.


First question is, the JavaScript site


was client-side rendered setup.


It's using dynamic JavaScript to change the title and react


and uses the React helmet component for description


and other social tags.


Sometimes in search results, you see the default title,


which is what is in the initial HTML, I'm guessing.


And the description looks corrupt as well.


Well, I'm not sure what that means.


The question is, does taking the helmet tags higher


in the head section will make it more stable?


How can I improve the stability for showing the correct title?


First things first, depending on where you see these,


as in if you're seeing these for actual search queries


or if you just see them in the site list of things,


you should be more or less worried.


If it's for actual queries, that's


something that I would be a little worried about.


If you're seeing it in the site query operator,


then that's not really an issue because we usually


don't use default bad titles and descriptions when


the actual queries are being used.


So I wouldn't worry too much about that.


If you move it up, I don't think it makes


that much of a difference.


I don't think that we go by that.


It might just be that we are sometimes picking up things


without rendering first.


And then rendering takes a little moment


until it actually kicks in.


And then we are updating the index.


Index updates take a bit, talking minutes here,


not hours or something.


So moving it up doesn't really make that much of a difference,


I think.


If you can server-side render the metadata,


that's definitely a way to avoid the situation there.


If you can't, then again, I wouldn't worry too much


about it.


Another related question to pagination,


they use an infinite scroll.


And they want to add classic pagination


at the bottom of the page with push state setup.


We don't really have a tutorial on this.


But basically, again, it doesn't really


matter if you have a classic pagination with pages 1, 2, 3,


4, 5, 6, 7, and so on and so forth, to 10,


or just next and previous links.


Because one, the pagination with more links to different pages


just allows us to discover these a little quicker,


the different pages.


Whereas without that, we would have


to do multiple cycles of crawl.


But it doesn't really make that much of a difference.


I know that for e-commerce, we are working on guidance


on this.


But I can't give an ETA on when that documentation


will be available.


All right.


I think those were the questions from YouTube.


Let me refresh, because sometimes some questions


trickle in after I start with no.


OK, fantastic.


In that case, let's open the floor to questions.


Do you folks have questions for me?


I'll have a go at one.


I'm not a super expert in JavaScript.


I've been recently playing with CDNs.


And something I realized was things like static resources,


they're great for that.


You get the resource closer.


But if you're looking at dynamic stuff,


like JavaScript doing APIs, I get the feeling


that the CDN actually slows things down there.


So there's a kind of compromise, because you have


to go via the CDN network.


Is there any kind of best practices?


Do you do API calls on a different domain or something?


So my general guidance for these kind of things


is CDNs are more, for me, strictly for static assets.


So your JavaScript files, your CSS files, your images,


your videos, your fonts, this kind of stuff


goes onto a CDN.


Also, because you can cache it, these things


have a tendency to be cacheable longer.


With JavaScript bundles, that's debatable.


But let's say CSS doesn't change that often, probably.


Fonts do not change that often.


Images do not change that often.


Sure, there's more images, probably.


But I rarely see an existing image being updated every week.


So these are really, really handy,


because they speed up things.


For API calls, I would put them on a separate domain.


I would have something like api.example.com instead of just


putting it on example.com, simply because browsers


have a connection limit.


And actually, Google, the same thing, has a connection limit.


So I think for Chrome, it's 12 connections, 6 connections.


So say you would load lots of images


or make lots of parallel API calls.


You could do six of them.


And then afterwards, they would have


to stall until the first one is finished.


And then it basically goes over.


You can avoid that by splitting it out to a separate domain.


To be fair, if pretty much the only thing


you fetch from the main domain is the static HTML,


and then all the assets come from the CDN,


then you should also be fine with doing your API


calls from the main domain.


It depends a little bit on what your setup looks like.


But in general, I would suggest you


try to design an API in a way that


minimizes the amount of network calls


that go from the browser to whatever server infrastructure


you've got.


That sometimes collides with clean API design,


especially if it's a REST API.


You want these different resources,


but you might need the bundle of resources.


One way to ship around this is a GraphQL API,


where you basically send one request


and say, I want all these things.


And then you get one response back with all these things.


The other way would be to build what I call an API proxy,


or an API facade.


We say, get data for this screen.


And then it makes the actual internal API calls


that it needs to make to fetch all the different bits


and pieces.


And then it gives back one response.


But unless you have lots of API calls,


that isn't really a problem.


But then, again, a CDN surely slows things down,


because you shoot to the CDN.


And then the CDN basically becomes a proxy.


And I don't see the value of that proxy,


especially if it's API calls that you can't cache.


If it's something that you can cache, then maybe,


like, I don't know, stock prices,


you can probably cache, because every user will more or less


get the same result within a certain time window.


Well, yeah, I've seen all of common practices


to put a version number on static resources


so you don't have to worry about time out.


Exactly.