as well as by celebrating milestones you've reached


How to grow your WordPress site with Site Kit? | Search Central Lightning Talks

like your first thousand visitors


or the first hundred dollars that you earn in AdSense.


So let's recap.


Here is how to make sure your site is working for you.


Step one, translate your offline goals into online metrics.


Step two, find the right tools for each metric.


Step three, establish baseline and analyze continuously.


Step four, take action on the metrics,


profit and go back to one.


And if you have a WordPress site, give SiteKit a try.


We made it to support you grow


and help you evolve your online presence


with an easy to understand dashboard


and helpful tips to guide you along the way.


You can find out more at sitekit.withgoogle.com.


JavaScript SEO office hours May 20th, 2020

Hello, and welcome to another JavaScript SEO Office Hours.


My name is Martin Schlitt.


I work for Google.


I am in the Search Relations team, same as John Miller


and Gary Leish.


And yeah, thank you so much for submitting your questions.


And thanks to everyone who joined.


These Office Hours are currently done in a weekly basis.


There's one in the earlier time slot for me


and one in the later time slot.


So you can figure out which one works better for you.


We announce them at youtube.com slash googlewebmaster


slash community.


So keep an eye out for these if you


want to join these recordings.


I'll post the links in the threads


that I create there where you can also submit your questions.


So I'll start with a few other questions.


We have a bunch of really good questions this week.


I'll look through them real quick,


and we'll pick one that is relatively easy to start with.


And then I'll give the audience, the live audience,


an opportunity to also ask.


So is Googlebot still crawling the sites in the second wave,


or has this been updated to include in the first wave?


That's a famous question, and I fear this question


every time it comes up.


Because the first wave, second wave


has been a simplification that we


use to illustrate how Googlebot processes things.


And a lot of stuff happens in parallel inside Googlebot.


So it is relatively hard to explain this properly.


And show this properly.


The second wave, as such, wasn't really a thing.


It is basically we process, we crawl,


which means we make the HTTP request to get your HTML


content and the resources.


And then we process the HTML that we see.


And then while we are processing at the same time,


we are also queuing for rendering,


and then render the page, which means executing


the JavaScript.


And then we process the rendered HTML again.


That means there isn't really a big concern.


The concern has been that, oh, the second wave takes


up to a week or longer.


But we looked at the data last year,


and we found that the median time in queue is five seconds.


So between rendering and between crawling and rendering,


there's on, in the median case, there's five seconds delay.


90th percentile is minutes.


So you shouldn't really worry about that.


All websites go through rendering anyway.


So you don't have to worry about that either.


And there is no such thing as rendering budget,


just to get all these out.


So the second wave has been a simplification


to explain a very, very complex concept and process that we


are no longer using, the metaphor we are no longer using,


because it is a little misleading.


And we haven't found a better metaphor of saying this.


But fundamentally, what we try to say


is, if your content is in the server-side rendered HTML,


that's what we already look at.


So you can, for instance, make sure


that we find important links a little quicker.


We discover them quicker by having it in the processing


right after crawling.


But it's not that huge of a difference for us


if it's only in the content after we render.


Makes no big difference these days.


Cool.


Another question that I get a lot of times


is, when I view a source on a client's website,


I see that content is contained within some sort of JavaScript


object.


But if I view the page in developer tools,


I see that the function has done something with it


and turned it into HTML.


Am I right in assuming that that's OK?


Yes.


Short answer is yes.


If you would look into the testing tools


that we have, like mobile-friendly tests,


your inspection tool, rich results test,


you see a rendered HTML tab.


And if the content is in the rendered HTML, you're good.


That's absolutely not a problem.


It's very normal that the view source is basically


the server sends.


So it's normal that if you use client-side rendering,


that the content isn't in the HTML when you source it.


But once the JavaScript executes,


it actually injects it into the DOM as proper HTML.


And then everything's fine.


So that's not an issue.


All right, before I go through more questions from YouTube,


any questions from the live audience?


OK, if there's no live question, then another YouTube question.


With dynamic rendering, dynamic rendering


means you serve a pre-rendered non-JavaScript version to bots


but give the normal client-side rendered or whatever version


to users.


With dynamic rendering, we're looking


to serve Googlebot a version of a page with navigation


accordions fully expanded with individual HREFs.


But users will land on the version where it's not.


Likewise, we plan to not pre-render cookie constant


banners for search engines versus having


to be using it for users.


Is there a quantifiable threshold guideline,


be it in this encoder or pixels where


dynamic rendering could mistakenly


be considered cloaking?


No.


There is no threshold guideline.


Cloaking detection is complicated


and for obvious reasons.


We don't want any of this to be gambled with.


So we don't really say much about this.


The thing is, as long as you are having a use case like this


where just auxiliary content like consent banners


or accordions are different from what Googlebot sees


and the actual user sees, it's not really an issue.


That being said, it shouldn't even


be a real big issue to begin with.


Even if you wouldn't dynamic render for Googlebot,


that should not be a big problem because we are seeing things


that are invisible as well.


We might just consider them slightly differently,


but that normally isn't hurting you


or isn't causing a problem.


So that's a very good question.


Generally speaking, just don't mislead the users.


And if you are tricking the user to believe something,


if you trick the user into clicking on something


that they didn't intend to do through Googlebot,


so if you tell Googlebot, hi, my website is about kittens.


And then on the website, when the user lands there,


it's about, I don't know, boats.


That's not great.


But generally speaking, making these adjustments


for dynamic rendering is not a problem.


Would you recommend dynamic and or server-side rendering


above using no script tags for showing JavaScript content


to Googlebot?


Yes.


And actually, looking at dynamic rendering or server-side


rendering, I would recommend server-side rendering


because server-side rendering generally


tends to produce faster results for faster websites for users.


So I would consider that over the added complex.


Both of these things, server-side rendering


and dynamic rendering, are complex.


They are not easy to implement.


If you take the effort onto you, then definitely


do the thing that also benefits your users, which


is server-side rendering.


So do that instead of dynamic rendering


unless you can't for whatever reason.


But I would definitely prefer that over no script.


Definitely.


Can you please expand a bit on the loading spinner


from the last Hangout?


Ooh.


Oh, I think, aha.


Right, if, for example, I see the spinner


in the mobile-friendly tool, but on the HTML view in the tool,


I see the content and dump, is there an issue?


Ah.


Last time, there was a website that had a specific question,


like, why is Googlebot not indexing?


Or why is Google not indexing this particular page?


And why does it think it's the same page as this other


completely unrelated page?


The issue there was that the HTML content, actually,


including the spinner, was the same on both pages.


And a page with just a spinner isn't fantastically


interesting in terms of content.


So we thought, hmm, all of these pages,


because they were all just displaying a spinner,


are the same because they are just


containing markup for a spinner, markup being HTML.


And then the page that we ended up picking as the canonical


from the set of pages, it's just a spinner.


So what would we put in search results?


Why would we put a page with just a spinner


into search results?


If you do see your content in the HTML,


you are more or less well off.


I would still inspect and investigate


why we are just seeing a spinner.


Why don't we see the actual content?


Is there something going wrong?


It normally isn't a big issue, especially


if we see it in the content.


We can index it and understand that these pages are


different from each other.


But I would still wonder, why is it showing the spinner?


There can be legitimately harmless reasons.


Like, I don't know, we can't, for some reason,


get to the content quick enough, or one specific thing somehow


specifically failed in Googlebot.


It doesn't fail for users.


As long as the content is there, we can see it.


We can index it.


But I would try to understand why you see a spinner


in the preview screenshot.


Also, the preview screenshot is an approximation,


so don't worry too much if that's not 100% accurate.


Cool.


Any questions in the audience in the meantime


while I'm scrolling on YouTube?


I've got a quick one.


It's about server-side rendering and AMP, really.


Because I guess AMP is a client-side rendering


to some extent.


And I believe you can sort of server-side render.


Is there much benefit to that from a Google perspective?


Or is AMP well understood enough as a framework


that it's kind of more of the performance thing


that you need to have?


It is pretty much a performance thing, really.


So there's not an inherent benefit.


AMP is just one way of making sure


that you build a fast website.


And you can actually, hilariously enough,


you can use AMP in ways that make it not as great and fast


as possible.


So it is a technology.


It is developed by Google.


But that doesn't mean that it brings you big benefits,


really.


The core of Web Vitals is a project


to actually widen the understanding of what


makes a fast website.


And AMP is trying its best to basically give you


a default path to a fast website.


But that's pretty much it.


So as long as you make your website fast and nice for users


and have good content, then AMP or no AMP


doesn't make that much of a difference.


One more YouTube question.


I have a new site in which each article comments


are enabled by users.


The comments are grouped with a JSON-LD called comment.


I've seen that Google does not cache those comments as part


of the HTML of the page.


Are you looking into the Google cache?


Because that's not a good testing tool.


I wouldn't worry about that, to be honest.


I would check with the testing tools


if it shows up in the HTML.


If it doesn't, then it's a little tricky,


because the JSON-LD on a page should augment the page


content, which means it should describe


what is on the page anyway.


If you have JSON-LD that doesn't match anything on the page,


might not give you as much benefit as you wish.


So tricky, tricky.


What else do we have here?


Does Googlebot recognize that select sections of a page's


content can be hidden by show and hide buttons


and therefore decrease the priority or value


of that page content based on the fact that it can be hidden?


In other words, does Googlebot assume


that because the developer created the content such


that it can be hidden by the user, therefore it must not be?


No.


We don't even click on things.


So if it's visible by default, we wouldn't even


know that you can hide it.


Also, if you choose to hide it, that


doesn't mean that we completely ignore it.


It just means that we might think,


OK, so this is apparently not the most important piece


of content here, as it is not visible.


So we might decide to look at it slightly differently.


But generally speaking, that's not an issue, really.


I have JavaScript links to related products in my web


store.


Is it convenient, or should they be in HTML?


Well, I asked the clarifying question


what they mean by JavaScript links


and what they look like in rendered HTML.


And the answer I got is yes.


So I'm not 100% sure.


But there is documentation for this.


If it is a normal Ahrefs URL, then yes, we can see it.


It's fine.


If it is anything else, like a button, or a diff, or a span,


or something on click, or even an A on click


without an Ahrefs, or an empty Ahrefs, or something like that,


then no, that's not good.


So there's two pieces of documentation


that are helpful in this case.


One is the webmaster guidelines on making links crawlable.


And the other document is the JavaScript SEO basics


guideline where this is being called out.


So as long as the links are proper links, they are fine,


even if they are JavaScript-generated links.


Anyone else with a question from the audience?


Martin, my agency is building a lot of PWAs right now.


And what resources could we use to make sure


that those sites are still crawlable


and there's no rendering issues?


That's a good question.


So we do have the JavaScript SEO basics guide


that explains a bunch of stuff that you should probably


have a look at.


There is even a code lab linked from that guide that


shows you what to look out for in single-page applications.


And for us, PWAs are basically first and foremost websites.


So we don't really care if it's a PWA or not.


If you want to test how Googlebot sees this,


you can use the mobile-friendly test.


You can use the rich results test.


If you have it somewhere online where you have a domain


authorship verified already, you can use Google Search Console


and the URL inspection tool to see


how we're seeing the content.


And Search Console also shows you


if there are problems indexing certain pages or views


of your progressive web app.


So definitely use the testing tools available from our site


to check the rendered HTML.


That's the most important bit.


Does that help?


AUDIENCE MEMBER 1 Yeah, it does.


Thank you.


MARTIN SPLITTENBERG Awesome.


Great.


All right, what else do we have here?


Then we have a question where I'm not 100% sure what to say


without looking at the site.


So there's one question saying, if you're


running a single-page app and it was set to render client side


and the HTML preview shows a blank page, what would


be the best way to fix this?


The answer to that question is it depends.


When you say by HTML preview, what you mean


is the rendered HTML in any of our testing tools,


then look at potentially there is


a problem with us fetching the JavaScript resources.


If we can't, if they are roboted, for instance,


what's the reason?


If there is a robots issue with one of the APIs


that you have to fetch to actually render the page,


then that might incur an issue.


Generally, check if the HTML content is entirely blank


or if there's just a display issue


if you are looking at the screenshot.


But it really depends.


There's no general answer, but I would definitely


look close at the testing tools.


Also look at the resources loaded.


If there is any 500s or robots issue or some resources not


going through, then that's something


that you want to check on.


I know that some CDNs apparently sometimes


do cause us to either see outdated JavaScript


or to see some sort of warning because they


think we are a bad bot or something like that.


I've heard stories of this.


I've never actually seen or experienced that,


so it's a tricky one.


But definitely use the testing tools


to dig deeper into why we are not seeing your HTML.


Right.


Any questions from the live audience?


I'm running low on questions from YouTube.


Just a quick clarifying one.


I see a lot of people sometimes try and use the Google cache


from the web thing, from the web search, to diagnose them.


My understanding is it, Brian, is probably


a pretty poor measure anyway.


But is that the initial HTML?


That's not the rendered HTML, is it, that's shown in there?


It often, yeah.


It often is just the, so sometimes


it is the rendered HTML.


Sometimes, well, most of the time it is the initial HTML.


A bunch of times it's even like an older version.


The Google cache is a convenience feature


in case a website goes down.


It has been built many, many moons ago.


And I don't think it's actually actively being maintained.


So as the indexing infrastructure keeps changing,


it doesn't really change with it because there's no one


attached to it, as in there's not a Google cache team that


works to keep this feature going.


Because as it works, it works reasonably well,


and it's meant as a fallback for a website going down.


It's not meant as a way to find out


if Google sees your content or it's not a debug tool.


I know that there is a Help Center article that


makes it sound like it is.


I'm actually working with the team behind that Help Center


article to figure out if we can clarify the wording


and actually point to the right debugging tools,


but don't use it as a debugging tool.


Tom, I got your question on the recent core update,


but I don't comment on ranking.


And I actually don't know about the core update


because that's not my area of expertise.


So I won't say yes or no to this because I don't know.


Good question.


Don't know.


Then there's a question 11 minutes ago.


Nicholas asked, is it normal to see sometimes the headline


tags only visible with JavaScript?


Is it a best practice to have headline tags in a JavaScript?


I mean, if your content is client-side rendering,


then you will see all of your content


only when you actually run JavaScript rendering


and crawling.


So that's fine.


But if you only see, like, if your content is there


but the headlines are not there, that's weird, I would say.


That's a really weird use case.


And I would definitely ask the developers


why the headlines are not loading


as part of the rest of the content.


But if all the content only loads with JavaScript enabled,


then that's OK too.


That's not a problem.


But it's not a normal thing to just load


your headlines with JavaScript and the rest of the content


is there.


You should not do that.


You should always have your HTML in a good shape,


be it either coming from JavaScript, that's fine,


or having it static, that's fine too.


There's a question about a really large React.js website.


The site was launched in January.


But as of today, there's only 37,000


indexed pages out of 550,000 or more.


Every couple of days, 800 pages get indexed.


I have tried to speed this up by not limiting


Googlebot crawling.


So we are now at 512,000 pages, apparently.


But yeah, it apparently takes a long time for us to index.


Well, first things first, I don't


know how you got to 550,000 good URLs with interesting content


for your users.


But I would say I have a feeling that a few of these URLs


might not be as relevant.


So maybe make sure that we don't spend time


on them crawling and indexing.


And yeah, there's no guarantee.


That's just the thing.


There's neither a guarantee that we will crawl and index


all these pages and URLs.


There's no speed guarantee.


Sorry.


We will have to see and wait.


But I think if we already have discovered 500,000 pages


since January, that's a relatively good rate.


But the other thing is indexing doesn't


mean that it's showing up.


And it doesn't mean that it's useful.


So you shouldn't hunt these metrics down without thinking.


If I have a page that is indexed,


but no one ever sees it in search results


and no one ever clicks on it, then what have I gained?


Not really much.


So think about a user's perspective,


or from a user's perspective, try


to understand what are the pages that you really, really care


about.


And I have a hard time believing that it's half a million pages


that you really, truly, definitely care about that


are for the masses.


Try to get the main vectors into your website indexed.


Because they probably have it.


Try to get those pages indexed that


have a high chance of actually driving traffic.


And then the rest is maybe not so important.


So indexing and ranking is not necessarily the same thing.


But that's something that I hear a lot of times.


Then there's another question 23 minutes ago from Roshan.


My crawl rate is very low.


It only crawls 20k to 40k URLs.


That's actually not too bad.


Could it be why I lost all my discovered traffic


in September last week?


And in September last week, it has been on decline since then.


90% drops in September.


No.


Crawl rate has nothing to do with either indexing or ranking.


If not much happens on your page,


then a low crawl rate is perfectly fine.


If we have seen all your content and have put it in the index,


then why would we continue to crawl?


The other thing is discover is an organic feature.


And that can always come and go.


More websites come in.


Interests keep shifting.


Happens.


And then, for instance, if you are a travel blogger,


you'll probably see less traffic now than you usually see.


That just happens.


That's how organic traffic works.


But there's nothing inherent to like, oh, crawl rate


is an indicator for it.


No, it's not.


It's not an indicator for quality.


It's not an indicator for demand or traffic or whatever.


So no need to worry about that.


Hello, what is the best way to clean unused JavaScript


code on a website?


Right, that's a good question.


Generally speaking, I think the Google Developer Tools


show you unused code.


And that's probably the best for developers, because they


can see which part of the code is actually


unused when they go to certain websites,


as they have the unlimited and unminified sources


available to them.


But I think looking at web page tests


to see where's the biggest amount of code that we have


is probably a good starting point


to see where you need to dig a little deeper.


If you use things like Webpack, they usually


have a sort of like bundle analyzer,


where you can see what kind of dependency


or where does the size of my bundle come from


or where does my JavaScript come from.


You can do tree shaking.


So your developers can write code in a way


that if they take in an entire library


but only use like five functions out of 100,


that the rest of the code is actually split off


automatically.


If you want to learn more, there's


a bunch of really good stuff around web.dev slash fast,


for instance, and more resources on web.dev in general.


So I would like look into those.


It's a whole huge topic on its own.


All right.


Any further questions from the audience?


Quiet audience today.


Let's see, maybe I missed the question here.


I've got another quick one, if that's OK.


A lot of the advice around speed and performance


always used to be about basically stitching


all your JavaScript files into one big, big file.


Is that still really the case with things like HTTP2


and things like that?


That's a good question, and it's a tricky one.


So HTTP2 does make that a little less important,


but still I would say first things first,


Googlebot still uses HTTP 1.1.


A bunch of other things that we've


seen Googlebot still uses HTTP 1.1.


A bunch of browsers might fall back to it in certain cases.


Not every web server is configured to use it.


So there's still a bunch of people


that are not looking at HTTP2 and its possibilities.


But generally speaking, even if we ignore HTTP2,


the idea of stitching the entire bundle in one


has become an it depends kind of situation.


As we build larger applications, a single bundle


is definitely greater than a bazillion small JavaScript


files, because as they have to get over one by one,


I think even multiplexing has some sort of limit, I guess.


Even if it doesn't, then still it


means a lot of data needs to be going over the wire


that you might not need.


If you imagine there, you have a huge website


with a blog and a forum and a shop,


and you have one big bundle, then that's


not exactly fantastic.


Because if I just come for the shop,


why would I need the JavaScript for the blog and the forum


and all that kind of stuff?


So if you can, split your bundles along reasonable


boundaries, I would say.


So try to bundle up as much as you can,


but try to split it as reasonably as you can.


Because if you really, really care for the traffic


to your blog, then don't make people


download the bundle for the rest of the website


when they don't need it.


You can prefetch these things so that you can speed things up


a little bit.


You can use a service worker in the background


to fetch these things in the background.


That's a much better practice than having everyone


download a huge bundle.


The other downside with one huge bundle is caching.


If you have this huge website that has a bunch of stuff


and one JavaScript file, like the app.js file,


the browser caches it.


Great.


But then I change something in the blog.


And everyone, even those who never visit the blog,


need to actually download the entire app.js again.


If I have a blog.js, let's say a common.js,


and I have a shop.js, then if I make a change to the blog,


only the blog.js needs to actually be re-downloaded,


and the rest can stay in the cache.


So the answer is it depends.


And it needs a lot of measuring and trying things out.


There's no one size fits all.


And HTTP2 makes that even more complicated,


because you then have to ask yourself, OK,


so how can I make the best use of the multiplexing


in the connection?


But realistically, even without HTTP in the equation,


you have to measure things for yourself


and try to make a reasonable call as to where


to cut your JavaScript so that it


fits the main avenues of your application or website.


Just to follow on that as well, I've


kind of heard always what I'm kind of a bit


talking about is that, obviously, if you say


if you're serving a couple of JavaScript files,


might be a break in there for allowing


things like interactivity.


So it seems maybe a bit of a fudge that.


I don't know, because it doesn't seem a predictable way


in many ways.


It isn't that predictable, and it's tricky.


I'm not saying I have easy answers.


But unfortunately, sometimes there


isn't a nice and easy answer.


Oh, there's a question I haven't seen.


Using Angular 9 with server-side rendering on my website.


As I add more features to the website,


my overall JavaScript file size increases.


Yeah.


Following is a page speed insight for one of my features.


OK.


You can see that the field data for that page in origin


is not looking good.


I want to know the magnitude of the impact that can.


I mean, there isn't an easy formula or something.


But seeing like a 70% drop in traffic over two-ish,


three-ish weeks, I doubt that that's exclusively


because of performance.


Also, because you actually have a relatively good performance


score, your first contentful paint is in 2.3 seconds.


I'm sure that can be done better,


but that's not like the worst thing ever.


You have a first input delay of 300 milliseconds.


You have a maximum potential first input


delay of 290 milliseconds.


That's not really a problem.


I guess you can improve caching a little bit


according to this audit.


But I don't think this is the biggest thing


that you need to look at.


Traffic drops can happen.


That's something that just organic traffic is tricky.


It is seasonal.


It is a little unpredictable because other websites


might come up.


It might have to do something with the core update.


It can be anything, really.


Just try to understand what still works,


where the traffic is coming from,


what traffic you are no longer getting,


and then try to find out why you are not


getting that traffic anymore.


But I don't see performance as the reason for this.


Good.


Any more questions from the audience?


I'll annoy you again with another one if nobody else wants


to.


Go for it.


No, yes.


Does the web rendering stuff ever take notice


of cache control header?


Or is that pretty much always ignored?


Ah, we pretty much always ignore the cache control header


because so many resources are under-cached


that it would make our rendering a lot,


or would put a lot of more load on our crawling and rendering


infrastructure.


So cache control isn't the best way


of telling us to actually re-download something.


That's why I am rallying for using versioned URLs


as an app.somehash.js so that when you change the content,


you change the file name because then we


have to download it because it's a new file name.


But we do look at it sometimes, but it's not a guarantee.


We might ignore the cache control header,


which is unfortunate sometimes.


But generally speaking, it is too unreliable a signal,


unfortunately.


Right.


So last chance for the audience to ask a question.


If there are no further questions,


then we'll wrap this up.


And three, and two, and one.


All right.


Thank you so much for joining.


I hope that this was helpful and a little bit of fun,


maybe, as well.


I was looking if the sheep that the neighbors have


are showing up.


But unfortunately, there's no sheep to show you.


If there were, I would show you the sheep.


But thank you so much for joining this.


Stay healthy.


Stay safe.


Have a fantastic time.


And see you next time.


Goodbye.


Bye.


JavaScript SEO office hours, June 24th 2020

Hello, and welcome to a new edition of the JavaScript SEO


Office Hours.


Today, I have 13 people here in the Hangouts.


We have a few questions on YouTube,


so stay tuned for this episode, so to speak.


I'm not even sure if this is an episode.


I think it's like an episodical thing.


It's a series of Hangouts, but I'm never


sure if it's like an edition or an episode


or what would I call it.


Anyway, whatever.


I'm digressing.


There is a bunch of questions on YouTube.


Let's start with the YouTube question


before we go into the audience questions.


So showing different content for an existing user


based on the JS cookies.


Will this hurt my SEO?


For example, A user is new.


For this user, we show one promotion, like get started.


And user B is already an existing user.


For this user, we show the latest promotions.


Does this impact my SEO?


Kind of, yes, it does.


I mean, it doesn't really, but it kind of


does in the sense of that Googlebot does not


run with set cookies.


So we would only see the content that a new user would see.


I would suggest to have different landing pages


and then actually expose both of these through links


to Googlebot so people can also see the latest promotions


if you care about that.


And that would allow you to show the same content to both users


and Googlebot just you would move the people


to different starting URLs.


For instance, you would redirect people if a cookie is present.


That is fine.


That wouldn't hurt your SEO, per se.


Can a JavaScript front end technology change,


cause a drop in Google ranking?


We have a few million visitors per month


and follow Google guidelines.


When we switched from Angular and PHP,


so server-side rendering, to Vue and Nuxt


with server-side rendering, we lost rankings immediately


about minus 20% traffic.


Our overall Lighthouse score went from 40 to 85.


So do you think it's possible for a site


to drop because of a change in JavaScript?


Bonus info.


We did notice that other sites have been hacked


and that the hackers have made subfolders on these pages


with our entire site copied in smart canonicals


to spam shop pages.


I'm not sure what smart canonicals are, but OK.


Could it be that because of the timing of this,


Google has considered some of the hacked sites


as the true source of the content that belongs to us,


as Google has to understand our new JavaScript?


And if so, what to do?


Examples of sites in case these are helpful.


These are helpful, but I'm not sure if I have the time


to look at them right now.


Generally speaking, a switch in technology


should not mean that much of a change in pretty much anything,


except for I'm guessing you didn't just change technology.


You probably also changed maybe site structure,


or maybe you changed the way that your content was


presented.


If you made changes above the threshold of where technology


is serving, we don't really care about the technology that runs.


And we care about the content.


So if you have made changes to the way


that you present the content, that


would mean that we would have to take time to actually


re-understand things.


It could coincide with the other hacks


that we're trying to take content away.


You can check if we consider your pages canonical


for the content that you produce.


So that's something that I would double check.


And ranking can always change a little bit.


There's things happening on the web all the time.


So ranking changes are not necessarily


something that comes from the technology that you choose,


but other factors might be part of that as well.


It could be that it was an update to the algorithms.


Ranking isn't really my area of expertise.


But generally speaking, assuming that all the content is


visible and present in the rendered HTML,


and that your website apparently got faster,


I wouldn't expect that to be the source of ranking changes.


It could be these hacked websites.


It could be that just the way that you represent your content


has changed fundamentally, and that we


need time to reprocess it.


That's also the same thing with people


are redoing their sites, revamping their sites,


changing the way that the content looks like


and is presented to the user.


And then they see changes like this,


because fundamentally, you have created a new website.


If you really just change the underlying technology


and everything else, the way that you


show the content and the URLs and all that stay the same,


then we wouldn't have seen that much of a difference.


Then there's a question.


Hi, Martin.


I am helping a nonprofit, and I'm


afraid that the content in a very important part


of the website is not indexable, because it's made off


of widgets loaded by scripts.


Will you be able to confirm and maybe have


some tips on how they could add the content differently


to make sure it's indexed by the bots?


What you're saying is you're afraid that,


which means you're probably not sure that this is actually


a problem.


Content loaded by scripts and widgets


is not exactly an issue because it's not


loaded by scripts and widgets is not exactly an issue per se.


I would plug this URL into any of the testing tools,


be it the URL inspection tool, be it the mobile-friendly test,


the rich results test, and look at the rendered HTML.


If the content that you care about


in this section of the page is present in the rendered HTML,


then there's nothing to worry about.


If it isn't, then you would have to investigate why


it is the rendered HTML.


There's a follow-up question.


A critical question on the first question,


the first question being the one regarding showing


different content based on cookies.


How about websites where new users, including Googlebot,


are shown an HTML site but logged in users


are shown a JavaScript home page without any textual content


at all?


Per se, what happens once you're logged in


is not something that search engines really care for.


I mean, do what you feel is right.


We can't see things behind the log in to begin with.


So I wouldn't worry about that, to be honest.


For SEO reasons, anything behind the log in


is invisible to us.


You're welcome.


I think it's time to take a few questions from the audience.


Anyone has a question today?


Yes.


Hi, Martin.


Hi, Christian.


I would like to start because I have two questions,


but maybe the first one.


It's from a client of mine, and I only have it in German.


So I tried to translate it.


You can actually ask the question in German hypothetically,


and I'll happily answer it in English.


But I think if you can translate it,


that would be fantastic for the audience.


Yes, yes.


I will try.


So it's about a single page application.


And it's under a domain which is redirected to the www domain.


And this causes problems with end users


who have a bookmark to the non-www domain.


It's causing a CORB, C-O-R-B, error


because the service worker who delivers pages


from the cache for the database content


points to the other domain, which is with www.


And yeah, this is causing these CORB problems.


And the question would be, is it OK to redirect to the www


only if the user agent is Googlebot or a crawler


and show the non-www version for the normal page visitors?


So just making sure that that was a lot and it's early.


You have a non-www and you have a www domain.


And the problem is with people who bookmark the non-www.


The www.


Why don't you redirect everyone to the www domain,


including those who have bookmarked the non-www domain?


Yeah, because then the client says


there's a problem with database fetching and the service


worker because it shows these CORB errors.


Right.


But that means that this service worker, or whatever it is that


is making these requests, is requesting some while being


on the www domain, request something


on the non-www domain.


Yes.


That's the root cause.


You should fix the root cause.


In that case, your service worker


should use the www domain to make fetches.


And then you redirect everything to the www domain.


So if that's difficult from a technical point of view,


I'm not into it so deep.


So I can't.


Yeah, would it be OK to only redirect the users


to the www and not Googlebot?


I think that is OK.


But to be honest, I think it's as much effort


to do that than to fix the service worker making


requests on the wrong domain.


I think the effort is pretty much the same.


It's different people having to do this, I agree.


It's very likely that someone else


makes the redirects versus the person who


writes the service worker.


But I would argue you can do it.


I think it's fine.


I don't see an inherent problem with it


to do the redirect only for Googlebot.


It might turn out to be tricky to test things later on.


And you might run into situations in the future


where it's harder to debug problems


with this kind of setup because you


are treating Googlebot differently than normal users.


So I would recommend to fix the root cause, which


is the service, like redirect everyone to www


and then make sure that the service worker does


the thing it needs to do properly on the www domain


rather than trying to make a request on the other domain.


This might be as simple as changing one constant


or variable somewhere in your scripts


to actually use the correct domain.


But if you can't do that, then sure, a technical workaround


is that.


But I would consider that not a solution but a workaround.


OK.


Thank you.


You're welcome.


Happy to help.


And maybe if I may ask my second question.


Sure, we have time.


OK, cool.


Another client who has a home page and a website.


And before loading the main content,


he shows content to collect user content for legal issues,


something like that.


And only if the user content is collected,


the main content is loaded.


And this all takes place on the same URL.


And the question would be, would it


be OK to not show Googlebot this legal user content page


and immediately load the main content?


So you treat Googlebot more frequently


than the other users.


Would that be OK from an SEO point of view?


I think that is fine.


Depending a little bit on our heuristics,


we might false qualify this as cloaking,


which then might cause issues.


But normally, normally, you would have to test this.


It should not be a problem.


I advise a little bit against it.


It's probably, unless there's literally legal reasons


not to load the content in the background


and then just make the user consent beforehand,


there might actually be legal reasons


to not load content before someone has given consent


to something.


In that case, yeah, that's fine.


That's a workaround that I would say is OK.


OK, and you would consider this a low risk


if something goes wrong?


I would not consider it low risk.


OK, so we have to try.


We have to test it.


I would try that very carefully and be ready to roll that back


if needs be.


OK, thank you very much.


You're welcome.


Suzuki has a question.


Suzuki-san, excuse me.


This may not be related to JavaScript.


Well, it kind of is probably related to JavaScript.


How can I identify the largest content when my LCP is slow?


And question two, how can I find what elements


cause poor content layout shift, cumulative layout shift?


The LCP, that's a tricky one.


I would probably use the web page test


and look at the film strip.


What is the largest blob where it basically is mostly white


or whatever the background color of your website is to suddenly


there is content, whatever loads in that large blob


is probably what's blocking the largest content full paint time?


Very likely going to be images, but it could hypothetically


also be text blocks.


But it's more likely that these are images


that make your LCP time high.


For which elements cause poor CLS,


you can look at the requests and then block individual requests.


You can either use a script that,


so if you are looking for, let me see


if I can find this real quick.


Tobias Wilmann wrote a script using Puppeteer for this,


which I think is pretty cool.


Haven't tried it out, so I don't know


how well that actually works.


But I think Tobias is usually producing good stuff,


so I wouldn't be surprised if this is actually pretty cool.


I posted it in the chat, and I'll


make sure that I put it in the YouTube description as well.


I might forget that.


Let's face it, but I'll try to remember and put it


in the YouTube description when this video goes up.


But basically, I can probably show this.


Maybe I can show this on an example.


Do I have a good example, though,


where I load different pieces of content?


Yeah, I think I do.


Well, we'll find out.


Let's see, 50 lines of code.


A blog, and then if I go out, I'll


share my screen with you in a second.


Let's see, a Chrome tab, and I want to share this Chrome tab.


So if I'm not sure what causes things here to go wrong,


I can go into the Network tab.


I can load things.


And then if I'm wondering, actually,


let me get this away here, and maybe I start with images.


There should be this one, for instance.


I can say I want this URL to not actually be loaded.


And then I can load again and see


if things are shifting or not.


And you can see that this image has no longer been loaded.


You can basically go through the different elements


and then run your metrics to see if you are in the hold on.


This is where I wanted to go.


You can run your metrics to see if that makes a difference


or not.


I think it respects request blocking, I hope.


But anyway, there is a puppeteer script


that does this for you, which is even nicer


and gives you a better feeling for what's happening.


But I'm pretty sure.


Wow, why is Lighthouse warming up so long?


Well, to be fair, my computer is acting up a little bit


this week, so I'm not super surprised.


Not super sure what we are getting.


But OK, unfortunately, we don't get


to see if the image was loaded or not.


But I'm pretty sure if I basically now go and block


pretty much every image that we will end up


getting a better score.


So you can have a look at what requests


make the biggest impact on your scores.


Oh, Lighthouse 6 gives you this info.


That's awesome.


I didn't know that we have that built in.


Dave, thank you very much for following up on this one.


So yeah, PageSpeed Insights does that.


Lighthouse 6 will roll out, I think,


with the next Chrome release probably.


Or you can install it from GitHub if you want to run that.


That's pretty cool.


Then it tells you which elements are affected.


Anyone else with a question from the audience?


We have a few questions on YouTube


if no one from the audience has a question.


All right.


So I'm not sure I understand this question,


but maybe some of you can help me with this.


Why would first contentful paint and largest contentful paint


be far apart in Lighthouse and PageSpeed Insights?


The Performance tab says they happen at the same time.


And as a real user, I see the whole page load,


all these elements at the same time.


Well, it is possible that with Lighthouse,


it runs on your computer unless you run it


from web.dev slash measure.


PageSpeed Insights runs from the cloud,


so it might give you different data if that's the question.


Also, first contentful paint and largest contentful paint


do not necessarily have to happen at the same time,


especially if you're on a device with slower CPU.


And these can divert quite substantially.


They might load at the same time on your device,


but they might, as in your computer.


But it's unlikely that this might take longer


on a slower network connection or on a slower


device with a slower CPU, like an older mobile phone.


That happens.


How is largest contentful paint by page type determined?


What does that mean?


What page type?


I don't fully understand the question,


because I'm not sure what you mean by page type.


The element seems to shift.


Well, that has nothing to do with largest contentful paint.


That would be CLS.


That would be Communicative Payout Shift.


Jennifer, if you want to ask this question with a little


more detail in the next Hangout, let me know.


Or basically, just go to YouTube and put it


in the next Hangout comments or in any of the next coming up


Hangout column threads, because I'm not sure what


you mean by page type.


Largest contentful paint, how it's determined


is explained quite nicely on web.dev.


But I'm not sure what you mean by page type.


Then we have a sports betting website


that streams sports games and data to end users.


We use JavaScript framework, Ember.js in this case,


to render the sportsbook views.


The page URL structure is mysite.com


slash live sports hash football slash England slash competition


ID slash match ID.


Our customers want us to get rid of the hash sign


and provide them with slash routes in order for Googlebot


to crawl them and index them.


However, when the match time comes and when the match ends,


the URL returns a 404.


Question, is there a real reason to remove the hash?


Will Google index those temporary JavaScript routes


if yes, then what would happen if a few days after it,


the page returns a 404 status?


So generally, if you want these things indexed,


you need to remove that hash.


But if these URLs are short-lived,


and by short-lived, I mean 90 minutes of a match,


then I don't see why you would do that,


unless you do have them up front and they are there


for a week or longer than a couple of days or minutes,


then I think if you want people to find it before the match


or while the match is running, then it


makes sense to have these URLs present and rendering properly


a couple of days beforehand before the match.


It's fine for URLs to go 404 afterwards.


What would happen is if you give us enough time


to discover these URLs, and enough time can be a few days


or even a week or so, because we might not


crawl your page quite often, quite that often,


or your site quite that often.


But if we do discover them and crawl them,


we would index them.


We can't do that if there's a hash in there,


so people would not find this while the match is running


or before the match starts.


So if your customer relies or wants people


to find this content before the match


or while the match is running, then you


should definitely get rid of that hash in the URL.


And once you are returning a 404, even using JavaScript,


you're returning a 404 because the match is over.


That's fine.


We'll eventually see that it's a 404 page


and then remove it from the index again.


So depends a little bit on what your customer wants


and how your customer thinks about these URLs.


It's fine to have these temporary URLs indexed.


Nothing wrong with that.


But if you want them indexed, you can't use a hash route.


Can we somehow only use JavaScript


to make it work in common form?


Yeah.


But with a common form, I mean, that's


a form that you use to enter comments.


You can totally use JavaScript to do that.


I'm not sure I understand that question.


If you have follow-up information,


please post a follow-up question in the next Hangouts.


Right.


That's it for the YouTube comments, I think.


Let me reload the page because sometimes people


are commenting while the Hangouts is happening.


No.


Any more questions from the audience then?


How is your time?


I've got a quick one, Martin, if that's all right.


Sure.


Came across a couple of questions and stuff


in the forums and that where people are pre-rendering.


But they're serving everything with the JavaScript


after the pre-render.


So they're enacting but leaving it in place.


It's my understanding that's missing some of the point of it.


Are you better to remove that JavaScript?


Does it particularly cause troubles if you do leave it in?


Will Google try and then render the page anyway?


That's a good question.


And I think if I think about this,


so if I want to pre-render, 90% of the time,


probably people don't have to pre-render for Googlebot.


They might want to pre-render for other bots


that don't run JavaScript.


But unless they have a technical reason


that they should fix elsewhere, then using pre-rendering


as a workaround is kind of, it's a workaround.


And if you are then not removing the JavaScript


from the pre-rendered page, then you're kind of, yeah,


you're kind of missing the point.


Because then, sure, we do have the content in the initial HTML.


But if your JavaScript kind of overrides it and overrides it


incorrectly, then we might end up


still seeing the incorrect content or the missing content


or whatever it is.


If your JavaScript just overrides everything


and doesn't work properly, and if that's


the reason that you pre-render in the first place,


I would suggest if you pre-render for Googlebot,


because your JavaScript causes the content to be incorrect


or missing or whatever, do test very, very carefully


if your pre-render solution actually fixes the problem


when the JavaScript remains on the page.


If it doesn't, I would just fix the JavaScript


and get rid of pre-rendering.


If it works, don't touch it.


If it's not broken, don't fix it.


It's fine.


You can do it.


I kind of feel like if your JavaScript renders fine


in the first place, then pre-rendering is just


a way to burn money.


Because servers cost money, and pre-rendering usually


takes server load.


So I would say if you pre-render,


the page should come out without JavaScript at the end.


But if it works for you otherwise, then fine by me.


You do miss out on the benefits, though.


AUDIENCE MEMBER 1 OK.


Thank you, Matthew.


MARTIN SPLITT I thank you for the question.


Anyone else with questions?


I see there's a bit of Twitter chit chat.


Oh, people are posting random stuff on Twitter.


Sorry, do we have a question?


AUDIENCE MEMBER 2 Yes, Martin.


MARTIN SPLITT Awesome, Avaro.


AUDIENCE MEMBER 2 How are you?


MARTIN SPLITT Great.


AUDIENCE MEMBER 2 I have a quick one.


It's not probably related with JavaScript,


but we're thinking about implementing FAQ schema markup


in JavaScript.


FAQ schema markup in some of our pages.


And I'm wondering if that is going


to decrease a little bit the page speed of the website


and adding this LAD JSON data inside the page.


And also, if this is also a good practice,


if you are trying to acquire customers for, let's say,


paying keywords, all the keywords that are bringing


direct sales, rather than informational or transactional


queries.


I don't know if you got my point.


MARTIN SPLITT I got your point.


AUDIENCE MEMBER 2 That's pretty it.


Thanks.


MARTIN SPLITT Right.


You're welcome.


Great question.


So FAQ markup is most helpful for informational queries, not


really customer acquisition, especially


because if you are too blatantly advertising your services


or products as an answer to a non-product-related question,


doesn't really have that much.


FAQ is more a way to give people information around something


that you offer quicker than having


them to go to your website.


Also, adding JSON-LD blocks does not really


impact page speed that much.


It does add a few bytes to the website,


but that's insignificant if you look


at the amount of JavaScript that you usually ship,


the amount of images that you usually ship.


It's a small, small, small percentage,


and it doesn't really make the browser slower while parsing,


because it basically parses the script,


sees it's not a JavaScript, and kind of skips it.


So I wouldn't worry too much about page speed implications.


I think having FAQ data can help, especially


for customer service-related informational queries,


but I don't think it has much impact in terms


of customer acquisition.


But I might be wrong about that last part.


GABRIEL KREIMAN-MARTINEZ-HENRYUHN.


I agree.


I agree with you, actually.


Thank you.


MARTIN SPLITTENBERG.


You're welcome.


Awesome.


Do we have other questions from the audience?


No further questions?


All right.


So five, four, three, two, one.


In that case, thank you so much for watching.


Thank you so much for posting questions on YouTube,


joining this Hangout, and asking them live as well.


I hope you have a fantastic day.


Stay safe.


Take care.


Thanks for joining, and see you soon in the next JavaScript


SEO Office Hours.


Bye-bye.


English Google SEO office-hours from March 4, 2022

All right, welcome, everyone, to today's Google Search Central


SEO Office Hours Hangout.


My name is John Mueller.


I'm a search advocate on the Search Relations team


here at Google in Switzerland.


And part of what we do are these Office Hours sessions,


where people can join in and ask their questions


around their website and web search.


We have a bunch of submissions on YouTube,


so we can go through some of those.


But maybe we can go through some of the live questions


beforehand.


Let's see.


I think, David, you're on top of my list here.


Yes.


Hi, John.


I brought you two questions today.


The first is, if I have a content project that


is updated daily, for example, like the rates


of cryptocurrency or something similar,


what is the best way to do it?


Is it to create a new article every day, so also a new URL?


Or is it to update the existing article daily


so the URL stays the same?


What would you recommend?


I guess I would try to figure out


if you're creating something new or if you're just


updating something existing.


If you have a page where you're just updating the prices,


then you don't need to make a new page for every price change


that you make.


You just update those prices.


On the other hand, if you're providing new information,


then that feels like something that should live


on a separate URL, where people can


go directly to that specific piece of information.


The advantage also of keeping the same URL


is that over time, it builds a little bit more value.


And then people understand, well, this is actually


the place to go for this piece of information.


So for example, if for every day for a new price,


you create a new page, then if people search for,


what is the current price for this product?


Then they're going to find some of these.


But it's going to be unclear to us which one of them


they should show.


On the other hand, if you have one page where you just


update the current price, then we know for the price,


this is the page.


So that's kind of how I would look at it there.


OK, so when I would do it like this,


would you recommend to force indexing also manually for this?


Or do you think it will get caught by Google over time?


I think it should get picked up automatically anyway.


Usually, if you rely on using the submit


to indexing in Search Console, to me,


that's almost a sign that Google isn't convinced


about your website yet anyway.


So even if you were to force it into the index,


then it's not a guarantee that it'll actually


be shown in a visible place.


So I think it's fine to use that in the beginning


to kind of get started.


And if you have really critical updates to use it then,


but for kind of the natural updates on a website,


that shouldn't be something that you have to use.


OK, but do you think it's necessary,


this one URL which is daily updated,


to also update the article date?


Should that be necessary?


I mean, if you're changing something significant


on the page, yeah.


Why not?


Yeah.


OK.


The second question I brought to you


is we have a service footer now where this footer shows up


on every site on our website.


And we have used H2 tags for the headings of this footer.


So when I look at the hierarchy of headings now,


they appear on every page, like the H2 tags.


And my question is now, is it the best way


to change these H2 tags to like normal paragraph things?


Or is there a way I can separate the footer, like for example,


with semantic HTML tags from the rest,


so that I still be able to use the H2 tags for the footer,


and they will not be assigned to the main content by Google?


So where are you seeing the problem


with the current setup?


When I look at the, like, for every article we have,


it doesn't matter which topic it's about.


We also have the headings of always the same service footer


in the heading hierarchy.


And is it visible in the search results,


or is it just like from a semantic point of view?


It's in the hierarchy.


From a semantic point of view.


OK.


Yeah.


So I thought maybe Google is now confused


if these headings, because these headings always


appear on every page.


Or can they recognize it, and can Google separate it


from the main content?


Yeah, usually that should just work.


In HTML5, there's also an element


that you can use for header, and I think footer.


And within those elements, you can have your headings again,


essentially.


So even from a semantic point of view,


you could set that up properly.


That should be possible.


I don't think it would cause any problems on Google's side


either way.


But do you think, because the service footer


is above the regular footer, could I


put both of these into one footer tag,


or should I?


OK.


Sure.


Yeah.


So from our point of view, what usually


happens with these kind of common elements


is we can recognize them across the site,


because they're more or less repeated across the site.


And then we can try to de-emphasize them


when it comes to search.


Because we realize it's the same text on all of these pages.


We will just pick one of these pages


to rank, essentially, for the text in that footer section.


It's not something that we would say


is critical for the rest of the page.


But it would be cleaner to tidy it up


and to separate it semantically?


I think it would be cleaner.


I don't think you would see any practical change.


OK.


That's always kind of the tricky part.


From a theoretical point of view, you can make it cleaner.


You can make it easier to understand.


But from a practical SEO point of view,


we're so used to these kind of footer elements


in all of the weird ways that they're used,


that's usually not a problem.


So it's more for my backlog, not for my to-do list today.


Yeah, exactly.


Yeah, thanks a lot.


That was it from me.


Cool.


Thanks.


All right.


Neerav.


John, so my first question is, Google Search Console


is throwing an error with this error in required structured


data element.


But when I check the same on validator.schema.org,


it does not show any warnings or any errors.


So first question is, is it the right side


to take the AMP implementation of a web page?


And if so, there is a contradiction.


Like, what should be the step over here?


Yeah.


So these testing tools are for slightly different purposes.


That's probably why you're seeing that difference.


The testing tool in schema.org is more


about understanding schema.org markup in general,


like overall, based on the requirements that schema.org


has.


And the testing tool in Search Console


is focused purely on what we can pull out of the structured data


and use to show in a search feature.


So it's really focused on the search part of that story.


And within search, we only use a small part


of the schema.org markup.


And sometimes we have slightly different requirements.


But maybe we require a specific element


more than the base schema.org markup would require.


And that's often why you see that difference.


And the schema.org validator is for the theoretical markup.


And the Google validator is really


for the practical Google search side of things.


Is there a way we can take what exactly is missing over there?


Basically, it's not an error.


It's a warning on the Search Console.


And when I check the details on the Search Console,


it just feels that you have not done it right.


So will there be a possible way, or should I


should my tech team figure it out?


Yeah.


If it's a warning, then I wouldn't worry about it.


It's basically just saying you could


have done something different.


So at the first hand, I wouldn't worry about it.


What I would do if you want to find out


what exactly the difference is is


double check the documentation on developers.google.com


for search, where we have all of the structured data documented


and all of the required and recommended fields.


And probably one of the recommended or optional fields


is what is triggering this warning.


Sure, sure.


We'll do that.


My second question is, last week on somewhere close to 25,000,


we did one experiment on our website


wherein we improved the page speed of oral website.


We moved from somewhere close to 30,


and we are now at 45,000 according to the page speed


and size score.


And in the next couple of days, we


saw a massive improvement in our ranks.


So I just wonder, can this be like this correlation,


is there a possibility that it is true


or there could be other external factors that can also


be impacting?


So, so quickly, in just two days and we are seeing a jump.


I was just intrigued to ask this.


Yeah, I don't think so.


Yeah, I think that would be due to something else.


Because the speed aspect is something


that we have to pick up through the Chrome User Experience


Report data.


And that takes a while to collect and to be aggregated.


And that's not something that you would see


within a couple of days.


So my guess is something else that you did beforehand,


you also did well.


Right.


Good, thanks, thanks, John.


Cool.


All right, Hazel.


Hey, John.


Hi.


Hey, can you hear me?


Yes.


Yeah, OK.


So I've got two questions.


The first one is about the sitemap.


When Googlebot crawls the sitemap in the server address,


does it firstly crawl the submitted sitemaps


in the JSC backend, or it goes directly to our server address


to crawl all sitemaps?


We don't crawl them all at the same time.


So it's not that there's a specific order to them.


It's more that for individual sitemap files,


we try to figure out how often they change


and how often they give us useful information.


And based on that, for individual files,


we will crawl them at different rates.


So it can happen that one in your Search Console account


is crawled very frequently, but also one that you submit


directly is crawled frequently.


And maybe another in Search Console


is crawled very infrequent.


So it doesn't depend on where you submit it.


It really depends more on the sitemap file itself


and what value it gives to us.


So even though one useless sitemap


we delete it in the JSC backend, the Googlebot still


can crawl it in our server address.


Yeah.


So especially when you delete a sitemap file from Search


Console, we will still have it in our systems.


And we will check it probably for, I don't know,


I'm making a guess, like several months,


we will probably try to recrawl that sitemap file just


to make sure that it's not temporarily gone.


So is there any priority?


Maybe Googlebot will firstly crawl the submitted one


and then maybe crawl the?


No.


No?


It's really about the individual sitemap files


and the value that our systems address to that sitemap file.


And sometimes it's due to how frequently we


see the sitemap file changing.


Sometimes it's just due to other ways


that our systems try to understand


the value of the content that is in the sitemap file.


OK.


I see.


So one more question is about the crawling.


Yeah, like what's the possible reason that maybe when


certain pages didn't get indexed,


even though they were crawled multiple times?


It can happen.


I would assume it's not that frequent,


because usually when we decide to crawl something,


we're also pretty happy to go off and index it.


But it can happen that we crawl a page and then in the end


decide, oh, well, actually we don't need to index it.


One common, I guess, some common situations


where that can happen, which perhaps don't apply


in your case, is if there's an error code on the page,


we have to crawl it first.


Then we see the error code.


If there's a no index on the page,


we also have to crawl it first.


And then we see the no index.


If the page is complete duplicate of something else


that we've already seen, then we crawl it.


We see it's a duplicate, but we focus on the primary page


again.


So those are kind of the normal situations


where we would crawl something and not index it.


But it can also happen that we crawl something,


and then by the time we get to indexing,


we decide, oh, well, actually we want to get something else


from the website instead.


So for this one guess something else,


does it have some example factors?


What other factors that may cause the Googlebot


to decide, oh, we don't want to index it at the end?


I don't know offhand.


I think the overall website quality definitely


plays a role there.


But usually, if we're not convinced about the website


quality, then we would probably not also crawl


the page in the first place.


So that's, I think, kind of a tricky situation.


And if you look in Search Console,


I think pretty much for every site,


you will have the grouping of discovered but not indexed


and also crawled and not indexed.


That's, I think, just pretty common across sites.


So the reason we ask this is we want


to know except page quality and some technical SEO factors


like you just mentioned above, like some knowing


that mental attack, something like.


Other than that, is there any other factors


that orthogram may use to determine


that whether this page is worth of indexing or not?


I don't think we have anything specific otherwise documented.


And I also think it's important to not overfocus


on that specific page.


So if you're sure that from a technical point of view


everything goes OK, I wouldn't assume


that the quality of that specific page is a problem,


but rather kind of the perceived quality


of that part of the website or the whole website itself.


That's kind of the place where I would


try to see what you can do to improve things.


Not just that individual page that didn't get indexed,


but kind of like what is the bigger picture around that page.


OK, so like the overall quality, overall site quality.


Yeah.


Yeah, I got you.


Thanks.


Cool, all right.


Akasi.


Hi, John, hi.


I've got two questions.


The first one is about our website not showing up


for a branded keyword.


So we've got two sites.


One's a global site and one's an Australia site.


And suddenly the Australia side was


ranking for our branded term for our number one position


for two, three months.


And suddenly in the last one week


it was replaced by the global website,


dot com website for a couple of days.


And I just wanted to understand why could that be the case?


It's hard to know without looking at the sites.


But I mean, if these are two sites that are essentially


part of the same bigger group, it


can happen that we switch between which one we would


show for ranking like that.


With hreflang annotations, you can give us


a little bit more information on how you want us


to treat that pair of pages.


So that can help a little bit.


Geotargeting can help a little bit.


But even then, it can happen that we


show a global version of a page in a country


where you actually also have a local version of the page,


perhaps just because the global version is so much stronger


than the local version.


Understood.


And then there's another question about,


so with the same website and the same global version


and the Australia version, we're planning


to take the same content that exists


on a couple of pages on the global version


onto the Australia website.


And we're happy to add in a canonical


to the global website.


So just wanted to know, is that going


to impact the Australia website's ranking in any way


or help it or anything?


I mean, if you're moving the content,


then at least for that content, the site that has the content


would be the one that would rank.


But for the rest of the things there,


I usually wouldn't expect bigger changes there.


For example, if you're moving one product


from one version of a website to another version,


then the ranking for your company name,


that's probably pretty detached from that single product.


On the other hand, if that one product is the primary product


that you're known for, then moving that over,


that could have an effect.


But assuming these are kind of like,


it sounds like kind of like random pages or random products


from one website that you're taking to another one,


I wouldn't expect that to affect your bigger picture


ranking for those websites.


Right.


Yeah, because these are just blog pages.


And we're not moving it from one site to the other side.


Instead, we're duplicating the same content,


but obviously providing a canonical.


So yeah, would that be OK?


Yeah, that's perfect.


I mean, that's a perfect use of the canonical.


If you have the same content on two different websites,


telling us which one you want to have as a canonical


is perfect use.


And you don't have to do it the same way for all


of the content on your site.


So it can be that one website has these blog


pages as canonical.


The other website has different blog pages as canonical.


That's perfectly fine.


Perfect.


Thank you.


Thank you, John.


Cool.


Thanks.


All right.


Anton.


Yes, Seijon.


So we run an e-commerce website.


And we are now in a stage where we


want to make major updates to our category pages.


And one draft wants, or like in one draft,


we want to get rid of the product listings.


So you have the product listings with a faceted search


and where you can filter for the products you are looking for.


And my question is this.


So when we remove the whole product listing of category


pages, would we have a disadvantage in the rankings?


Because first, all the other competitors


have these kinds of product listings.


And second, my guess is this is such an established element


for e-commerce pages that the users,


they expect something like this to have


some kind of overview of all the products


and filters to search for the products they want.


I wouldn't see any problems there from an SEO point of view.


I think there are different things


you would want to watch out for, which probably you will,


so that we can still find all of the individual products


that we have clean links there.


But if you're just redesigning this kind of a category page


and making it look more like an informational page,


I wouldn't expect any problems with that.


I also don't think we do anything


special with those kind of category pages in search anyway.


So from that point of view, you're


just changing the design, essentially.


I think it would be different if it were a product page


and you were to change it completely, because we


do try to recognize product pages and figure out,


oh, where is the price?


Where is the availability?


Those kind of things.


And if you made it look completely differently,


I don't know how.


But if it were, then I could imagine


that affects how we pick up the product pages


and if we can show it in product search results or not.


But the category pages, as far as I know,


we don't do anything special with them.


So if you hide them, essentially,


and make sure that we can still find the links


to their products, that's kind of an extreme case.


You could do that.


But if you want to make them more useful by providing


more information on them, I think that's a good idea.


OK, great, thank you.


And I think one thing I would double check, though,


because you mentioned, would users be confused?


I would double check that stuff.


So from the SEO side of things, I think that's perfectly fine.


But from the user side of things,


it's probably something you'd want to test first.


Yeah, no credit to us.


Thank you very much.


Cool.


OK, maybe I'll just go through some of the submitted questions


and I'll get back to the folks who have their hands raised


as well.


And we have a bit more time afterwards as well.


And it looks like there are not a ton of people here.


So I'm sure we can get through most of the things.


And it looks like I have your questions on top, Anton.


But hopefully that was a useful answer.


All right, next one I have is, if you have structured data


for breadcrumbs set up, is internal linking still


important for SEO?


And yes, absolutely.


It's something where internal linking


is super critical for SEO.


I think it's one of the biggest things


that you can do on a website to guide Google and guide visitors


to the pages that you think are important.


And what you think is important is totally up to you.


You can decide to make things important where


you earn the most money, or you can make things important


where you're the strongest competitor,


or maybe you're the weakest competitor.


With internal linking, you can really


kind of focus things on those directions


and those parts of your site.


And that's not something that you can just


replace with structured data.


So just because there is structured data in a page


somewhere, I wouldn't see that as a replacement


for normal internal linking.


Even if in the structured data you also provide URLs,


we don't use those URLs in the same way


as we would use normal internal links on a page.


So it's definitely not the case that hreflang annotations


replace kind of links between country versions


or breadcrumb annotations replace links


between different levels of a website.


You should really have normal HTML


links between the different parts of your website.


And ideally, you should not just have


kind of like a basic set of links,


but rather you should look at it in a strategic way


and think about what do you care about the most.


And how can you highlight that with your internal link?


All right, let's see.


Next question is a long one about localization or hreflang.


And it starts off with I have a quick question,


and then it's long text, about localization.


We're localizing our website with a subdirectory structure.


We have a TLD covering the MENA audience.


But due to some reasons, we had to get


another domain for a locale.


How should I approach the localization here?


Should I introduce an alternate tag for the other domain?


As our global domain is currently ranking in that locale,


we want the newer domain to appear there.


So the example structured data that


are the hreflang annotations that you have there


looks kind of OK.


But one of the things I noticed there


is you have x-default set up on both of these sites separately.


And from our point of view, we really


need to just have one x-default.


So one way to look at this is regardless


of if it's on one domain or on multiple domains,


on a per-page basis, all of the hreflang annotations


should essentially match up.


And it should be clear that for any particular page


on this website, if we have a requirement for what


we would like to show users, the hreflang annotations guides us


to one specific location.


So for example, if someone is, let's see,


the languages you have here are Turkish.


In Turkey, if someone comes into your website like that


and we would rank one of these pages in the search results,


it should be clear that from the cluster of hreflang annotations


that you have exactly which page would be the optimal one


to show.


So that's kind of the general guidance there.


And it doesn't matter if it's all on one domain


or across multiple domains.


It should just be one clear place per country and language.


The other way around is a little bit more flexible


in that you can have multiple countries and languages


on one page.


So the mapping from country and language to a page


should be one to one.


But the mapping of per page which countries


are on that page or which language is on that page,


that's up to you.


And hreflang annotations look very simple at first,


but it is very complicated to set up.


So don't worry if it feels like, oh, this is something


that you should just know.


It is something that takes a lot of practice,


and people struggle with these hreflang annotations.


It's not always easy.


Let's see.


For product listing page, can we implement multiple product


schemas on the product listing page?


From our policy point of view, I don't


think you should be doing that.


At least the last time I checked the policy,


these around structured data, because for product structured


data, we really want that to apply


to the primary element of the page.


And if you have multiple products on a page,


it's not that one of them is the primary element of the page.


So from that point of view, you should not


use multiple product structured data elements


on a kind of a category page or something like that.


Is there a best practice for pages with mixed language use?


For example, our international school in Japan


caters to Japanese and non-Japanese families,


but we keep most of the information


on our home page in English.


We add support on the page in Japanese as well.


When we had the pages separate, it


made the community's parents feel separate.


Since our communication in real life is mixed language,


having the home page reflect that felt more natural.


Are we punished in search if a page is


intentionally mixed language?


So I wouldn't necessarily say that a page is punished


in a case like that, but we do try


to understand what the primary language is of a page.


And that helps us to understand for which kinds of queries


we would be able to show this page.


So that, I think, is kind of tricky in a case like this.


We can understand when there are multiple languages,


we can understand when there are multiple languages on a page


too.


It just makes it a lot easier for us


to really be clear that if someone is searching in English,


this is the right page to show it to you.


So I could imagine for something like a home page,


maybe it makes sense to have that mix or a slight mix.


Like if you have one home page as primary English,


then maybe include some elements in Japanese.


If you have another version that's primarily Japanese,


some elements in English is fine.


But it helps us to really understand that for the most


part, this is an English page.


And if someone is searching in English


for a specific kind of international school in Japan,


then it makes sense for us to say, well,


here's an English piece of content


that we know fits your needs and that matches the queries


that you gave us.


So from that point of view, I wouldn't necessarily


say that the page is punished, but it


makes it a lot harder for our systems


to figure out how to rank that page properly.


One of the things you can think about here,


though, is look in Search Console


what queries are going to your website or for your home page,


and think about which of these queries


might be affected if Google didn't understand the language


properly.


And it could well be that if most people are searching


for your name, for the brand of your school,


essentially, then probably that would not be affected at all.


On the other hand, if most people


are searching for broader queries, more generic queries,


kind of, I don't know, almost like a sentence that


would match something on your home page,


then I could imagine that would be a little bit harder for you


to peer in search results for just because we're not


sure if your home page is actually


in that language of that query.


So that's kind of my approach there.


One thing you could also do, if you're still


kind of looking at options here, is


to make your home page kind of this bilingual version


that you're talking about there, but to create separate pages


additionally for the individual language.


So that if someone is looking for long-form information


about an international school like this,


they can still find those kind of pure English,


or mostly English pages.


And then from there, transition to the rest of your website.


So lots of different options to think about.


Yeah.


I think it might be tempting to try to do something sneaky


and maybe hide one version of the page


and say, well, users can see it, but search engines can't.


For the most part, I think you run into more technical issues


when you do something like that when you actually


get any value out of it.


So that's one thing I would avoid doing.


If there's a difference between content


in the mobile and desktop version,


does it mean that Google will punish the website


and affect the website's ranking?


Or does it simply mean that Googlebot can find it


in the mobile version, but it won't be able to rank?


So for the most part, we shifted most of our indexing


to mobile-first indexing, which means


we would only look at the mobile version of a website.


In a case like that.


So essentially, if there is something


that is slightly different on the desktop version


of a website, we would, for the most part,


not even use that for search.


And so it's not that we would punish a website because


of a difference, but rather it's like we just


look at one version of the website,


and we don't even know what is on the other version


to treat it differently.


And for the handful of sites that


are still in desktop indexing, that


applies the other way around.


Of course, if there's something on the desktop version that's


not on the mobile version, or wait,


something on the mobile version that's not on the desktop


version, and you're being indexed by the desktop crawler,


then we wouldn't really see that.


We do crawl the alternate version from time to time,


but we don't crawl it to pick up more information,


but rather just to confirm that there


is this connection between the desktop URL and the mobile URL.


In my case, I have a hotel that shows microformats


in a different language than the organic results.


How can we make them show in the same language?


It's hard to say without an example here.


So if you're here in the Hangout, doesn't look like it,


it would be really useful to see what exactly you're seeing.


It might be that the structured data on your page


is actually in that location.


The structured data on your page is actually in that language,


and that's where we're picking it up from.


I don't know where we would otherwise pick up things


that we would show in the rich results, but it would be useful


to take a look at an example like this.


My niche is about poetry, and for years,


leading poems topic in the search results.


But in recent years, I've found I compete with larger media


sites, which is main websites that have nothing


to do with poetry, like top 10 poetry keywords won


by e-commerce sites, news websites.


At the end of the day, specific niche website like mine


will be gone, although I write about poetry for seven years.


Websites that are actually my rival for years,


which is the same in my niche, one by one they disappear.


I don't know what's happening here.


Only large media sites are the winner,


although if it's an article from an e-commerce site,


I also see those e-commerce sites sometimes


win on a health topic.


So it's not only about e-commerce.


I honestly don't know exactly what kind of a website you have.


And one of the things that I think is always a bit tricky


is if your content is essentially commodity content,


which sometimes is with some of these kind of poetry sites


where you're publishing poems written by well-known authors,


for example.


And essentially, the content of your website


is something that one to one anyone else could have,


because it's in public domain or whatever,


then I think that's always kind of a tricky situation to be in.


And it can be the case that for years,


there are a few number of people who


are doing this kind of content.


But due to the nature of the content, where essentially


anyone can take this poem and publish it on a website


and maybe say, I have a t-shirt with this poem,


so I have to put the poem in my description as well,


these are things where it's almost


like a little bit on unstable ground, where


you can build something up.


And maybe for a while, you'll be the only one or one


or few people who kind of creates a website like that.


But in the long run, it's something


where you need to make sure that you're providing something


of significant value so that if someone else, especially


a large player, comes into the area


and also starts including poems in their content,


then that you can kind of compete with them


to some extent.


And this is the case with, I think, a lot of content


on the web that is almost like this commodity content.


And that could be things like poems.


It's like well-known sayings.


All of these things where essentially the information


is available and anyone can use it.


And usually what I recommend when I see websites like this


and they come to me with general questions


is I recommend that they take the time where they're actually


doing well in the search results and use that to build up


and make something much larger that essentially provides


something significant that other people can't just copy.


Obviously, looking back, it's a lot easier to think, well,


I should have done that.


But it's still something maybe you still


have a chance to kind of go out in that direction.


But again, it's really hard to say without looking


at your website.


If you're essentially publishing your own poems


and you're the only person publishing your own poems,


then obviously that's a little bit different.


But if you're publishing well-known poems


by existing authors and anyone can publish those poems,


then it's a tricky situation to be in.


And sometimes it's worth just taking that and saying, well,


I did really good for a while because I was one of the few


people to do that.


And now I have to find something different to focus on


because I don't want to compete with these larger


sites, for example.


We have a food blog website.


Our recipe posts are long form and straight to the point.


I don't know, long form and straight to the point.


And recipes, that sounds like a weird mixture, but fine.


At the end of the post, we put an internal link


for a separate 20 FAQ posts about that recipe.


And let's see.


We include pro tips in the recipe itself,


such that even if readers don't click the FAQ link,


they can still make the recipe.


The FAQ post is like an additional reading source


for them.


My question is, is our content strategy


OK from an SEO point of view?


With separate FAQ posts for each recipe,


we are providing the best user experience


by giving them additional information about how


to make the recipe better, even if they


have to click out of the original recipe


to get that information.


I think that's a perfectly fine strategy.


So from Google's point of view, this


is definitely not problematic.


It's something where, especially with recipe content,


I get a lot of complaints from people


that these are long recipes, and I don't have time


to look at the life story and go through


all of these other items on this page.


I really just want the recipe.


So if you can separate that out and focus on the recipe itself


and maybe separate out some of the more,


I don't know, the questions around that recipe,


from a user point of view, that sounds like a great thing.


It is something where I think the balance might


be tricky with regards to how strong you


make these individual pages.


In particular, if you're splitting things up


on your website, maybe with regards to internal linking


as well, that suddenly every recipe ends up being two pages,


then of course we have to index two pages per recipe,


and we have to give value to two pages per recipe, which


means you're potentially diluting things a little bit.


On the other hand, if you set it up in a way


that maybe these FAQ pages are only linked from the recipe


and the primary content on your website is the recipe itself,


then I would expect that we'd be able to focus on the recipe.


And if someone searches for something


that's within the FAQ part of that recipe,


then we would be able to show the FAQ page in the search


results.


From a structured data point of view,


from a rich results point of view,


people sometimes like to put FAQ markup on all pages


because it adds extra room in the search results.


I think that's almost on the way out from our point of view


in that we've seen that people do this,


and we do try to reduce the amount of FAQ entries


that we show in the search results


just because it's not that useful for users


where every result has this set of FAQs on it.


So moving that out to a separate page, from my point of view,


makes sense there too, in that you're not


kind of overburdening the recipe results with all


of this extra structured data.


You're really kind of like, if people want more information,


we have that information.


If people just want to focus on the recipe,


we have that information as well.


Now, curious to see what kind of feedback we get about recipes.


It's always one of those tricky topics.


I have an e-commerce site with more than 4,500 URLs.


I already submitted the proper site for that site.


I know that Google doesn't index everything,


but Google is only indexing 86 URLs, essentially.


So what can I do there?


I think some of the things we've talked about before with


regards to crawled and not indexed,


they would apply here too.


So the technical things, I would double check.


But also, I would really give the overall quality


of the website a review and ideally do that also


with people who are not directly involved with your website.


So that's something where I think


we have a blog post a while back about questions


that you can ask yourself about your website.


And I would try to find some people who are not


associated with your website, give them


some tasks to do on your website,


and go through those questions with them.


And try to get as objective feedback as possible


to see what you can really do to significantly improve


the quality of your website overall.


And I realized with an e-commerce site,


it's easy to just take a catalog of products


and dump them on a website and then submit them.


And suddenly, you have 10,000 pages.


But just because you can make a website with 10,000 pages


or 4,500 here doesn't mean that you're


providing something that is unique and valuable


for the internet that we must have indexed


and that we must show to people immediately.


It's something where the value has to fit as well.


OK, let's see.


Maybe we'll switch back to some more live questions.


And I also have a bit more time at the end.


Should we run out of time?


Let's see.


Luis, I think you're up next.


Yes, that's correct.


Hi, John.


Hi, everybody.


I have a question about the sitemaps.


We have a really huge page with millions of URLs.


And right now, we are currently, as the sitemaps,


are being currently renovated.


And our IT team is considering storing the new files,


as the new sitemap files, in our cloud service.


That means from example.com slash sitemaps


to cloud.com sitemaps.


And we are wondering, is this that a problem?


Or if we store the sitemaps in the cloud,


and if that's not a problem, should we


also create a permanent ready deck for the old URL


for this example.com sitemap?


Or how should we plan the move?


Yeah, so it's definitely possible to host the sitemap


file somewhere else.


There are two ways that you can do that.


One is if you have both of those domains verified in Search


Console, then that works.


The other way is if you submit it with the robots.txt file,


where you specify sitemap colon and then the URL of the sitemap,


that can also go to a different domain.


So that's something where if you have a separate server where


you're creating sitemap files, or if you have kind of like


a staging setup where it crawls and checks the files


and then creates a sitemap file somewhere else,


that would all work.


I would also redirect the old sitemap file


to the new location just to be clean.


But probably even if you just delete the old sitemap URL


and make sure to submit the new one properly,


then that should just work.


What might be a little bit tricky


is I don't know how Search Console would


show that directly in the UI.


In particular, if the sitemap file


is in a different location, if Search Console


would show the sitemap information in the indexing


report, for example.


But that's a reporting problem.


That's not something that kind of relies


on the functionality of the sitemap file.


It's really just Search Console doesn't show it properly.


And again, maybe it does.


I'm just not 100% sure.


SPEAKER 2 OK, but we can survive with this problem.


But with the main problem, that means it's not a problem.


If we have the sitemap also not in the main domain,


it's totally fine.


JOHN MUELLER That's perfectly fine.


In the beginning, when sitemaps first came out,


there were some services that would crawl your website


and create a sitemap file for you.


And they did exactly that.


And that's perfectly valid.


SPEAKER 2 OK, great.


Thank you.


JOHN MUELLER Cool.


All right.


And next up, I don't know how to pronounce your name.


So sorry.


SPEAKER 2 It's Gosia.


JOHN MUELLER Gosia, OK.


SPEAKER 2 All right.


Hi, so I have a follow-up question on the answer


that you gave us actually during the previous meeting.


So we posted a question about domain with a history


as an escort service provider.


So you answered that what we need is patience and time.


But I wanted to ask some follow-up questions.


So basically, the domain has a long history


because the first snapshot of that website is from 1997.


So it is a really long history where actually we relaunched


our website in June last year.


So it's very new and fresh.


And the main issue that we are having


is like, as in the message we stated,


that we still are getting flagged.


And additionally, we have this issue


that you already mentioned here today,


like, role currently not indexed.


And we are trying to understand if the history of the domain


can actually affect that we are having issues with indexing.


And answering the questions from your blog,


we do believe that the content we are publishing


is a good quality, is internally linked.


And we do build, like, we try to build a quality site.


Where we struggle at the moment is paid performance.


So it's kind of in progress at the moment


to optimize for that.


But we use prerender.io.


So kind of what we show for Google


is already prerendered version.


So when it comes to our Lighthouse score, it's all good.


So if you could give us maybe some advice,


like, what can we improve or look for to understand,


like, why are we not getting indexed?


I'm happy to share the URL as well.


Yeah, if you can drop the URL into the chat,


I can take a look at that and pass that on to the team,


probably.


If you have a bit of time afterwards,


after we have gone through the questions,


I can double check maybe some first things as well.


So kind of like to see if there is something super obvious


that I can help with.


All right, that would be great.


Yeah, and usually the indexing side of things


would not be related to kind of like if there was


adult content on the website before.


The indexing side might be affected


if the content that was on there before was very spammy.