We don't use direct traffic.


English Google SEO office-hours from June 18, 2021

We don't really have a measure for that anyway.


So that's something that we wouldn't use.


Can we safely delete the HTTP version


of a site in Search Console?


The site has been HTTPS for a couple of years now.


And all the requests are forced to HTTPS.


Yes, you can remove the HTTP version in Search Console.


I don't think it would change anything positive or negatively


for the website.


And if you have the domain verified,


which is kind of my recommended approach,


then you can always add the HTTP version


should you ever need to do something


specific with the HTTP version.


So for the most part, it doesn't harm


to keep the HTTP version in Search Console.


But if you want to make room, then of course,


feel free to remove it, especially


if you don't need it anymore.


Our English articles are ranking for featured snippets


for some English queries only in some countries.


We notice it's only English-speaking countries


like US, UK, Australia, and Canada that


were not ranking for snippets.


Could we be marked as not good English by Google?


I don't think so.


So featured snippets are essentially a way


that we highlight things in the search results


with a little bit of a bigger snippet.


And they're a part of our normal ranking systems.


And we rank slightly differently across the different countries.


We use geotargeting.


We try to figure out which results make sense


to show to users.


And all of that can lead to slightly different results


being shown in the search results.


So that's something where it's not that our systems would


need to go in there and say, oh, this website doesn't


have good English.


Therefore, we shouldn't show it in an English-speaking country.


But it's more that, well, we have better results


that we would show here in other countries.


Therefore, we'll show those instead.


Can the URL structure be different than the breadcrumbs


on a page?


For example, if the URL is just website.com slash blog dash


post as name, but the breadcrumbs show home,


main topic, subtopic, blog post name, would that be an issue?


That's perfectly fine.


So breadcrumbs don't have to match the URL structure.


In some cases, you have a very flat URL structure.


Everything in one file name and no subdirectories.


In some cases, you have a very organized folder-based


structure and a URL structure.


That's perfectly fine as well.


None of these don't have to map to breadcrumbs on a page.


I think where some of the confusion comes from


is that if you don't use breadcrumb markup on a page,


then our systems will try to guess breadcrumbs sometimes.


And they try to guess that based on the URL at some point.


And then it can happen that our breadcrumb guessing systems


will say blog dash post dash name.


That's like a set of breadcrumbs.


We should show those.


But if you specify breadcrumbs with structured data,


then we will use those instead of our guessing metrics.


Do Google My Business clicks and impressions show up


in Search Console as clicks and impressions?


Yes, when the Google My Business listing is shown


in the normal search results.


So if you do a normal web search and you


have a map with your Google My Business listing shown,


then we will show that as clicks and impressions in Search


Console.


However, if you're looking at Google Maps


and you see the Google My Business listing,


then that's something that would be outside of Search, which


we wouldn't show.


Is there an impact to linking twice


to the same page from the menu?


No.


So there's a little bit more detail on the question.


But in short, the answer is no.


If you link to the exact same page twice


in the menu with different anchor text or something


like that, that's perfectly fine.


That's not going to cause any problems.


If you do that for UX reasons, go for it.


It's not going to give you any advantages like linking twice.


But it's totally fine to do.


I see a lot of advice online that


references data saying short URLs perform better


than long URLs.


I've also seen quotes from you saying


that Google doesn't look at the URLs or the slashes in the URL,


so it's just preference.


Do shorter URLs actually make an impact compared to long URLs,


or is that just an SEO myth?


My guess is this is based on things like correlation studies


that are done, where people say, well,


when I look at these search results,


the top so-and-so results tend to have shorter URLs


than the ones way in the bottom.


And from that point of view, it's


easy to say, well, if you want to rank high,


therefore you should have short URLs.


But I think that's misleading.


It just happens to be that some pages, when they tend


to be very, I don't know, concentrated on a website, very


important for a specific website,


they tend to have shorter URLs because they're closer


to the root of the website.


And because of that, it looks like shorter URLs lead


to higher ranking.


But it's actually just, well, these


are kind of like the primary URLs of a website.


And it would be kind of normal for them


to rank a little bit more visibly


than some random product page that is listed,


like, I don't know, 10 steps further down in a website.


So from that point of view, shorter URLs


are not going to make your website rank better.


The one place in our systems where we do use the URL length


is when it comes to canonicalization.


So in particular, when we know that there


are multiple URLs on your website that show exactly


the same content, then one of the kind of factors


that we take into account there is the length of the URL,


where we say, well, if it's a shorter URL,


then usually that looks nicer for users.


So we tend to use that one.


And usually that comes into play when


you have things like URL parameters attached to a URL


and you tell us this version with all of these parameters


is the same as the same version without parameters,


then our systems will say, well, the one without parameters


looks nicer.


We'll just use that one instead.


And canonicalization is not something


that just uses one factor.


We use a lot of different factors there.


So it's not always going to be the short one that we


pick as a canonical.


But it tends to go in that direction.


And when it comes to canonicalization,


it doesn't change rankings.


It's purely which URL is shown in the search results.


The news agency, I think we looked at aggregate ranking


rating.


We looked at outdated news pages.


What's the best practice to apply to old news pages?


By that, I mean news of last year.


Would deleting these be the best option or no indexing?


Or what should be the best way from an SEO point of view?


Generally, I recommend keeping the old content online,


maybe putting it into an archive section if you want.


If you really don't want to keep it anymore, then deleting it,


no indexing it, all of that is perfectly fine.


But for the most part, I don't see a need


to kind of remove old content on a website.


So from that point of view, it's not


that I would say you should always delete old news.


Sometimes it's useful to find.


Why would Google exclude a large amount of URLs


that were already indexed?


We have about 1.8 thousand URLs excluded.


And now within the past month, we have over 10,000 excluded.


We haven't changed anything.


So I think this is kind of tricky


because we don't have any guarantees


that we will index everything from a website.


So it's completely normal that sometimes we


have more indexed and sometimes we have less indexed.


When I look at some of my older test sites


where I have a ton of pages kind of just for test purposes


in Search Console to see how things kind of perform


with the technical reports there,


it's completely normal for me to see things


fluctuating quite a bit.


And sometimes they fluctuate very strongly.


And I haven't changed anything on these sites for years.


But still, the number of indexed pages


goes up like crazy amounts.


The amount of crawling goes up and down all the time.


And that's completely normal.


So that's something where, from my point of view,


the amount that we index from a website, that varies over time.


And that isn't something where we


would say we will try to index everything from all websites


because it's just purely from a technical point of view,


it's impossible to get everything from all websites.


The way I act that, it was my question.


Oh, great.


So we have product schema right now,


but we don't have it filled out.


So we're getting error for that.


Would that be the reason why we are getting de-indexed?


No.


The structured data is purely for understanding


what else is on the page and for showing


that in the rich results.


So it wouldn't be a sign that we would say we will drop


these pages from the index.


OK.


Thank you.


Sure.


Usually, the best approach that I've


seen to getting more content indexed in a consistent way


is to make sure that the quality of the website overall


is as high as possible.


So that's kind of a trick from, I don't know.


I don't know if you'd call it a trick.


But essentially, you're telling Google


that this is really important content,


and then Google will try to actually go off and make sure


that all of that is indexed.


So that's something that kind of comes


into play with the crawl budget discussions.


And we have a blog post, I think, a couple of years


back now with regards to crawl budget.


And that also includes the topic of crawl demand,


which is basically how much Google


wants to crawl from a website.


And crawling is kind of the first step to indexing.


So understanding a little bit how


you can make sure that the quality of your website


overall is as high as possible so that our systems, when they


see pages from your website, that they say,


well, it's really important that we crawl and index


this as quickly as possible.


And it's kind of tricky because it's not


that much of a technical factor where you can say,


well, I will add a meta tag and say,


Google should index this page.


OK, so deleting all of that schema wouldn't help.


Exactly, yeah.


OK.


And if we have a lot of, so it's an e-commerce website,


and we have a lot of industrial products


that are very similar, could that play a factor as well


that Google are saying these pages are very similar?


Just the difference is a part number.


Sometimes.


Usually, if that comes into play,


it would be something that we would show probably also


in the coverage report where we would say a different URL was


chosen as canonical.


So it's not so much that we wouldn't index those pages,


but rather we would see those pages and say,


oh, this is very similar to the other page


that I already know about.


I will take that one as a canonical.


Thank you.


Sure.


What could be the best average period


to measure the impact of our website changes?


For example, if I disintegrate the AMP feature


from a news website, note a traffic drop in the next hour.


Could that be due to removing AMP?


OK.


I think this is kind of tricky because on the one hand,


it sounds like, what if I make normal changes on my website?


How long does that take?


Where that's something where I'd say, well, it depends.


And it's something where you tend


to see changes within a couple of days


and then the majority of the changes


maybe within the next month and then some remaining changes


over, I don't know, half a year or longer even.


But specifically about AMP and non-AMP,


one of the things specifically with AMP on news pages


is that this tends to be very quick in reaction.


So it's not something where it's a normal technical change,


where if you change all of the titles on your page


to something different, where it takes a while


to be reprocessed, with AMP, it's


something that we try to pick up as quickly as possible.


And if, for example, the URLs are no longer in the AMP cache,


then we would have to kind of react fairly quickly


and swap all of that out.


In practice, I would expect just removing AMP or disabling AMP


is something where you would tend to see within,


I don't know, a couple of days, you


tend to see those changes fairly quickly.


And you would specifically see that the AMP pages are no


longer being shown in search, but rather we


show the normal pages again.


And it's not so much that you would see a ranking


drop during that time, but rather a swap in the URLs


that are shown.


And depending on how you're tracking things,


that could look like a ranking drop.


So that's kind of the effect there.


I don't think you would see an effect within an hour


when removing AMP.


It feels like something where it's more


like it's a fast change, but it's


something that would come within a couple of days.


The one effect with regards to AMP that


might be visible with regards to ranking


is that the top stories carousel, where by default,


the AMP pages would be eligible to be shown there.


But non-AMP pages kind of previously


were not eligible to be shown there.


And I think since this week, since we started rolling out


the page experience update, non-AMP pages are also


eligible to be shown there, provided they're, I think,


fast enough or good enough with regards to the page experience


ranking factors.


So a lot of these small details kind of


come into this question.


I would not expect an effect within one hour,


but you could probably see some effects when you remove AMP,


specifically for a news website, over the course of,


I don't know, a couple of days until things settle down


a little bit again.


Also, usually with a news website,


the website tends to rank more for the new content.


So it's something where, for the most part,


big changes that you make will be reflected in the new pages


that you have.


It's a little bit different with a traditional website


where traffic might be going to all parts of the website,


where if you make a change, all of those parts


have to be updated over time.


I have one website with your brand, DE, in German,


with language D-E-D-E, and one with.com in English,


with E-N-G-B or U-S. One is basically


a clone of the other, but properly translated.


Are there any disadvantages not using rel alternate hreflang?


Are there any advantages to applying it?


So essentially, the hreflang links between those two pages


will not change the rankings of those pages.


So that's kind of the first step.


However, they can swap out the URLs that are being shown.


So it's hard to say whether there


is an advantage or disadvantage from using it.


I think there's definitely no disadvantage to using it.


But essentially, the advantage is


if we're showing the wrong language URL in the search


results, then with hreflang, we can swap that out.


So usually, you would see this if someone


has a brand that is the same across multiple languages.


If users are just searching for that brand name,


then from the query alone, we don't


know which language content to show,


which language content to rank for that page.


And it can happen that we would rank the wrong version


for that.


So I don't know.


For example, if you search for Google in Germany,


we would not necessarily know that someone


is looking for the German version of Google.


So it could happen that the dot-com version


is shown to users.


And then with hreflang, we would swap out


the URL against the German version


because we know, oh, this user is in Germany


and has their settings set to German.


So we should use the German version for that page.


So it's not so much that it will rank better,


but rather that the corrector version will rank.


On the other hand, if users are always


searching in a way that is clearly localized, where


if you have, say, different brands


across different countries, and someone in one country


searches for your brand, and there's only


that one page which is really relevant for that result,


then probably hreflang will not change anything.


So that's something where I would double-check first,


are users seeing the wrong versions?


And if so, then use hreflang.


If not, then probably you don't need to use that.


SPEAKER 2


John, regarding this hreflang, can I just have a follow-up?


JOHN MUELLER 1 Sure.


OK, so this is also very confusing for us,


especially for branded queries.


So John, for instance, if I have English page


and I have folder for German or the folder for Germany,


in Germany, if a person is searching,


obviously brand name would remain in English,


or a person can just type the brand name.


But his browser language is in German.


So in that case, using hreflang means


Google will show German version on branded query, right?


And if his browser language is setting is in English,


then English page will appear.


Because query is the same.


Does it mean that Google will just


try to see only browser language setting, and just accordingly


then serve the right page?


It's not quite that simple.


Yeah.


So


SPEAKER 2 This is why I just said that it is always


confusing to us.


Yeah, I mean, because there are some situations where you,


so on the one hand, if you're signed in,


you can specify which language you


would like to have for Google.


So it might be that your browser is set to English,


but the language that you prefer is German or French.


And we would use that in a case like that.


The other thing is that, especially when


we're talking about English, a lot of times


the default language on a computer is English,


and these computers are distributed worldwide.


And if we can recognize that this is essentially


a default setting, then we might say,


well, we can't trust the setting to be accurate.


So we will use whatever is locally relevant.


So this is particularly, I think,


an issue in English, where if we see that a browser is set


to US English and everything looks default,


then we might say, well, probably the user is


in Germany, so probably they want German content,


even though their browser is set to English,


because we assume that maybe they didn't set the browser


language at all, and it's just the default setting.


They don't know how to change that.


So purely based on the browser language


is usually not enough.


So John, I just wanted to understand only


for branded queries, because brand remains the same.


And technically, it happens that if we


are expanding to different language, then


the management, the company, and even the SEO


want for branded query in the same language,


the person should lend to respective language


page rather than home page or English page.


Yeah.


I think, in general, with regards to internationalization,


you should not rely on Google always getting this right,


and instead have some kind of a fallback mechanism


on your pages where you're saying,


you landed on the German version of our home page,


but it looks like you have your browser set to French,


so here's also the French version,


and make it easy for people to get to the appropriate version


of your site.


So always have that backup set up there.


And it particularly makes sense when


you're across different countries, where people might


also just enter the URL and say, well, I entered the.com


because it's the main site, but I'm located in India,


so I should be getting the Indian version of the site.


And having a banner on top saying,


hey, do you want to go to the Indian version of our site?


Click here.


That makes it easy for people to get there.


And at the same time, with the banner,


you're making it so that when Google crawls that page,


that Googlebot doesn't automatically


get redirected to any particular version.


SPEAKER 2


We have been using this.


Cool.


Thank you.


Sure.


Let me just grab one more question from the list,


really short one.


What's the best practice for rel canonical on a URL


that is now returning 404?


You can do whatever you want on a 404 page.


If the result code is 404 or 410 or any error,


then we will ignore all content on the page.


What you put in the robots meta tags and the rel canonical


is totally irrelevant for us because we already


recognize that there's nothing there that we need to.


So that was kind of an easy one.


And so let me get back to all of these millions of people


who have raised hands.


And we'll get, I don't know, one or two more in,


and then pause the recording, and then kind of continue


on afterwards as well.


Julian.


Hey, John.


How's it going?


Hi.


So my question is about best practices


for a website that's a database of products that monetizes


via affiliate links.


So the site is UGC and community driven,


and readers can contribute unique content, reviews,


and other cool stuff.


So just up front, I know how skeptical Google


is of thin affiliate sites that offer no unique value.


And I know you've talked about it a lot on here,


and I totally get that.


So on this particular site, let's say


you've got 100,000 products or so, and 20%, let's say,


have accrued a lot of great unique content,


and they're helpful.


And after that, it's kind of a sliding scale


where there's some that are medium that


are getting to be useful.


And then on the other end, you have


some that would probably be considered pretty thin.


So my goal is not to have Google judge


the overall picture of the website as, hey,


this is a thin affiliate.


So I feel like, tactically, I have a chicken and egg problem


because you need starter pages created


in the first place that may be viewed as thin, I think,


from a Google lens.


But you kind of need them so they can accrue


unique content over time.


Some products just aren't going to be as popular, et cetera.


So my inclination is just make use of noindex,


which sounds kind of like the simplest answer,


for pages that haven't hit a certain quality threshold.


And I know you've gone on record to say that Google tries


to look at pages regularly.


So if there's 1,000 pages, one is great, 999 are crap.


It behooves Google to show that one great page.


But you judge trust based on larger sections of the site.


So you could look at all product pages,


how trustworthy is the site?


So I guess my two kind of pointed questions


are, is it enough to noindex the ones that haven't


hit the quality threshold?


Or should I go as far as to remove affiliate links


from the not so great pages?


So broadly, just how do I get in Google's good graces


for this type of a website, which


I know is kind of a hot button issue a little bit?


Yeah.


So noindex would be perfectly fine.


I don't think you would see any effect if you just


removed the affiliate link there and left those pages.


Because we would just judge the pages on their own


and say, well, there's not much content here.


And the matter of the affiliate link


is almost like a side note there.


So I think the approach with noindex makes a lot of sense.


Figuring out how to recognize what pages are quality or not,


that's sometimes a bit tricky.


But it sounds like you have some ideas.


So that's kind of the direction I would head there,


to really say, like, I mean, one of the kind of mixed approaches


you could also take is to say, well, if we have new products,


then we will let those be indexed for a while


and see if after a period of a month or so,


they kind of reach that quality threshold, then fine.


And otherwise, we'll take them out again


to kind of make it possible for new things


to also be visible from time to time.


But generally, especially for all of the content


that you collect over the years, if you're saying, well,


someone submitted this five years ago.


Nobody has commented on it since then.


Maybe we should just noindex this.


And you could also go so far as to say at some point,


well, I have noindexed it for a year.


And nobody has manually gone there either.


So maybe we don't need it at all.


But that's something where Google wouldn't really


care what you would do after you've noindexed it.


So noindex really tells us, don't


take this to account in our index.


And we won't take it into account with our quality


systems either.


OK, that makes sense.


Thank you.


Sure.


Let's see, Nathan.


I was wondering about site vitality in Search Console


as we have a website that just doesn't get a lot of traffic.


So there's no data there.


Would that mean that the core web vitals couldn't even


be taken into approach as to taking it


to effect for SEO purposes?


If we don't have data for a site at the moment,


we wouldn't be able to use that for the ranking factor.


But we have a lot of practice with dealing with sites


where we don't have the full data set.


So it's something where I wouldn't necessarily


worry about it.


And I wouldn't see it as something like, oh, I


have to build more traffic to my website


in order for it to be able to rank.


But rather, if your website just doesn't have a lot of traffic,


then that's perfectly fine.


A lot of websites don't have a lot of traffic.


I know.


Yeah.


All right.


OK, thanks.


Let me pause the recording here.


And like I mentioned, you all are


welcome to hang around a little bit longer afterwards as well.


It looks like there's still tons of questions left.


So we'll see how far we can make it before I fall over.


Thank you all for watching this recording


if you're watching it on YouTube.


And feel free to jump on in in one of the future episodes


if you'd like to join us live.


Do URL structure changes affect SEO? #AskGooglebot

Today, a question from YouTube is about tweaking a website's URL structure.


Unfortunately, while this may at first sound like a small change within a website,


it's not that simple for search engines.


In particular, search engines like Google store their index on a per-page basis.


So, if you change the address or the URL of a page,


that page's data has to be forwarded somehow, otherwise it gets lost.


It doesn't matter if you're completely rebuilding a website


or if you're just removing a slash from the end of URLs,


these are all essentially site moves.


We have extensive documentation that I'll link to in the description.


In particular, we recommend first, research the options and the potential effects.


Since these changes take time and have ranking effects,


it's also recommended to consider the timing of when you make the move.


Second, create a list of the old and the new URLs.


This will help you to track and check the changes afterwards.


Then, start the migration.


301 redirect all the old URLs to the new ones.


Also, update all internal mentions, such as links, forms, structured data,


sitemaps, and the robots.txt file.


Finally, monitor their migration.


Check all pages for the redirect.


In Google Search Console's report, you should see a quick change


for the most important pages and then a slower change


as our systems reprocess the rest.


Overall, this can take several months to complete.


Redirects should remain in place for at least one year.


Whew, that's a lot, right?


Well, these changes are not always easy.


However, with good research, documentation, and preparation,


I'm confident that you can make them work.


Good luck and see you next time.


Clarence manages paper analysis with coveredized data.


It includes the idea of Dress fine Society research.


I like lace organization.


And the reason it's super easy is to detail


what happened to these Assteams who use race data


and where different distributions have been reached.


So, excuse me.


So, this will be a quick 208-minute video,


then we'll go songs and language Within


which we have all the posted products opportunity,


the ear scandals, and statements.


Now I'll be taking a short Greens on


our Social Labour web reserve and launching


JavaScript SEO Q&A September 1st, 2021

Hello, and welcome to another JavaScript SEO Q&A.


JavaScript SEO Q&A September 1st, 2021

Today is the 1st of September.


JavaScript SEO Q&A September 1st, 2021

Ooh, holy shit.


JavaScript SEO Q&A September 1st, 2021

We've done like eight months of the year already.


JavaScript SEO Q&A September 1st, 2021

That's bananas.


JavaScript SEO Q&A September 1st, 2021

Usually, the format goes like this.


JavaScript SEO Q&A September 1st, 2021

You can post questions on the YouTube thread.


JavaScript SEO Q&A September 1st, 2021

I'll answer the YouTube questions,


JavaScript SEO Q&A September 1st, 2021

and then everyone who joins the live recording


JavaScript SEO Q&A September 1st, 2021

gets to ask me a question live as well,


JavaScript SEO Q&A September 1st, 2021

or gets to ask a follow-up question


JavaScript SEO Q&A September 1st, 2021

if they posted a question.


JavaScript SEO Q&A September 1st, 2021

And then these recordings go up on our YouTube channel as well.


JavaScript SEO Q&A September 1st, 2021

And in two weeks, there will be another opportunity


JavaScript SEO Q&A September 1st, 2021

to ask us questions.


JavaScript SEO Q&A September 1st, 2021

So let's go through the YouTube questions real quick.


JavaScript SEO Q&A September 1st, 2021

Guru4aDay, interesting name, asks me,


JavaScript SEO Q&A September 1st, 2021

they have a challenge with a single-page application


JavaScript SEO Q&A September 1st, 2021

where they have a solution for users to find


JavaScript SEO Q&A September 1st, 2021

all service answers on one page, but this page is not indexed.


JavaScript SEO Q&A September 1st, 2021

They tried to use canonicals on the customer side


JavaScript SEO Q&A September 1st, 2021

and are in the process to implement page ref.


JavaScript SEO Q&A September 1st, 2021

Hope you can share some pointers to a non-technical person.


JavaScript SEO Q&A September 1st, 2021

Not really a JavaScript question, but in general, I


JavaScript SEO Q&A September 1st, 2021

would say use Google Search Console to see what's going on,


JavaScript SEO Q&A September 1st, 2021

why it's not being indexed.


JavaScript SEO Q&A September 1st, 2021

It is possible, especially when a single-page application


JavaScript SEO Q&A September 1st, 2021

says that there's something preventing us


JavaScript SEO Q&A September 1st, 2021

from rendering the page properly.


JavaScript SEO Q&A September 1st, 2021

And in that case, we wouldn't see content.


JavaScript SEO Q&A September 1st, 2021

And if there is no content, we might think this is an error


JavaScript SEO Q&A September 1st, 2021

page, or we might think it's just not


JavaScript SEO Q&A September 1st, 2021

very good in terms of content, because there is none.


JavaScript SEO Q&A September 1st, 2021

And then that's why it might not be indexed.


JavaScript SEO Q&A September 1st, 2021

The thing that you say, we are in the process


JavaScript SEO Q&A September 1st, 2021

of implementing href.


JavaScript SEO Q&A September 1st, 2021

I'm not so sure what you mean by that.


JavaScript SEO Q&A September 1st, 2021

If it's href lang, then that doesn't really


JavaScript SEO Q&A September 1st, 2021

have bearings on how pages are being indexed.


JavaScript SEO Q&A September 1st, 2021

If what you mean is that your links do not have hrefs,


JavaScript SEO Q&A September 1st, 2021

then that's probably why this page is indexed,


JavaScript SEO Q&A September 1st, 2021

because we probably haven't even discovered it.


JavaScript SEO Q&A September 1st, 2021

But I would definitely use Search Console


JavaScript SEO Q&A September 1st, 2021

and the Live Test, the UI inspection testing tool,


JavaScript SEO Q&A September 1st, 2021

to try to see what might prevent us from indexing the page,


JavaScript SEO Q&A September 1st, 2021

because it could be pretty much anything at this point.


JavaScript SEO Q&A September 1st, 2021

Very, very vague to say what the problem here might be.


JavaScript SEO Q&A September 1st, 2021

We have another question that's not really a JavaScript


JavaScript SEO Q&A September 1st, 2021

question, which is from Ankur.


JavaScript SEO Q&A September 1st, 2021

He's asking, what would be the cause for block by robots.txt


JavaScript SEO Q&A September 1st, 2021

in the coverage report in Search Console?


JavaScript SEO Q&A September 1st, 2021

Even though those pages are noindex,


JavaScript SEO Q&A September 1st, 2021

Googlebot still continues to crawl them at scale.


JavaScript SEO Q&A September 1st, 2021

So again, a little unclear what's


JavaScript SEO Q&A September 1st, 2021

happening, what you're seeing.


JavaScript SEO Q&A September 1st, 2021

But a noindex on the page does not


JavaScript SEO Q&A September 1st, 2021

mean that we are not crawling it.


JavaScript SEO Q&A September 1st, 2021

It just means that we are crawling it and not


JavaScript SEO Q&A September 1st, 2021

putting it in the index, so it wouldn't show up


JavaScript SEO Q&A September 1st, 2021

in search results.


JavaScript SEO Q&A September 1st, 2021

Then again, if you block it by robots.txt,


JavaScript SEO Q&A September 1st, 2021

we wouldn't really crawl it, in which case,


JavaScript SEO Q&A September 1st, 2021

if you are seeing Googlebot crawling it,


JavaScript SEO Q&A September 1st, 2021

it's probably fake Googlebot.


JavaScript SEO Q&A September 1st, 2021

Because if you tell us, please don't patch this URL,


JavaScript SEO Q&A September 1st, 2021

then we wouldn't actually do that, generally speaking.


JavaScript SEO Q&A September 1st, 2021

If you are wondering why the page still shows up


JavaScript SEO Q&A September 1st, 2021

in the index, then that's because we are not


JavaScript SEO Q&A September 1st, 2021

seeing the noindex when we can't crawl it.


JavaScript SEO Q&A September 1st, 2021

So we might put it in the index, because something


JavaScript SEO Q&A September 1st, 2021

links to this page, and we figured out,


JavaScript SEO Q&A September 1st, 2021

oh, there is a page about something,


JavaScript SEO Q&A September 1st, 2021

but it would probably not rank very high anyway,


JavaScript SEO Q&A September 1st, 2021

because we don't really have information


JavaScript SEO Q&A September 1st, 2021

if we can't crawl it.


JavaScript SEO Q&A September 1st, 2021

Then there's a question from Charles.


JavaScript SEO Q&A September 1st, 2021

How will Google deal with and interpret web pages


JavaScript SEO Q&A September 1st, 2021

using window history push state to change the display


JavaScript SEO Q&A September 1st, 2021

URL on the browser?


JavaScript SEO Q&A September 1st, 2021

Will Google associate the page with a URL modified


JavaScript SEO Q&A September 1st, 2021

or with a URL before the modification to push state?


JavaScript SEO Q&A September 1st, 2021

That's a good question.


JavaScript SEO Q&A September 1st, 2021

I haven't really tried that.


JavaScript SEO Q&A September 1st, 2021

I think I tried it a long time ago.


JavaScript SEO Q&A September 1st, 2021

And when I tried, it's basically treated


JavaScript SEO Q&A September 1st, 2021

as if it's a redirect kind of.


JavaScript SEO Q&A September 1st, 2021

So it's being associated with the URL after the modification.


JavaScript SEO Q&A September 1st, 2021

But I will have to verify that.


JavaScript SEO Q&A September 1st, 2021

I'll let you know.


JavaScript SEO Q&A September 1st, 2021

I think I have you on Twitter as well.


JavaScript SEO Q&A September 1st, 2021

Or I'll reply to the thread on YouTube.


JavaScript SEO Q&A September 1st, 2021

But I think my hypothesis is that we are treating it


JavaScript SEO Q&A September 1st, 2021

as if it is a redirect and will associate


JavaScript SEO Q&A September 1st, 2021

the information with the displayed URL after history


JavaScript SEO Q&A September 1st, 2021

push state.


JavaScript SEO Q&A September 1st, 2021

Then there's a question from Sanjay.


JavaScript SEO Q&A September 1st, 2021

In December 2020, we revamped and migrated our site


JavaScript SEO Q&A September 1st, 2021

to Node.js.


JavaScript SEO Q&A September 1st, 2021

Since then, Google Search Console


JavaScript SEO Q&A September 1st, 2021

is reporting mobile usability issues on many URLs


JavaScript SEO Q&A September 1st, 2021

of all page types on our site.


JavaScript SEO Q&A September 1st, 2021

But these issues are not permanent.


JavaScript SEO Q&A September 1st, 2021

And sometimes they are reporting pages


JavaScript SEO Q&A September 1st, 2021

as having issues sometimes as valid pages.


JavaScript SEO Q&A September 1st, 2021

And they noticed that if they test the source code instead


JavaScript SEO Q&A September 1st, 2021

of the URL, they get these problems.


JavaScript SEO Q&A September 1st, 2021

And then they figured out that the viewport tag, if they add


JavaScript SEO Q&A September 1st, 2021

a minimum scare parameter, then it seems to be working.


JavaScript SEO Q&A September 1st, 2021

But some URLs are still failing.


JavaScript SEO Q&A September 1st, 2021

I would say, generally speaking, the mobile friendliness


JavaScript SEO Q&A September 1st, 2021

validator is a little flaky sometimes.


JavaScript SEO Q&A September 1st, 2021

It oscillates a little bit between issues and non-issues.


JavaScript SEO Q&A September 1st, 2021

Can be caching issues.


JavaScript SEO Q&A September 1st, 2021

I wouldn't worry too much about it,


JavaScript SEO Q&A September 1st, 2021

especially if it's just five URLs that are clapping


JavaScript SEO Q&A September 1st, 2021

between valid and non-valid.


JavaScript SEO Q&A September 1st, 2021

But if it looks good on mobile for you,


JavaScript SEO Q&A September 1st, 2021

it should generally be fine.


JavaScript SEO Q&A September 1st, 2021

And mobile friendliness is one out of many, many factors


JavaScript SEO Q&A September 1st, 2021

that we are looking at in ranking.


JavaScript SEO Q&A September 1st, 2021

So you shouldn't worry too much unless you're really


JavaScript SEO Q&A September 1st, 2021

having issues ranking properly.


JavaScript SEO Q&A September 1st, 2021

But I wouldn't worry too much about it.


JavaScript SEO Q&A September 1st, 2021

So yes, the classifier is a little flaky.


JavaScript SEO Q&A September 1st, 2021

We're working on making it less flaky.


JavaScript SEO Q&A September 1st, 2021

And I think we have already made good progress.


JavaScript SEO Q&A September 1st, 2021

But yeah, I wouldn't worry too much about it.


JavaScript SEO Q&A September 1st, 2021

And I would also check if it's like CSS sometimes not being


JavaScript SEO Q&A September 1st, 2021

fetched properly or if there's some flakiness in your CDN


JavaScript SEO Q&A September 1st, 2021

when you're delivering CSS to a CDN.


JavaScript SEO Q&A September 1st, 2021

Sometimes they behave a little funky.


JavaScript SEO Q&A September 1st, 2021

And if we can't fetch the CSS, then the sliding


JavaScript SEO Q&A September 1st, 2021

might be off.


JavaScript SEO Q&A September 1st, 2021

And then that might be where this issue is coming from.


JavaScript SEO Q&A September 1st, 2021

But in general, if it's basically


JavaScript SEO Q&A September 1st, 2021

flapping between the two states, that


JavaScript SEO Q&A September 1st, 2021

might just be noise from our classifier


JavaScript SEO Q&A September 1st, 2021

not being super determinant.


JavaScript SEO Q&A September 1st, 2021

And I think that's it for the YouTube questions.


JavaScript SEO Q&A September 1st, 2021

There was one question.


JavaScript SEO Q&A September 1st, 2021

When will the SEO update come?


JavaScript SEO Q&A September 1st, 2021

I can't make pre-announcements that we don't have made yet.


JavaScript SEO Q&A September 1st, 2021

So I don't know the future.


JavaScript SEO Q&A September 1st, 2021

And it's not a JavaScript question either.


JavaScript SEO Q&A September 1st, 2021

So sorry I'm not going to answer that much more.


JavaScript SEO Q&A September 1st, 2021

And with that, I open this stage to you


JavaScript SEO Q&A September 1st, 2021

all to ask me your burning questions.


JavaScript SEO Q&A September 1st, 2021

Any questions here in the group?


JavaScript SEO Q&A September 1st, 2021

I have one.


JavaScript SEO Q&A September 1st, 2021

All right.


JavaScript SEO Q&A September 1st, 2021

I posted the other question on the Google for a date.


JavaScript SEO Q&A September 1st, 2021

Maybe I can elaborate a bit.


JavaScript SEO Q&A September 1st, 2021

As I explained, Martin, I'm not the technical guy.


JavaScript SEO Q&A September 1st, 2021

I'm just sort of more of the product owner.


JavaScript SEO Q&A September 1st, 2021

So we created this product where you


JavaScript SEO Q&A September 1st, 2021

can get from a question to an answer really fast on one page.


JavaScript SEO Q&A September 1st, 2021

It's a single-page application.


JavaScript SEO Q&A September 1st, 2021

So it helps you guide through different steps.


JavaScript SEO Q&A September 1st, 2021

And then it shows you the answer.


JavaScript SEO Q&A September 1st, 2021

But on the back end, it is the single-page application


JavaScript SEO Q&A September 1st, 2021

all in JavaScript.


JavaScript SEO Q&A September 1st, 2021

Challenge for us is that we have up to 16,000 answers


JavaScript SEO Q&A September 1st, 2021

on one page.


JavaScript SEO Q&A September 1st, 2021

And we hope that it will increase findability also


JavaScript SEO Q&A September 1st, 2021

for the users from Google.


JavaScript SEO Q&A September 1st, 2021

But it's indexed by Bing, but not by Google.


JavaScript SEO Q&A September 1st, 2021

And we tried a couple of things to improve on this.


JavaScript SEO Q&A September 1st, 2021

But it seems that there are so many different ways


JavaScript SEO Q&A September 1st, 2021

to challenge this that I thought,


JavaScript SEO Q&A September 1st, 2021

well, let's ask the expert on how to best approach it.


JavaScript SEO Q&A September 1st, 2021

MARTIN SPLITT-SANDHOLMSTADT JR. Do you happen to have a sample


JavaScript SEO Q&A September 1st, 2021

URL that I could take a look at?


JavaScript SEO Q&A September 1st, 2021

Yeah, I have a live URL where you can test it.


JavaScript SEO Q&A September 1st, 2021

Exactly.


JavaScript SEO Q&A September 1st, 2021

You can fill this in here.


JavaScript SEO Q&A September 1st, 2021

And then I'll just quickly.


JavaScript SEO Q&A September 1st, 2021

All right.


JavaScript SEO Q&A September 1st, 2021

Is it OK?


JavaScript SEO Q&A September 1st, 2021

This is going to be on the recording.


JavaScript SEO Q&A September 1st, 2021

If that's OK, I can share that.


JavaScript SEO Q&A September 1st, 2021

OK, good.


JavaScript SEO Q&A September 1st, 2021

I really like it when people are fine with that,


JavaScript SEO Q&A September 1st, 2021

because I feel like I don't want to show something in public


JavaScript SEO Q&A September 1st, 2021

that you are not comfortable showing in public.


JavaScript SEO Q&A September 1st, 2021

No, no, no.


JavaScript SEO Q&A September 1st, 2021

I'm very comfortable with it.


JavaScript SEO Q&A September 1st, 2021

I'm really, really grateful.


JavaScript SEO Q&A September 1st, 2021

Because I think this is the moment where


JavaScript SEO Q&A September 1st, 2021

learning happens publicly.


JavaScript SEO Q&A September 1st, 2021

And I think that's generally a good thing.


JavaScript SEO Q&A September 1st, 2021

I understand that a lot of people


JavaScript SEO Q&A September 1st, 2021

are hesitant to share, which is why I'm even more glad


JavaScript SEO Q&A September 1st, 2021

that you are not hesitant to share so that I can go through


JavaScript SEO Q&A September 1st, 2021

this so that you learn as well as everyone watching


JavaScript SEO Q&A September 1st, 2021

this video probably learn something from here.


JavaScript SEO Q&A September 1st, 2021

Oh, come on.


JavaScript SEO Q&A September 1st, 2021

Why does this take so long?


JavaScript SEO Q&A September 1st, 2021

Ah, here we go.


JavaScript SEO Q&A September 1st, 2021

Let's have a look at the tested page,


JavaScript SEO Q&A September 1st, 2021

just roughly the screenshot.


JavaScript SEO Q&A September 1st, 2021

OK.


JavaScript SEO Q&A September 1st, 2021

Let's have a look at the page in a normal browser


JavaScript SEO Q&A September 1st, 2021

tab at the same time.


JavaScript SEO Q&A September 1st, 2021

So what I'm doing is I'm basically just visiting


JavaScript SEO Q&A September 1st, 2021

the URL for the moment.


JavaScript SEO Q&A September 1st, 2021

And oh, it's translating into English for me.


JavaScript SEO Q&A September 1st, 2021

Thank you very much.


JavaScript SEO Q&A September 1st, 2021

That's very nice.


JavaScript SEO Q&A September 1st, 2021

The reason why I do this, and now I'm


JavaScript SEO Q&A September 1st, 2021

also looking at it in a mobile view


JavaScript SEO Q&A September 1st, 2021

so that I have comparable information here.


JavaScript SEO Q&A September 1st, 2021

I'll remove the dev tools.


JavaScript SEO Q&A September 1st, 2021

So OK, that's unfortunate because I, oh, god.


JavaScript SEO Q&A September 1st, 2021

OK, hold on.


JavaScript SEO Q&A September 1st, 2021

Maybe I can zoom out a little bit,


JavaScript SEO Q&A September 1st, 2021

click the window so that I can actually get rid of this banner


JavaScript SEO Q&A September 1st, 2021

here.


JavaScript SEO Q&A September 1st, 2021

This might be my internet connection.


JavaScript SEO Q&A September 1st, 2021

I don't know.


JavaScript SEO Q&A September 1st, 2021

But the banner doesn't go away.


JavaScript SEO Q&A September 1st, 2021

Huh, strange.


JavaScript SEO Q&A September 1st, 2021

Let's reload the page real quick,


JavaScript SEO Q&A September 1st, 2021

just making sure that it's not my internet connection


JavaScript SEO Q&A September 1st, 2021

or something that's going.


JavaScript SEO Q&A September 1st, 2021

Oh, it might be because we are translating it


JavaScript SEO Q&A September 1st, 2021

and Google Translate might have ideas.


JavaScript SEO Q&A September 1st, 2021

Ah, yeah.


JavaScript SEO Q&A September 1st, 2021

There we go.


JavaScript SEO Q&A September 1st, 2021

OK, policy, yes.


JavaScript SEO Q&A September 1st, 2021

So what you see here is the snippet that's in the page.


JavaScript SEO Q&A September 1st, 2021

And it shows you different steps and asks you questions


JavaScript SEO Q&A September 1st, 2021

to come to a specific answer.


JavaScript SEO Q&A September 1st, 2021

And each time you click on a specific answer,


JavaScript SEO Q&A September 1st, 2021

we get a specific URL.


JavaScript SEO Q&A September 1st, 2021

And I would have to click on one of these four, I guess?


JavaScript SEO Q&A September 1st, 2021

OK, in that case, I click on that.


JavaScript SEO Q&A September 1st, 2021

And here you have an answer.


JavaScript SEO Q&A September 1st, 2021

And then the bold text is the solution part.


JavaScript SEO Q&A September 1st, 2021

OK, OK.


JavaScript SEO Q&A September 1st, 2021

So the question is, is this for free?


JavaScript SEO Q&A September 1st, 2021

Yes, the app is for free.


JavaScript SEO Q&A September 1st, 2021

OK.


JavaScript SEO Q&A September 1st, 2021

So yeah, the app is Gratius.


JavaScript SEO Q&A September 1st, 2021

Let's see if it's in the HTML as well.


JavaScript SEO Q&A September 1st, 2021

Yeah.


JavaScript SEO Q&A September 1st, 2021

Or was it like, yeah, comma?


JavaScript SEO Q&A September 1st, 2021

I think it is.


JavaScript SEO Q&A September 1st, 2021

So I go into the mobile-friendly test again.


JavaScript SEO Q&A September 1st, 2021

You could do this in Search Console.


JavaScript SEO Q&A September 1st, 2021

I just don't have access to your page in Search Console.


JavaScript SEO Q&A September 1st, 2021

So yeah, comma.


JavaScript SEO Q&A September 1st, 2021

And here we see something, right?


JavaScript SEO Q&A September 1st, 2021

It's not in the content.


JavaScript SEO Q&A September 1st, 2021

So any of the answer content here in the page,


JavaScript SEO Q&A September 1st, 2021

as in I'm sorry, that was stupid.


JavaScript SEO Q&A September 1st, 2021

So any of the content here, like any of this,


JavaScript SEO Q&A September 1st, 2021

Googlebot doesn't see.


JavaScript SEO Q&A September 1st, 2021

Nope.


JavaScript SEO Q&A September 1st, 2021

So that's a bit unfortunate, because that


JavaScript SEO Q&A September 1st, 2021

means that probably we are not seeing much in terms of content


JavaScript SEO Q&A September 1st, 2021

here.


JavaScript SEO Q&A September 1st, 2021

It's a little tricky to judge this from the HTML right now.


JavaScript SEO Q&A September 1st, 2021

I could copy and paste this over and see what actually is.


JavaScript SEO Q&A September 1st, 2021

Let's actually do that.


JavaScript SEO Q&A September 1st, 2021

So I'll quickly put this into something that


JavaScript SEO Q&A September 1st, 2021

shows us the HTML a little nicer.


JavaScript SEO Q&A September 1st, 2021

At this point, I am experimenting a little bit.


JavaScript SEO Q&A September 1st, 2021

Let's tidy this up so that we can see.


JavaScript SEO Q&A September 1st, 2021

So this is pretty much the content that Googlebot sees.


JavaScript SEO Q&A September 1st, 2021

And when it comes, you said the main content


JavaScript SEO Q&A September 1st, 2021

is hidden behind here, right?


JavaScript SEO Q&A September 1st, 2021

Yeah, so that's the subject selections.


JavaScript SEO Q&A September 1st, 2021

So you have, and I think this is the path.


JavaScript SEO Q&A September 1st, 2021

So this is the first user tab.


JavaScript SEO Q&A September 1st, 2021

And then the user can go in the page,


JavaScript SEO Q&A September 1st, 2021

make the selection from different tabs,


JavaScript SEO Q&A September 1st, 2021

and then go to the solution.


JavaScript SEO Q&A September 1st, 2021

So this is an improvement where you have FAQ websites


JavaScript SEO Q&A September 1st, 2021

with a lot of trees where a customer loses their way.


JavaScript SEO Q&A September 1st, 2021

And here we have in one page, you go from a question


JavaScript SEO Q&A September 1st, 2021

to an answer.


JavaScript SEO Q&A September 1st, 2021

And we help guide the customer as good as possible


JavaScript SEO Q&A September 1st, 2021

to help them out.


JavaScript SEO Q&A September 1st, 2021

And it works really good for customers.


JavaScript SEO Q&A September 1st, 2021

But this is still a challenge.


JavaScript SEO Q&A September 1st, 2021

Yes, exactly.


JavaScript SEO Q&A September 1st, 2021

So looking from a Google perspective,


JavaScript SEO Q&A September 1st, 2021

if I'm coming to this website and I'm seeing,


JavaScript SEO Q&A September 1st, 2021

so this is not really content.


JavaScript SEO Q&A September 1st, 2021

This is just navigation, which is fine, right?


JavaScript SEO Q&A September 1st, 2021

This is not really content.


JavaScript SEO Q&A September 1st, 2021

This is also kind of like still, I mean, I don't speak Dutch.


JavaScript SEO Q&A September 1st, 2021

So this is a little tricky.


JavaScript SEO Q&A September 1st, 2021

But it looks like this is not that useful as content


JavaScript SEO Q&A September 1st, 2021

in terms of a user would not necessarily search


JavaScript SEO Q&A September 1st, 2021

for something like this here.


JavaScript SEO Q&A September 1st, 2021

I know that from a user's perspective,


JavaScript SEO Q&A September 1st, 2021

if I tap on this on your site as a human being,


JavaScript SEO Q&A September 1st, 2021

I get interesting content for these things.


JavaScript SEO Q&A September 1st, 2021

But if I'm just looking at this, this is not content either.


JavaScript SEO Q&A September 1st, 2021

This is a few words.


JavaScript SEO Q&A September 1st, 2021

And then down here, this is privacy policy, contact,


JavaScript SEO Q&A September 1st, 2021

Twitter stuff.


JavaScript SEO Q&A September 1st, 2021

There is not much in terms of content on this page, right?


JavaScript SEO Q&A September 1st, 2021

If I'm looking at this, I know as a user,


JavaScript SEO Q&A September 1st, 2021

there is content hidden behind these things.


JavaScript SEO Q&A September 1st, 2021

But if I'm looking at it as a user on the way


JavaScript SEO Q&A September 1st, 2021

that I'm seeing it as Googlebot would see it,


JavaScript SEO Q&A September 1st, 2021

this page doesn't really have much content, right?


JavaScript SEO Q&A September 1st, 2021

All right, yeah.


JavaScript SEO Q&A September 1st, 2021

I mean, we know that there is more content hidden


JavaScript SEO Q&A September 1st, 2021

behind these, but it's not here.


JavaScript SEO Q&A September 1st, 2021

It's not present in the HTML as Googlebot sees it.


JavaScript SEO Q&A September 1st, 2021

So the problem here seems to be that the way that you


JavaScript SEO Q&A September 1st, 2021

implemented it, for the user, all the content


JavaScript SEO Q&A September 1st, 2021

is hidden behind these things.


JavaScript SEO Q&A September 1st, 2021

But Googlebot doesn't see it at all.


JavaScript SEO Q&A September 1st, 2021

It's just not there.


JavaScript SEO Q&A September 1st, 2021

And I think that's because this is not a link.


JavaScript SEO Q&A September 1st, 2021

This is probably something else.


JavaScript SEO Q&A September 1st, 2021

Oh, it turns out this is a link.


JavaScript SEO Q&A September 1st, 2021

It's just not really a link.


JavaScript SEO Q&A September 1st, 2021

It is a link in the sense of the HTML tag is a link tag.


JavaScript SEO Q&A September 1st, 2021

Good.


JavaScript SEO Q&A September 1st, 2021

But it doesn't have anywhere to go.


JavaScript SEO Q&A September 1st, 2021

It doesn't point to a URL.


JavaScript SEO Q&A September 1st, 2021

So what you're effectively having here is buttons,


JavaScript SEO Q&A September 1st, 2021

basically, because you have to click on things.


JavaScript SEO Q&A September 1st, 2021

Googlebot doesn't click on things.


JavaScript SEO Q&A September 1st, 2021

So it doesn't see the content that


JavaScript SEO Q&A September 1st, 2021

is loaded when I click on this.


JavaScript SEO Q&A September 1st, 2021

You have multiple opportunities on how to improve that.


JavaScript SEO Q&A September 1st, 2021

One easy way to improve this would be to make this into,


JavaScript SEO Q&A September 1st, 2021

now I know what you mean with we are implementing HREF.


JavaScript SEO Q&A September 1st, 2021

And that's a clever thing.


JavaScript SEO Q&A September 1st, 2021

What you could do is you could make this point to a URL


JavaScript SEO Q&A September 1st, 2021

where you already load this.


JavaScript SEO Q&A September 1st, 2021

So basically, you have, I don't know, I'll, what I,


JavaScript SEO Q&A September 1st, 2021

quick question, can you actually see,


JavaScript SEO Q&A September 1st, 2021

when I share the Chrome tab, can you see the developer tools


JavaScript SEO Q&A September 1st, 2021

that I'm navigating in right now?


JavaScript SEO Q&A September 1st, 2021

Or do you only see the page?


JavaScript SEO Q&A September 1st, 2021

I see the, I see the, uh,


JavaScript SEO Q&A September 1st, 2021

Ah, you can't see the developer tools.


JavaScript SEO Q&A September 1st, 2021

I can't see the developer tools.


JavaScript SEO Q&A September 1st, 2021

Interesting.


JavaScript SEO Q&A September 1st, 2021

Hold on.


JavaScript SEO Q&A September 1st, 2021

Let me actually share the entire window


JavaScript SEO Q&A September 1st, 2021

so that's a little easier to show you what I'm doing here.


JavaScript SEO Q&A September 1st, 2021

Share a window.


JavaScript SEO Q&A September 1st, 2021

Share this window, please.


JavaScript SEO Q&A September 1st, 2021

Yes, do share this window.


JavaScript SEO Q&A September 1st, 2021

This is going to be freaky because it's going to be,


JavaScript SEO Q&A September 1st, 2021

OK, here we go.


JavaScript SEO Q&A September 1st, 2021

So now there should be developer tools on the bottom as well.


JavaScript SEO Q&A September 1st, 2021

And I'm trying to increase the size a little bit.


JavaScript SEO Q&A September 1st, 2021

Ah, Jesus.


JavaScript SEO Q&A September 1st, 2021

This is slow.


JavaScript SEO Q&A September 1st, 2021

All right, here we go.


JavaScript SEO Q&A September 1st, 2021

So one way of dealing with this is actually


JavaScript SEO Q&A September 1st, 2021

making this into a link that a crawler can follow.


JavaScript SEO Q&A September 1st, 2021

And it could be, I don't know, slash service contact


JavaScript SEO Q&A September 1st, 2021

question mark or like even better would be like something


JavaScript SEO Q&A September 1st, 2021

like slash, uh, aktiviren.


JavaScript SEO Q&A September 1st, 2021

And then basically when someone would come to this specific URL


JavaScript SEO Q&A September 1st, 2021

you would already load this content so that you actually


JavaScript SEO Q&A September 1st, 2021

have or whatever content I clicked on.


JavaScript SEO Q&A September 1st, 2021

I can't remember what I clicked on,


JavaScript SEO Q&A September 1st, 2021

but it looks like I'm on this page.


JavaScript SEO Q&A September 1st, 2021

So that Google has a page where this content is actually


JavaScript SEO Q&A September 1st, 2021

present.


JavaScript SEO Q&A September 1st, 2021

The alternative would be to have this content load


JavaScript SEO Q&A September 1st, 2021

and just hide it and then show it when someone clicks on it.


JavaScript SEO Q&A September 1st, 2021

That way Googlebot would also see it,


JavaScript SEO Q&A September 1st, 2021

but the user still has the same experience.


JavaScript SEO Q&A September 1st, 2021

So the problem here seems to be that Googlebot does not


JavaScript SEO Q&A September 1st, 2021

actually see the content.


JavaScript SEO Q&A September 1st, 2021

And actually what I'm wondering is if I even


JavaScript SEO Q&A September 1st, 2021

shared the window properly if I did.


JavaScript SEO Q&A September 1st, 2021

So the problem really seems to be that Googlebot


JavaScript SEO Q&A September 1st, 2021

can't see the content.


JavaScript SEO Q&A September 1st, 2021

So when we don't see the content,


JavaScript SEO Q&A September 1st, 2021

it looks like the page doesn't really have much.


JavaScript SEO Q&A September 1st, 2021

And it's interesting to see that Giacomo answered in the chat


JavaScript SEO Q&A September 1st, 2021

as well, and your guess is right.


JavaScript SEO Q&A September 1st, 2021

So yeah, make this into links that


JavaScript SEO Q&A September 1st, 2021

are having a URL that loads the content so that Googlebot can


JavaScript SEO Q&A September 1st, 2021

be like, so this URL doesn't really show me much,


JavaScript SEO Q&A September 1st, 2021

but these links here, they point me


JavaScript SEO Q&A September 1st, 2021

to content that is interesting for users


JavaScript SEO Q&A September 1st, 2021

so that I would index this.


JavaScript SEO Q&A September 1st, 2021

It looks like we are not seeing the content


JavaScript SEO Q&A September 1st, 2021

and that's your problem.


JavaScript SEO Q&A September 1st, 2021

Cool.


JavaScript SEO Q&A September 1st, 2021

Does that make sense?


JavaScript SEO Q&A September 1st, 2021

Yes, yes.


JavaScript SEO Q&A September 1st, 2021

But I think you click on one button


JavaScript SEO Q&A September 1st, 2021

and then it showed content, but there is a possibility


JavaScript SEO Q&A September 1st, 2021

to click to make it five levels deep.


JavaScript SEO Q&A September 1st, 2021

So to go from a main topic to a subtopic to a subtopic


JavaScript SEO Q&A September 1st, 2021

and then to a specific question.


JavaScript SEO Q&A September 1st, 2021

Yeah.


JavaScript SEO Q&A September 1st, 2021

And then would this then still work?


JavaScript SEO Q&A September 1st, 2021

It would still work.


JavaScript SEO Q&A September 1st, 2021

And in that case, what I would do


JavaScript SEO Q&A September 1st, 2021

is you have to balance things a little bit.


JavaScript SEO Q&A September 1st, 2021

The simple answer to the question is it depends.


JavaScript SEO Q&A September 1st, 2021

The slightly longer answer, I'm so


JavaScript SEO Q&A September 1st, 2021

sorry if I'm taking time away from everyone else.


JavaScript SEO Q&A September 1st, 2021

Everyone is fine with it, so I'll just continue.


JavaScript SEO Q&A September 1st, 2021

You want to have each URL should have enough content


JavaScript SEO Q&A September 1st, 2021

to be relevant, which is like if you have very specific


JavaScript SEO Q&A September 1st, 2021

questions with a one word or one sentence answer,


JavaScript SEO Q&A September 1st, 2021

then that's not much in terms of content.


JavaScript SEO Q&A September 1st, 2021

So you probably want to structure it


JavaScript SEO Q&A September 1st, 2021

so that a page always has enough coherent content, which


JavaScript SEO Q&A September 1st, 2021

means content that belongs kind of together.


JavaScript SEO Q&A September 1st, 2021

And then you can still hide things,


JavaScript SEO Q&A September 1st, 2021

but you would load the content anyway.


JavaScript SEO Q&A September 1st, 2021

So Googlebot finds enough content


JavaScript SEO Q&A September 1st, 2021

to say this page looks like it's answering enough


JavaScript SEO Q&A September 1st, 2021

around the question and the question in question.


JavaScript SEO Q&A September 1st, 2021

That sounds weird.


JavaScript SEO Q&A September 1st, 2021

The specific question.


JavaScript SEO Q&A September 1st, 2021

But yeah, you would have to be a little,


JavaScript SEO Q&A September 1st, 2021

you would have to try things out a little bit, basically.


JavaScript SEO Q&A September 1st, 2021

But I think the main problem here really


JavaScript SEO Q&A September 1st, 2021

is that we are not seeing the content,


JavaScript SEO Q&A September 1st, 2021

and we have no way of reaching the content right now.


JavaScript SEO Q&A September 1st, 2021

So that's why the page is not being indexed,


JavaScript SEO Q&A September 1st, 2021

because we are not seeing any content.


JavaScript SEO Q&A September 1st, 2021

We are seeing boilerplate and links,


JavaScript SEO Q&A September 1st, 2021

but not really the content.


JavaScript SEO Q&A September 1st, 2021

But you can do that with lots of levels of nesting.


JavaScript SEO Q&A September 1st, 2021

It's just a matter of how much nesting makes sense


JavaScript SEO Q&A September 1st, 2021

and where do you want to have the page contain


JavaScript SEO Q&A September 1st, 2021

all the questions to a specific product or service,


JavaScript SEO Q&A September 1st, 2021

and then just hide them and show them


JavaScript SEO Q&A September 1st, 2021

as users navigate on the page, but show all of them in one


JavaScript SEO Q&A September 1st, 2021

go to Googlebot.


JavaScript SEO Q&A September 1st, 2021

That's something that you would have to discuss probably


JavaScript SEO Q&A September 1st, 2021

with an SEO expert to help you, guide you


JavaScript SEO Q&A September 1st, 2021

in making decisions there.


JavaScript SEO Q&A September 1st, 2021

Thank you so much.


JavaScript SEO Q&A September 1st, 2021

You're very welcome.


JavaScript SEO Q&A September 1st, 2021

Giacomo is asking if I answered the question from Charles


JavaScript SEO Q&A September 1st, 2021

regarding the history push state.


JavaScript SEO Q&A September 1st, 2021

I did.


JavaScript SEO Q&A September 1st, 2021

I said I'm not 100% sure, but I think


JavaScript SEO Q&A September 1st, 2021

I tested this a long time ago.


JavaScript SEO Q&A September 1st, 2021

And I think it behaved the way that we


JavaScript SEO Q&A September 1st, 2021

are associating the page with the URL after push state.


JavaScript SEO Q&A September 1st, 2021

But I'll test that.


JavaScript SEO Q&A September 1st, 2021

I'll test that.


JavaScript SEO Q&A September 1st, 2021

And then I'll update the YouTube question


JavaScript SEO Q&A September 1st, 2021

with the answer to the question.


JavaScript SEO Q&A September 1st, 2021

Cool.


JavaScript SEO Q&A September 1st, 2021

In that case, any other questions?


JavaScript SEO Q&A September 1st, 2021

Eric?


JavaScript SEO Q&A September 1st, 2021

Dave?


JavaScript SEO Q&A September 1st, 2021

Giacomo?


JavaScript SEO Q&A September 1st, 2021

Oh, Giacomo raised his hand.


JavaScript SEO Q&A September 1st, 2021

There we go.


JavaScript SEO Q&A September 1st, 2021

So hi, Martin.


JavaScript SEO Q&A September 1st, 2021

Hi, Giacomo.


JavaScript SEO Q&A September 1st, 2021

So when WRAS is rendering a page as it is, for example,


JavaScript SEO Q&A September 1st, 2021

very bad coded, let's say an infinite loop,


JavaScript SEO Q&A September 1st, 2021

sometimes the page crash.


JavaScript SEO Q&A September 1st, 2021

And in the end, Google indexing the non-rendered version


JavaScript SEO Q&A September 1st, 2021

because can't render the page.


JavaScript SEO Q&A September 1st, 2021

So for this specific behavior, there


JavaScript SEO Q&A September 1st, 2021

is an API, the reporting API, that is sending also


JavaScript SEO Q&A September 1st, 2021

some the crash report to a specific server.


JavaScript SEO Q&A September 1st, 2021

So I think that in the Google Web Rendering Service,


JavaScript SEO Q&A September 1st, 2021

this is not enabled.


JavaScript SEO Q&A September 1st, 2021

We can't get all the other reporting API with the reporting


JavaScript SEO Q&A September 1st, 2021

observer and JavaScript, like sending back the beacon,


JavaScript SEO Q&A September 1st, 2021

as you know, I'm doing at the moment.


JavaScript SEO Q&A September 1st, 2021

Did you try beacon API?


JavaScript SEO Q&A September 1st, 2021

Does it work in WRAS?


JavaScript SEO Q&A September 1st, 2021

Yeah.


JavaScript SEO Q&A September 1st, 2021

Yeah, it's working perfectly.


JavaScript SEO Q&A September 1st, 2021

Beacon API, JavaScript, post and fetch without any problem.


JavaScript SEO Q&A September 1st, 2021

And we can use the reporting observer


JavaScript SEO Q&A September 1st, 2021

that there is in Chrome now.


JavaScript SEO Q&A September 1st, 2021

So we can get all the stuff from the reporting API,


JavaScript SEO Q&A September 1st, 2021

but not the crash report, obviously,


JavaScript SEO Q&A September 1st, 2021

because the page is crashing before sending back the beacon.


JavaScript SEO Q&A September 1st, 2021

I'm asking if there is a possibility,


JavaScript SEO Q&A September 1st, 2021

if you can enable only the crash report in the Web Rendering


JavaScript SEO Q&A September 1st, 2021

Service.


JavaScript SEO Q&A September 1st, 2021

This is just a question.


JavaScript SEO Q&A September 1st, 2021

I don't know the answer to that.


JavaScript SEO Q&A September 1st, 2021

I can ask the rendering team.


JavaScript SEO Q&A September 1st, 2021

That I can do.


JavaScript SEO Q&A September 1st, 2021

Yeah, I'll ask the rendering team.


JavaScript SEO Q&A September 1st, 2021

Maybe we can enable that.


JavaScript SEO Q&A September 1st, 2021

I think that would be a cool addition.


JavaScript SEO Q&A September 1st, 2021

But maybe they're like, oh, this is a lot of work.


JavaScript SEO Q&A September 1st, 2021

Or maybe they're like, no, we don't


JavaScript SEO Q&A September 1st, 2021

want to do that because X-Ray is that.


JavaScript SEO Q&A September 1st, 2021

But I can definitely ask them.


JavaScript SEO Q&A September 1st, 2021

Yeah, because in that case, we don't really


JavaScript SEO Q&A September 1st, 2021

know if the page is rendered correctly or not.


JavaScript SEO Q&A September 1st, 2021

In all the other stuff, we can get from Search Console


JavaScript SEO Q&A September 1st, 2021

or everything.


JavaScript SEO Q&A September 1st, 2021

In this case, obviously, you probably


JavaScript SEO Q&A September 1st, 2021

find that the page is not rendering correctly,


JavaScript SEO Q&A September 1st, 2021

also visiting the page because if it's not rendering correctly


JavaScript SEO Q&A September 1st, 2021

on your browser, it's probably not


JavaScript SEO Q&A September 1st, 2021

rendering correctly on the Web Rendering Services.


JavaScript SEO Q&A September 1st, 2021

But sometimes you have weird stuff in the service.


JavaScript SEO Q&A September 1st, 2021

And it's not.


JavaScript SEO Q&A September 1st, 2021

I can see so many edge spaces that would only crash in WRS.


JavaScript SEO Q&A September 1st, 2021

Yeah, I feel you.


JavaScript SEO Q&A September 1st, 2021

I see it's a good question.


JavaScript SEO Q&A September 1st, 2021

I'll bring this up with the rendering team.


JavaScript SEO Q&A September 1st, 2021

Thank you, thank you, thank you.


JavaScript SEO Q&A September 1st, 2021

Thank you.


JavaScript SEO Q&A September 1st, 2021

Thanks for making the suggestion.


JavaScript SEO Q&A September 1st, 2021

All right, any other questions?


JavaScript SEO Q&A September 1st, 2021

I've got one.


JavaScript SEO Q&A September 1st, 2021

Is there any way you can suggest with some of the official tools


JavaScript SEO Q&A September 1st, 2021

or Google to see something that relies on Intersection


JavaScript SEO Q&A September 1st, 2021

Observer that's maybe further down the page,


JavaScript SEO Q&A September 1st, 2021

testing whether that will work before it's indexed


JavaScript SEO Q&A September 1st, 2021

because the suit tools don't really expand the viewport,


JavaScript SEO Q&A September 1st, 2021

it seems.


JavaScript SEO Q&A September 1st, 2021

So current case, I've got a client in there


JavaScript SEO Q&A September 1st, 2021

in trying to introduce some components that are lazy loaded


JavaScript SEO Q&A September 1st, 2021

down towards the bottom of the page.


JavaScript SEO Q&A September 1st, 2021

I can't understand why they want to do it as well.


JavaScript SEO Q&A September 1st, 2021

But they're not very keen on do trust me.


JavaScript SEO Q&A September 1st, 2021

It looks like it'll work.


JavaScript SEO Q&A September 1st, 2021

I'm sure it will work.


JavaScript SEO Q&A September 1st, 2021

I can't see any reason why it wouldn't work.


JavaScript SEO Q&A September 1st, 2021

But they're just very nervous of do trust me, it'll be fine.


JavaScript SEO Q&A September 1st, 2021

I mean, obviously, when something's indexed,


JavaScript SEO Q&A September 1st, 2021

that's great.


JavaScript SEO Q&A September 1st, 2021

We can look in the rendered HTML and see if it's worked.


JavaScript SEO Q&A September 1st, 2021

But are there any things?


JavaScript SEO Q&A September 1st, 2021

I mean, I don't know.


JavaScript SEO Q&A September 1st, 2021

Would it follow if you put something like an anchor tag


JavaScript SEO Q&A September 1st, 2021

in there or something like that?


JavaScript SEO Q&A September 1st, 2021

Could that work?


JavaScript SEO Q&A September 1st, 2021

That could work.


JavaScript SEO Q&A September 1st, 2021

So the thing is, it's hilarious that you bring this up


JavaScript SEO Q&A September 1st, 2021

because technically, what you found is a bug in the suit


JavaScript SEO Q&A September 1st, 2021

tools.


JavaScript SEO Q&A September 1st, 2021

It's basically configuration drift,


JavaScript SEO Q&A September 1st, 2021

where at some point, for some reason,


JavaScript SEO Q&A September 1st, 2021

and I don't know why it happened.


JavaScript SEO Q&A September 1st, 2021

It wasn't a seemingly unrelated comment in the past.


JavaScript SEO Q&A September 1st, 2021

They changed the configuration for mobile in the suit tools.


JavaScript SEO Q&A September 1st, 2021

And I tried to fix that.


JavaScript SEO Q&A September 1st, 2021

But then I broke a lot of other things.


JavaScript SEO Q&A September 1st, 2021

And I was like, ah, damn it.


JavaScript SEO Q&A September 1st, 2021

Basically, on and off for the last two years,


JavaScript SEO Q&A September 1st, 2021

I have been trying to fix this bug.


JavaScript SEO Q&A September 1st, 2021

And I haven't really made significant progress on it.


JavaScript SEO Q&A September 1st, 2021

And to be fair, no one's here.


JavaScript SEO Q&A September 1st, 2021

You're the first one since 2018.


JavaScript SEO Q&A September 1st, 2021

And I know this bug existed to mention this


JavaScript SEO Q&A September 1st, 2021

and to recognize this.


JavaScript SEO Q&A September 1st, 2021

So I didn't really have that much motivation


JavaScript SEO Q&A September 1st, 2021

to really push for fixing this.


JavaScript SEO Q&A September 1st, 2021

What I would suggest, if it works, and it might not,


JavaScript SEO Q&A September 1st, 2021

is I test these things with the rich result test


JavaScript SEO Q&A September 1st, 2021

because there, it lets me pick which crawler I want.


JavaScript SEO Q&A September 1st, 2021

And then I just go with the desktop one.


JavaScript SEO Q&A September 1st, 2021

And in desktop, it should expand the viewport.


JavaScript SEO Q&A September 1st, 2021

And that's how I tested it for it.


JavaScript SEO Q&A September 1st, 2021

So that might be a weird workaround.


JavaScript SEO Q&A September 1st, 2021

I am very well aware of it.


JavaScript SEO Q&A September 1st, 2021

But that might get you there.


JavaScript SEO Q&A September 1st, 2021

Yeah, that's cool.


JavaScript SEO Q&A September 1st, 2021

I've noticed that word.


JavaScript SEO Q&A September 1st, 2021

That's brilliant.


JavaScript SEO Q&A September 1st, 2021

Yeah, that's great.


JavaScript SEO Q&A September 1st, 2021

And the reason, really, is just configuration drift


JavaScript SEO Q&A September 1st, 2021

where this has been disabled.


JavaScript SEO Q&A September 1st, 2021

And it seems that that's also part of the reason why you're


JavaScript SEO Q&A September 1st, 2021

only getting above the fold in the mobile-friendly test


JavaScript SEO Q&A September 1st, 2021

and in the screenshot for the suit tools,


JavaScript SEO Q&A September 1st, 2021

wherever it is using the mobile version of the crawler.


JavaScript SEO Q&A September 1st, 2021

It's because expand viewport is not happening.


JavaScript SEO Q&A September 1st, 2021

And it's not a real thing in indexing.


JavaScript SEO Q&A September 1st, 2021

In indexing, that happens to be correct.


JavaScript SEO Q&A September 1st, 2021

But it's one of the tiny differences between the testing


JavaScript SEO Q&A September 1st, 2021

tools and the real-world indexing,


JavaScript SEO Q&A September 1st, 2021

which is unfortunate.


JavaScript SEO Q&A September 1st, 2021

I'm aware.


JavaScript SEO Q&A September 1st, 2021

But the bug doesn't seem to be important enough


JavaScript SEO Q&A September 1st, 2021

to get enough priority to get fixed.


JavaScript SEO Q&A September 1st, 2021

And I don't assign it much more priority either.


JavaScript SEO Q&A September 1st, 2021

It's got so much to do.


JavaScript SEO Q&A September 1st, 2021

Yeah, but I'm actually very happy that someone


JavaScript SEO Q&A September 1st, 2021

spotted the bug.


JavaScript SEO Q&A September 1st, 2021

I really enjoy people doing a bunch of tests


JavaScript SEO Q&A September 1st, 2021

and figuring these things out.


JavaScript SEO Q&A September 1st, 2021

It's usually either you, Dave, or Giacomo, who are like,


JavaScript SEO Q&A September 1st, 2021

I tested this thing.


JavaScript SEO Q&A September 1st, 2021

It does this weird thing.


JavaScript SEO Q&A September 1st, 2021

Can you tell me why?


JavaScript SEO Q&A September 1st, 2021

And I'm like, yeah, I know.


JavaScript SEO Q&A September 1st, 2021

I know.


JavaScript SEO Q&A September 1st, 2021

I can't remember who of the two it


JavaScript SEO Q&A September 1st, 2021

was that brought up the Web Worker thing as well.


JavaScript SEO Q&A September 1st, 2021

But yeah, it's these edge cases where I'm like, oh, I


JavaScript SEO Q&A September 1st, 2021

wonder if anyone notices it.


JavaScript SEO Q&A September 1st, 2021

And very few people do.


JavaScript SEO Q&A September 1st, 2021

Kind of funny.


JavaScript SEO Q&A September 1st, 2021

Yeah.


JavaScript SEO Q&A September 1st, 2021

I think concerning me and Giacomo noticing this stuff,


JavaScript SEO Q&A September 1st, 2021

probably not that high priority.


JavaScript SEO Q&A September 1st, 2021

There's also Adam from Is it DeepCurl?


JavaScript SEO Q&A September 1st, 2021

Is it Oncurl?


JavaScript SEO Q&A September 1st, 2021

I can't remember which company he works for.


JavaScript SEO Q&A September 1st, 2021

But Adam Gend, if I may?


JavaScript SEO Q&A September 1st, 2021

Adam Gend, yeah.


JavaScript SEO Q&A September 1st, 2021

He used to be DeepCurl.


JavaScript SEO Q&A September 1st, 2021

He's a solo now.


JavaScript SEO Q&A September 1st, 2021

He's a freelance.


JavaScript SEO Q&A September 1st, 2021

Ah, OK.


JavaScript SEO Q&A September 1st, 2021

Because he was the first one to spot when Evergreen Googlebot


JavaScript SEO Q&A September 1st, 2021

rolled out.


JavaScript SEO Q&A September 1st, 2021

He was the first one to ping me on Twitter being like, hey,


JavaScript SEO Q&A September 1st, 2021

we just tested this API that isn't


JavaScript SEO Q&A September 1st, 2021

available in the old Chrome, but Googlebot seems to support it.


JavaScript SEO Q&A September 1st, 2021

Is that like, I'm like, yep, you just


JavaScript SEO Q&A September 1st, 2021

came across a URL that is already rolled out


JavaScript SEO Q&A September 1st, 2021

for the Evergreen Googlebot.


JavaScript SEO Q&A September 1st, 2021

And that was kind of cool.


JavaScript SEO Q&A September 1st, 2021

Because we were teasing the rollouts at some conference


JavaScript SEO Q&A September 1st, 2021

right before it started happening,


JavaScript SEO Q&A September 1st, 2021

and no one picked up on it.


JavaScript SEO Q&A September 1st, 2021

Everyone was into all sorts of conspiracy theories about SEO.


JavaScript SEO Q&A September 1st, 2021

And we were like, Jan and me were


JavaScript SEO Q&A September 1st, 2021

dropping hints of Evergreen Googlebot to happen,


JavaScript SEO Q&A September 1st, 2021

and no one picked up.


JavaScript SEO Q&A September 1st, 2021

And Adam was like, huh, and then tested things.


JavaScript SEO Q&A September 1st, 2021

And then he's like, this one URL reports differently


JavaScript SEO Q&A September 1st, 2021

than all the other URLs.


JavaScript SEO Q&A September 1st, 2021

What is this?


JavaScript SEO Q&A September 1st, 2021

What is this about?


JavaScript SEO Q&A September 1st, 2021

And we're like, ta-da.


JavaScript SEO Q&A September 1st, 2021

Welcome in the 10% of URLs that already


JavaScript SEO Q&A September 1st, 2021

have the Evergreen Googlebot.


JavaScript SEO Q&A September 1st, 2021

So that's kind of funny.


JavaScript SEO Q&A September 1st, 2021

Yeah, but it's an inconsistency in the testing tools.


JavaScript SEO Q&A September 1st, 2021

Sorry for that.


JavaScript SEO Q&A September 1st, 2021

OK, now it's great if the rich results in the browser


JavaScript SEO Q&A September 1st, 2021

and desktop and that's problem solved,


JavaScript SEO Q&A September 1st, 2021

then we've got solution.


JavaScript SEO Q&A September 1st, 2021

That's great.


JavaScript SEO Q&A September 1st, 2021

Thank you.


JavaScript SEO Q&A September 1st, 2021

And yeah, I mean, that's the thing.


JavaScript SEO Q&A September 1st, 2021

In the other tools, you can't choose


JavaScript SEO Q&A September 1st, 2021

which crawler you're getting, because it


JavaScript SEO Q&A September 1st, 2021

gets the crawler that is actually


JavaScript SEO Q&A September 1st, 2021

responsible for the thing and actually is


JavaScript SEO Q&A September 1st, 2021

running on the thing, but rich results


JavaScript SEO Q&A September 1st, 2021

has less to choose, and desktop doesn't have the problem.


JavaScript SEO Q&A September 1st, 2021

Excellent.


JavaScript SEO Q&A September 1st, 2021

Cool.


JavaScript SEO Q&A September 1st, 2021

Any other questions?


JavaScript SEO Q&A September 1st, 2021

Anyone else?


JavaScript SEO Q&A September 1st, 2021

Or do they have more questions?


JavaScript SEO Q&A September 1st, 2021

I don't know.


JavaScript SEO Q&A September 1st, 2021

Either way, fine.


JavaScript SEO Q&A September 1st, 2021

Regarding the suite tool and all the testing tools,


JavaScript SEO Q&A September 1st, 2021

would it be possible in the future, in the far future,


JavaScript SEO Q&A September 1st, 2021

to have a different user agent, or at least


JavaScript SEO Q&A September 1st, 2021

to have something specific that we can just


JavaScript SEO Q&A September 1st, 2021

understand that those are testing tools,


JavaScript SEO Q&A September 1st, 2021

and the other one is the real Google bot?


JavaScript SEO Q&A September 1st, 2021

Knowing from previously having spoken to the team,


JavaScript SEO Q&A September 1st, 2021

it's very unlikely to happen, because they are really, really


JavaScript SEO Q&A September 1st, 2021

reluctant to do that.


JavaScript SEO Q&A September 1st, 2021

Because then, obviously, people would do stupid things,


JavaScript SEO Q&A September 1st, 2021

as in sniff the user agent for the testing tools


JavaScript SEO Q&A September 1st, 2021

and then do something different.


JavaScript SEO Q&A September 1st, 2021

Then people are like, you're testing tools, show me this.


JavaScript SEO Q&A September 1st, 2021

And then we're like, well, it's because you're


JavaScript SEO Q&A September 1st, 2021

treating them differently.


JavaScript SEO Q&A September 1st, 2021

And you know how it is in web development these days.


JavaScript SEO Q&A September 1st, 2021

It's layers upon layers upon layers.


JavaScript SEO Q&A September 1st, 2021

So it might be that they are not even


JavaScript SEO Q&A September 1st, 2021

aware that that's happening, because the framework made


JavaScript SEO Q&A September 1st, 2021

that choice for them.


JavaScript SEO Q&A September 1st, 2021

And they are just blindly using the framework


JavaScript SEO Q&A September 1st, 2021

without noticing.


JavaScript SEO Q&A September 1st, 2021

And it's interesting that you bring that up,


JavaScript SEO Q&A September 1st, 2021

because I had this conversation with a team about being


JavaScript SEO Q&A September 1st, 2021

able, in Search Console, I was like, hey,


JavaScript SEO Q&A September 1st, 2021

would it be possible to let me, similar to a rich result test,


JavaScript SEO Q&A September 1st, 2021

would it be able to let me choose which crawler I want


JavaScript SEO Q&A September 1st, 2021

to see?


JavaScript SEO Q&A September 1st, 2021

And they're like, no.


JavaScript SEO Q&A September 1st, 2021

They're like, why not?


JavaScript SEO Q&A September 1st, 2021

And they're like, because we want, no, it makes sense.


JavaScript SEO Q&A September 1st, 2021

They are reasonable about this.


JavaScript SEO Q&A September 1st, 2021

I'm not saying that this is a stupid thing.


JavaScript SEO Q&A September 1st, 2021

The reasoning behind it is they want


JavaScript SEO Q&A September 1st, 2021

to make sure that when you are using the URL inspection


JavaScript SEO Q&A September 1st, 2021

tool in Search Console, you are seeing what Googlebot would


JavaScript SEO Q&A September 1st, 2021

be seeing within the limitations of not using cache and blah,


JavaScript SEO Q&A September 1st, 2021

blah, blah.


JavaScript SEO Q&A September 1st, 2021

So it makes sense that you can't choose the user agent,


JavaScript SEO Q&A September 1st, 2021

because then if you choose the user agent that is not


JavaScript SEO Q&A September 1st, 2021

the user agent or not the user agent is the wrong term,


JavaScript SEO Q&A September 1st, 2021

the crawler.


JavaScript SEO Q&A September 1st, 2021

If you're choosing the crawler that we are not actually


JavaScript SEO Q&A September 1st, 2021

using to crawl the page, you might get confused about it.


JavaScript SEO Q&A September 1st, 2021

But it looks like everything's loading.


JavaScript SEO Q&A September 1st, 2021

The URL's like, yeah, that's because you chose


JavaScript SEO Q&A September 1st, 2021

to use the desktop crawler.


JavaScript SEO Q&A September 1st, 2021

But we're actually using the mobile crawler.


JavaScript SEO Q&A September 1st, 2021

So that's why we are not seeing it.


JavaScript SEO Q&A September 1st, 2021

It's like, oh.


JavaScript SEO Q&A September 1st, 2021

So I see their point, even though I still


JavaScript SEO Q&A September 1st, 2021

feel like it would be nice to have some sort of indication


JavaScript SEO Q&A September 1st, 2021

if it's the testing tools or not.


JavaScript SEO Q&A September 1st, 2021

But I see that point that the testing tools are


JavaScript SEO Q&A September 1st, 2021

trying to be as close to the real thing as possible


JavaScript SEO Q&A September 1st, 2021

so that you should have the ability to either detect


JavaScript SEO Q&A September 1st, 2021

if it's the testing tool or to change the way


JavaScript SEO Q&A September 1st, 2021

that the actual crawler would interact with the page.


JavaScript SEO Q&A September 1st, 2021

Even though I think it makes sense, if you think about it,


JavaScript SEO Q&A September 1st, 2021

95% of the hesitation and confusion around the testing


JavaScript SEO Q&A September 1st, 2021

tools is people assuming they're different from what actually


JavaScript SEO Q&A September 1st, 2021

happens in real life when they are not asterisked.


JavaScript SEO Q&A September 1st, 2021

I mean, they are slightly different.


JavaScript SEO Q&A September 1st, 2021

We know that.


JavaScript SEO Q&A September 1st, 2021

But yeah, I see where they're coming from.


JavaScript SEO Q&A September 1st, 2021

And that's why I think it's unlikely that we'll ever


JavaScript SEO Q&A September 1st, 2021

get a way to distinguish between testing


JavaScript SEO Q&A September 1st, 2021

tools and the real crawler.


JavaScript SEO Q&A September 1st, 2021

AUDIENCE MEMBER 1 It would be awesome if they could limit it


JavaScript SEO Q&A September 1st, 2021

maybe to a range of IP addresses, although in itself,


JavaScript SEO Q&A September 1st, 2021

that is something different.


JavaScript SEO Q&A September 1st, 2021

Because quite often, we get people trying to end


JavaScript SEO Q&A September 1st, 2021

Grok and stuff.


JavaScript SEO Q&A September 1st, 2021

And the performance on this page is terrible.


JavaScript SEO Q&A September 1st, 2021

It's OK.


JavaScript SEO Q&A September 1st, 2021

And they just want to be able to let the testing tools only,


JavaScript SEO Q&A September 1st, 2021

obviously not real, go into the thing.


JavaScript SEO Q&A September 1st, 2021

And maybe on the other side of that, although once again,


JavaScript SEO Q&A September 1st, 2021

ignore something like robot text just so it could get through.


JavaScript SEO Q&A September 1st, 2021

But once again, then that's a different experience.


JavaScript SEO Q&A September 1st, 2021

It's always bad.


JavaScript SEO Q&A September 1st, 2021

AUDIENCE MEMBER 2 Exactly.


JavaScript SEO Q&A September 1st, 2021

I see use cases like this and others


JavaScript SEO Q&A September 1st, 2021

where I'm like, it might make sense to do that.


JavaScript SEO Q&A September 1st, 2021

But I don't see much motivation coming from the team


JavaScript SEO Q&A September 1st, 2021

to actually do these things.


JavaScript SEO Q&A September 1st, 2021

I'm not going to promise that I can place it with the team


JavaScript SEO Q&A September 1st, 2021

again, but I have a feeling that they'll say like, no,


JavaScript SEO Q&A September 1st, 2021

because we want it to be as close to the.


JavaScript SEO Q&A September 1st, 2021

And that's the other thing, because it is


JavaScript SEO Q&A September 1st, 2021

so close to the real thing.


JavaScript SEO Q&A September 1st, 2021

It just basically uses roughly the same parameters.


JavaScript SEO Q&A September 1st, 2021

I just explained that there is a parameter drift from the view


JavaScript SEO Q&A September 1st, 2021

port expansion.


JavaScript SEO Q&A September 1st, 2021

And thus, we can't limit it to an IP range or something,


JavaScript SEO Q&A September 1st, 2021

because it's just using the Googlebot infrastructure


JavaScript SEO Q&A September 1st, 2021

saying like, hi, as far as you are aware,


JavaScript SEO Q&A September 1st, 2021

I'm just a regular request.


JavaScript SEO Q&A September 1st, 2021

And yeah, which is unfortunate, I'm aware.


JavaScript SEO Q&A September 1st, 2021

But I have a feeling that we'll be stuck


JavaScript SEO Q&A September 1st, 2021

with that for a long time.


JavaScript SEO Q&A September 1st, 2021

AUDIENCE MEMBER 3 Yeah, I mean, as long as you don't fix the recites


JavaScript SEO Q&A September 1st, 2021

stuff on mobile, I'm already using an event listener


JavaScript SEO Q&A September 1st, 2021

on the recites of the window.


JavaScript SEO Q&A September 1st, 2021

So I know when it's Google, and it's not Google.


JavaScript SEO Q&A September 1st, 2021

And there are also other small things like images.


JavaScript SEO Q&A September 1st, 2021

You know that Googlebot is not downloading images,


JavaScript SEO Q&A September 1st, 2021

but you know that in the rendering test,


JavaScript SEO Q&A September 1st, 2021

they are downloading images.


JavaScript SEO Q&A September 1st, 2021

So there are a few ways to understand this.


JavaScript SEO Q&A September 1st, 2021

But yeah, I don't want to give people too many ideas.


JavaScript SEO Q&A September 1st, 2021

I love this.


JavaScript SEO Q&A September 1st, 2021

I love this so much.


JavaScript SEO Q&A September 1st, 2021

This is great.


JavaScript SEO Q&A September 1st, 2021

Thank you so much.


JavaScript SEO Q&A September 1st, 2021

There are not very many people who


JavaScript SEO Q&A September 1st, 2021

are paying attention to the details like we do.


JavaScript SEO Q&A September 1st, 2021

And I really appreciate that.


JavaScript SEO Q&A September 1st, 2021

Trust me.


JavaScript SEO Q&A September 1st, 2021

I appreciate it.


JavaScript SEO Q&A September 1st, 2021

I like it.


JavaScript SEO Q&A September 1st, 2021

Awesome.


JavaScript SEO Q&A September 1st, 2021

Anyone else?


JavaScript SEO Q&A September 1st, 2021

Any questions?


JavaScript SEO Q&A September 1st, 2021

Anything that I can help with?


JavaScript SEO Q&A September 1st, 2021

No, maybe.


JavaScript SEO Q&A September 1st, 2021

OK.


JavaScript SEO Q&A September 1st, 2021

In which case, I'd like to say thank you


JavaScript SEO Q&A September 1st, 2021

so much, everyone who submitted questions


JavaScript SEO Q&A September 1st, 2021

and everyone who's watching the video when


JavaScript SEO Q&A September 1st, 2021

it was uploaded to YouTube.


JavaScript SEO Q&A September 1st, 2021

And also, thanks a lot for the lovely people


JavaScript SEO Q&A September 1st, 2021

here in the Hangouts.


JavaScript SEO Q&A September 1st, 2021

It has been a great pleasure.


JavaScript SEO Q&A September 1st, 2021

Always fun to do these.


JavaScript SEO Q&A September 1st, 2021

I miss these a lot.


JavaScript SEO Q&A September 1st, 2021

And they'll be back in the biweekly or fortnightly


JavaScript SEO Q&A September 1st, 2021

treatments.


JavaScript SEO Q&A September 1st, 2021

So thanks a lot for joining.


JavaScript SEO Q&A September 1st, 2021

Thanks a lot for all the questions.


JavaScript SEO Q&A September 1st, 2021

Have a great time.


JavaScript SEO Q&A September 1st, 2021

Stay safe.


JavaScript SEO Q&A September 1st, 2021

Stay healthy.


JavaScript SEO Q&A September 1st, 2021


English Google SEO office-hours from October 1, 2021

All right.


Welcome, everyone, to today's Google Search Central SEO


Office Hours Hangout.


My name is John Mueller.


I am a search advocate on the Search Relations team


here at Google in Switzerland.


And part of what we do are these Office Hours, where


people can jump in and ask their questions around search,


and we can try to find some answers.


And it looks like we have one person raising their hand.


So we'll start off with some live questions,


and then we'll go off and look at the submitted ones.


Daniel, do you want to go ahead?


Yeah, I can't believe I was the first one as well after that.


Mine's just a quick one, actually.


And I think I know the answer, but I just


wanted to double check.


So if we've got internal URLs that are relative URLs,


and they're pointing to the canonical,


and the canonical is the absolute URL,


is that fine for Google?


I think the reason why I ask is because in Screaming Frog,


we struggle to crawl relative URLs because it


has to have the domain.


It has to have the protocol in there.


And so we can only really crawl it


if we then change the configuration


to add XML Sitemaps.


So in the past, we've said, OK, if it's a relative URL,


regardless of pointing to a canonical, absolute,


you should make everything absolute.


And I just wanted to get your idea or opinion,


because I see this a lot on Shopify now.


So relative URLs are perfectly fine.


So also the mix of relative and absolute URLs,


that's perfectly fine.


I'm kind of surprised if you can't crawl through that,


because it sounds like it's more that the relative URLs might


be wrong, or there might be some mistake there.


But it's mainly products I've noticed.


Yeah, it's mainly products.


I've seen it a couple of times.


And it might be that we just need to change configuration,


but that was something that I'd noticed.


And I know I think it is maybe, I suppose, a bigger issue


if the canonical was the relative URL, because you don't


know what the canonical is.


But I just thought I'd just double-check


if it was the other way around, if there was an issue there.


Should be fine.


Awesome.


That was it for me.


Thank you.


Appreciate it.


Cool.


Michael.


Hi, so a couple of years ago, Google


noted that when it came to ranking results,


BERT would better understand and impact about 10%


of searches in the US.


So my question is twofold.


One is, has that percentage changed for BERT?


And you probably can anticipate this one.


What percentage is MUM expected to better understand and impact


searches?


I have no idea.


So I'm pretty sure that the percentage changed since then,


because everything is changing.


But I don't know if we have a fixed number that goes for BERT


or that goes for MUM.


MUM is, as far as I understand it,


is more like a multipurpose machine learning library


anyway.


So it's something that can be applied in lots


of different parts of search.


So it's not so much that you would isolate it


to just ranking, but rather you might


be able to use it for understanding things


like on a very fine-grained level.


And then that's kind of interwoven


in a lot of different kinds of search results.


But I don't think we have any fixed numbers.


I have to say, besides it sort of blowing my mind what


MUM can do, it seemed to me like it


was going to open more opportunities, actually,


for different products or queries to be discovered.


It seemed like it was just sort of exponentially going


to blow it out, what one could learn.


I don't know.


We'll see.


I think it's always tricky to look


at the marketing around machine learning algorithms,


because it's very easy to find, I don't know,


very exceptional examples.


But that doesn't mean that everything


is as flashy as that.


But in talking with some of the search quality folks,


they're really, really happy with the way


that these kind of machine learning models are working.


Thank you.


Cool.


Julian.


Hey, John.


Thank you for doing the office hours.


I have a question about a message in the Search Console


indexed, though blocked by robots.txt.


Of a certain page type, we have about 20,000 valid index pages,


which looks about right.


And however, we're alarmed that we


have a warning for about 40,000 pages that says,


indexed, though blocked by robots.


When we inspect these, it turns out


they are auxiliary pages that are linked to from a main page


that we want to rank for.


And these URLs, they're either not accessible by users


who are not logged in, or they're just kind


of thin informational pages.


So we did, indeed, add a disallow for these on robots.


And I guess Google must have hit these a while back


before we got a chance to disallow them,


because it knows about them.


And what's alarming is that Console


says that they're indexed, right?


So I wanted to ask you, does this actually


mean that they are?


And more importantly, are they counted


toward my site's quality?


Because when I try to pull them up using a site colon query,


I only get about one in five to show up.


So I'm a little confused.


If you could shed some light on that message, please.


Yeah.


So I would see that mostly as something


of a warning for a situation where you're not


aware of what's happening.


So if you're certain that these should be blocked by robots.txt,


then that's pretty much fine.


But if you weren't aware that they were blocked by robots.txt


and you actually wanted to have them indexed and ranking well,


then that's something where you could take action on.


So from that point of view, I wouldn't see this


as something that's alarming.


It's mostly just, well, Google found these pages,


and they're blocked by robots.txt.


And by the way, if you care about these pages,


you might want to take some action.


If you don't care about them, just leave them the way


they are.


And it's not going to count against your website.


It can happen that they appear in the search results,


but usually that's only for very artificial queries,


like a site query.


And the main reason it's mostly those kind of queries


is because for all of your normal queries that


are going to your pages, you almost certainly


have reasonable content that is actually indexable