it takes to the pipeline, the user agents


JavaScript SEO office hours January 13th, 2021

might differ a little bit.


But it's not per tool.


It is generally just the way that the crawling


infrastructure works.


And then they asked a follow-up question like,


but why is it that in one tool it looks slightly


different than any other?


Well, for one, if one tool shows you the desktop version,


like the URL inspection tool, if your site isn't mobile-first


yet, mobile-first indexing enabled yet,


you might see the desktop version


where the mobile-friendly test always


shows you the mobile version for the obvious reason


that it's tailored towards mobile.


The other thing is because of the way


that the infrastructure works, in the real indexing pipeline,


we heavily use caching.


But with a live test, like in the mobile-friendly test


or the URL inspection tool live test,


you don't want to see the cached version.


You want to see what would happen if we would fetch


freshly, which also means that we might run into timeouts.


That's not because of you.


That's because of us.


The way that our infrastructure is built


is not really meant to be doing live fetches, which we are then


effectively doing for the tools.


So sometimes some resources timeout,


and then the outcome looks different.


So even within the mobile-friendly test


or the URL inspection tool, running the test three times


might give you two different results or three


different results, depending on how the requests go through.


That's not something you need to worry about.


That's just something to keep in mind.


Sometimes you get false positives,


as in false things that would get you alarmed about, oh,


this part of the content is missing.


And it's just like, oh, actually, the resource


wasn't fetched this time.


You can always look at what we have in the index using


the Search Console.


If you use the URL inspection tool


and look at what we have in the crawl page,


you see the rendered HTML from what we actually


did when the infrastructure was running as it was intended,


which is a batch processing over a longer time.


And then you would see if the content is there or not.


All right.


Anyone else?


John, yes.


I do have a question, but it's actually


not JavaScript-related.


I'll try.


OK.


And I'll put my camera on to join you.


If we have internal links that need


to have tracking parameters in them,


more for political reasons than anything else at this point,


and obviously, it's an issue that we've been running into


and seeing the tracking parameters appearing even


in our Google's index, is there anything else


we can do on our end to help steer Google


in the right direction to avoid those pages?


You can.


So OK.


Are these pages that you definitely


want in the index just without the parameters,


or is it like you don't want them in the index whatsoever?


We have their category pages, like top-level pages.


We want them in the index.


We just don't want these.


Right.


You can use the URL parameter tool in Google Search Console.


That's one thing that you can do.


The other thing is you can canonicalize.


You can give us a canonical saying


like this without the weird parameters


is what we consider canonical.


And if that is a strong enough signal,


then we would pick that up and actually


pick that as the canonical.


So that would be what we would show in the search results


rather than the one with parameters.


And if you have it in the site map and in the canonical


that you specify without the parameters,


and we see like, OK, so the parameter one


that is linked here is the same content as the non-parameter


one, and then we would collapse them.


And then basically, it's very likely


that we would canonicalize the one without the parameters.


No guarantees made, but that's the approach that I would try.


OK.


That's what we have in place.


But I think our issue has been that our top navigation


links to pages with tracking.


And everything shared and social and everything


is always to these tracking pages.


Yeah, that's unfortunate.


And that's a strong signal to use them


as a canonical instead.


All right.


Well, thank you.


You're welcome.


So Miguel, Dave, and John were brave and asked questions.


Anyone else having a question?


You can also use the chat feature


if you don't feel comfortable voicing your question using


voice chat, or if you don't want to turn on your camera,


that's fine.


There's a chat in this lovely Hangout


that you can use to also ask questions.


Miguel, another question?


I have a question.


Oh, awesome.


Yes, please.


So hi, everyone.


My name's Ian.


So I know this is more centered towards JavaScript.


I'm not sure if that's the problem that I'm


facing with a particular website,


but it very well could be.


So let me give you some context.


I'm working on a website that we would consider


to be a large website.


It's got about 25 million pages on it.


They have been on Google for quite some time,


but they have never been able to get more than, say,


I want to say 700,000 pages in the index.


So the last thing we did was we compiled all the pages.


We made sitemaps.


We made a sitemap index.


We submitted that and then waited for a little bit.


And then eventually in the coverage report,


it showed up and it basically said 24 million of those pages


were excluded.


And then after I waited till the coverage report was updated,


which was a couple of days ago, and then I drilled down


into it, and it says that these are discovered,


not currently indexed.


So I wanted to see if, first of all,


if I could get any other information about what


that condition tends to mean.


And then following that, maybe just talking a little bit


about how this site is organized,


because it does use JavaScript in a lot of different places.


And maybe it's because of the use of JavaScript


that it's got all these pages being excluded.


Right.


First things first, it's not very likely


that JavaScript is the reason, but I'll


come to what might be the unlikely thing


with JavaScript in a moment.


Discovered currently not indexed means that we have seen it.


We might not have gotten around to crawling it.


That can have various reasons.


Like it can be that we have never actually looked


at the page yet, because we didn't get around to it.


Why we haven't called them?


Well, there's plenty of reasons for that one in itself.


It could just be that your server is responding


in a way that makes us believe that we shouldn't be requesting


too many pages in one go.


It might be that we are predicting,


based on the pages that we have in the index,


that we don't give it as much priority for whatever reason.


Turns out the web is really, really big.


So we have to start somewhere.


And that means that some other pages


might get the attention later.


It might also be that we have crawled them.


That would usually say, crawl but not indexed.


So discovered usually just means we are aware


that this page exists.


We haven't visited it yet.


We haven't gotten the content yet.


And that's exactly why I don't think


JavaScript is your problem.


Because if we have never crawled it,


that means we have never even looked at the HTML.


So it doesn't really matter if there's


JavaScript involved or not if you didn't get around


to these pages yet.


Could it be that we have, or should it turn out,


that we have crawled them but not indexed them,


then that is maybe a JavaScript issue in one specific case,


which is that the JavaScript somehow


does something that prevents us from seeing the content.


That can be verified or falsified


by putting the URL into the URL inspection tool


and running a live test where you get


the crawl and the render.


And then you see if the content in the rendered HTML


looks like what you would expect.


If the rendered HTML has all the content that you would expect


then it's not JavaScript's fault.


Because JavaScript is only involved in the rendering phase.


Once we have the content, JavaScript


doesn't really matter anymore.


If you run into the situation of crawled not indexed,


this might also just mean that we have found the content not


to be so relevant that we would consider including it


in the index.


Because we don't include everything in the index.


Not every page goes into our index.


That's just the reality of it.


Because it turns out the cloud also


has a limited amount of storage.


So that's something that you can then only figure out


what content do you really care for


and then work on these pages specifically to improve them.


OK.


Yeah, because I've sampled pages from the 24 million


and just put them in the inspection tool.


And they came back fine.


So yeah, you're probably right, it's not a JavaScript issue.


Some of these pages, yes, are thin.


It's a website that's devoted to legal documents.


So yeah, I mean, we have a bunch of pages


that list all the companies that are on a case, for instance.


And then we have pages for dockets.


We have pages that are just big, long lists of PDFs.


It's sort of like a court listener,


which is kind of technically, I don't want to say a competitor,


but somebody in the same space.


And so I guess we're just trying to figure out


how court listener has millions of its pages already indexed


and why it's just taking so long for us


to get our stuff indexed.


So I mean, like I said, the 24 million just


popped up on coverage about a month ago.


It's slowly going through 10,000 pages every day.


It goes up like 10,000 into the green.


So I'm wondering if it's just, like you said,


taking its time.


And if that's the case, my only other question


is, because this website, like I mentioned before,


because this website has had pages indexed in Google


for quite some time, is it possible


that Google kind of just said, this entire website really


isn't too important and just started assigning a very


small crawl budget to it?


And if so, is there a way to tell Google that, no, we're


really taking this website seriously,


it's mobile-friendly now, et cetera, et cetera?


And can you please give it more of a budget?


That is so obviously we can't just give everyone


the opportunity to say, I'm serious about this.


Give me more budget.


Because then everyone would do that,


which is exactly the problem we have with the priority


attribute in site maps.


There used to be a priority attribute.


And then areas is like, this is the first part.


All these 25 million pages are first priority,


which means none of them are.


So that's not how that works.


But what you can do is, in Search Console,


you can have a look at the crawl statistics.


If you go into Settings, Crawl Stats,


you see how much we crawled.


And you also should see a trend line.


And if the page has been lingering for a while


and we have seen like, oh, there's not that much content


here, so we don't have to assign it


a large chunk of crawl budget, then you


can try to update your site map XML file to say,


hey, all of these pages have updated.


Would you like to come back and have a look?


You can try to request indexing on the pages


that you care most for.


That's not really a thing that scales well.


But that's a way to at least make


sure that some of the more important items


are getting into it.


And then over time, we'll figure out, oh, yeah, this


is actually useful, good content.


And then the crawling will very likely also increase.


Given that there are no server issues,


that would suggest to us that we should probably not


be crawling as much.


Again, the Crawl Stats report is your friend here telling you,


like, oh, there's issues with, I don't know,


like 500s or 504s or 502s or 400s or 401s


or something like that.


404s are not really a problem.


That just tells us that the link does not go to where


it should go.


And that's not an issue per se unless, obviously, it's


a case where the URL is not a 404 and you want that included.


But yeah, you can have a look at what does the Google


crawler see?


What does the Google crawler try to accomplish?


And then you can nudge it with site maps


and probably also nudge it with keeping your important content


well-linked within the pages and request them for indexing


to get them crawled again.


SPEAKER 1 OK.


No, no.


Thank you.


I appreciate that.


Yeah, I'm looking at the budget right now.


Yeah, it doesn't have any problems with responses.


Server host SaaS is green.


It's interesting.


When the $24 million appeared on the excluded report,


there is a spike in traffic for a total call request.


It went up to about $57,000 one day, $58,000.


And then it just dropped off again.


So I guess it's kind of like maybe Google is aware of it


and is just processing it slowly.


SPEAKER 2 Yeah, it takes time to process the entire web.


Also, out of curiosity, as you are already


in the cross-sats report, what's the average response time?


SPEAKER 1 Average response.


SPEAKER 2 If you go up, you see total request,


total download size, and average response time.


That's the interesting one.


SPEAKER 1 Yeah, it says 888 milliseconds.


OK, so 0.8 seconds, that's OK, I would say.


That's not a bad metric.


Good, OK.


OK, the quicker the better.


Yeah, thank you.


You're welcome.


Very happy to help.


Ian, was it, right?


Yes.


Awesome, thank you very much, Ian.


All right.


And also, actually, Ian, very nice of you


to use the Raise Hand feature.


I really like when people do that.


I should encourage them more often.


Yeah, it's really, really cool, because that makes it easier.


People usually don't talk, because they


worry about talking over someone else.


And then the Raise Hand feature is a really nice way


to counteract that.


And I see that Kevin has raised his hand.


Hey, Martin, thank you very much for getting the Google Meet


and doing these weekly.


I know it's a lot just to really get this stuff going.


And yeah, anyways, so to my question,


I did post it in the forum.


And I'm a Google Street View photographer


for a lot of people.


And I've been doing it on, go ahead, what was that?


Ah, OK.


No, it was like you were breaking up.


But I know that you asked the question about the 360 photos,


right?


Yes, so the question was, because I


was rereading what I had read.


And it doesn't really clarify either.


But with 360 photos on the Street View,


especially on their Google My Business listings,


and I don't know if you're the exact person to ask.


But I know they don't really have a Street View team.


But I thought you might have some clairvoyance on kind


of what my issue is.


I've had people tell me that metadata on Google Maps


is nothing with 360 photos, that it's


stripped once it goes there.


But then the vice versa, if I'm taking a 360 photo,


and then they'll hold someone's showroom floor,


especially around their tables or chairs or something,


and I'm creating a virtual tour for their website,


would including all of the actual items inside the photo,


obviously the ones that are very easily seen,


is it worth putting the item's description or the item's names


inside the 360 photo?


I'm so sorry.


Thanks for the very elaborate explanation and question.


But I'm the wrong person, unfortunately.


I really only know about Google Search.


I have no idea about Google My Business or Street View,


for that matter.


And to be honest, I have been taking 360 pictures just out


of fun and curiosity because I have a 360-degree camera.


And so for people that I know who happen to have a shop,


I sometimes wander by and take a picture for them


so that they get a little bit more visibility


in Google Maps.


But that's a hobby thing.


And I don't know who is the right person to address this.


I wonder if there is a support forum for Google My Business.


I'm pretty sure.


It's both a part of the local guide community,


but unfortunately, they dropped that and are saying,


well, the Street View program is getting completely reworked,


at least from my understanding.


Right.


Other than that, they really haven't clarified it.


There is Facebook pages, but it's all community brand.


I don't think anyone from actual Google or the even Maps team


is really in there.


So if you go to the Ask the Community thing,


like the forum kind of situation that they have in Google My


Business and the Google My Business Help, there are,


yes, it's a community-driven effort,


but there are community managers from Google who look


into these questions as well.


And community members of the rank of product expert


can escalate this to the community managers.


And then it should get routed in the right direction.


The alternative is you can also use the Send Feedback


about our Help Sender button in the Google My Business Help


and ask a question.


I have asked a few questions in the GMB help desk,


because that's kind of what my business is.


I usually do 360 photos for people,


kind of quarterly with the season.


And I do the Google My Business management for them.


Right.


And so usually fixing people's mapping addresses or just


kind of affiliate it.


OK, your map point's not over there.


It's over here.


All those kind of things.


A lot of what a local guide does.


Doing it for a variety of businesses specifically.


Yeah, I'm very sorry that I'm the wrong person


to ask, unfortunately.


No, that's all right.


I didn't know if you knew anything


about the virtual tours on it as well for a website.


So if you're implementing the photos inside their website,


if that would change anything or even help with their SEO.


It doesn't really do anything on the search side


as far as I'm aware.


OK.


All right.


Thank you.


Sorry.


I'm trying my best, Kevin.


Thanks for asking.


How dare you?


No, you're good.


Awesome.


All right, Miguel raised his hand.


Hi, yes.


My colleague was really intrigued by your response


and wanted to know if you believe


that removing the links that are not accessible or visible


to users is a good practice.


I mean, if they are not visible to users, yes.


If they are not accessible to users, yes.


I would also probably remove them.


I wouldn't necessarily say it's a best practice,


but I don't think we value these links as much as people think.


Because if users can't reasonably reach them,


then I'm not so sure that Google would necessarily


value them highly.


That gets us into a really tricky world of stuff,


like what does accessible mean and how


does Google determine that.


And I can't really answer that one.


That's fine.


But I think only having links that


are meant for users to actually navigate is a good approach.


Yeah.


Thank you.


And if I might be so bold, in relation to the JavaScript,


are links loaded only upon user action using JavaScript called?


Do those count as part of the link quota on the page


if they only load after the JavaScript action?


There's no link quota.


Not link quota, but as part of the crawl.


No, because if they only exist after a user interaction,


we can't really see them.


We are not interacting with the page.


So let's say you click on something,


and then JavaScript generates a bunch of links


that haven't been there before.


Googlebot would not really see them


unless they happen to already be in the source code,


and they're just hidden, and then they're only made visible.


Because then we would see them in the HTML,


and then we would also follow them very likely.


So like an ARIA hidden versus ARIA hidden false?


Yeah, that wouldn't help, because then they're still


in the source code.


And we extract the links before we even render the first time,


and then we extract them again after we render.


So we would probably find them before we render,


because they are in the DOM.


Great, thank you.


You're welcome.


We might choose not to follow them,


but we will see them at least.


OK, excellent.


Anyone else?


We have seven more minutes before I need to wrap this up.


Feel free to use the chat or the Raise Hand feature.


I see that there's lots of love for the GMB Office Hours idea.


I'll forward that to the team.


That's all I can do.


OK.


Actually, I'm not sure, but Alan Kent from our team


works a lot with the Google My Business folks.


So I think it's AJ Kent on Twitter, AJ Kent 99


or something on Twitter, if I remember correctly.


Let me see if I can find him on Twitter, AJ Kent 99.


I think it is, but I might be wrong.


And then I'm like, oh, yeah, no, that's not him.


Oh, god.


OK, so I nearly threw someone under the bus who has nothing


to do with this, A Kent 99.


I'll post it in the chat as well.


That gentleman is from my team, and he works a bit


with the GMB team as well.


So you can voice the desire for GMB Office Hours


and that in his direction.


Or ask him, because he does the e-commerce SEO Office


Hours every now and then, or used to do them.


I'm not sure if he still does them.


So you can pop by on his Twitter account


and ask him if he still does the e-commerce Office Hours


and what he thinks about GMB Office Hours specifically.


That's something that I can at least do.


SPEAKER 1 Thank you.


I will definitely follow up with Alan as well.


MARTIN SPLITT I will definitely follow up with Alan as well.


MARTIN SPLITT You're welcome.


Happy to help.


All right, as no one else is raising their hands


or using the chat to post questions,


I would say thank you.


So no, Anastasia, it is not.


You can use the Office Hours here right now


to ask a question, even using chat.


That is fine.


You can use the Webmaster Forum, or you can use public Twitter,


not direct messages.


I cannot provide private support.


It has to go over the public channels


for the obvious reasons that we don't


want to give preferential support to any one party.


Right.


So unless Anastasia does have a question for a chance,


then I'll happily wait for that to arrive.


If that's not the case, then I would say thank you very, very


much for joining the Office Hours.


It has been a great pleasure answering all the questions


from YouTube and here in person.


I hope that you stay safe and healthy and have a great time.


And hope to see you again in, I think, two weeks.


All right.


Bye-bye.


Always the awkward fumbling for stopping the recording.


Google Search News (July ‘20) - Web Stories, Page Experience Benchmark, and more

Hi, everyone, and welcome back to the Google Search News


series.


I hope life is treating you reasonably well wherever


you are.


I'm your host today, John Mueller, here from home again


in reasonably sunny Switzerland.


With this show, we want to give you


a regular summary of what's been happening around Google Search,


specifically for webmasters, publishers, and SEOs.


If you find these useful, which I hope you do,


and if you'd like to stay up to date,


make sure to subscribe to the channel.


It's been a bit since our last update.


It turns out that times like these can be pretty challenging.


So regardless of where you are and of how the situation is


in your region, I hope that you, your family, friends,


and co-workers are all doing well.


And now, without further ado, let's move on


to the recent Google Search News.


Today, we'll take a look at Web Stories, an upcoming ranking


factor, structured data updates, and more.


Keep playing the video.


Let's check it out.


Let's start with Web Stories, an exciting and somewhat new


approach to content on the web.


The Web Stories format is a neat way


to create high-quality, bite-sized informational content


directly on your own website.


It's easy for users to view them and to move on to your website.


Like other parts of your website,


they're under your control.


You can monetize them, and they can show up in Search.


The format is built on AMP, so they're normal HTML pages too.


While Web Stories themselves aren't new,


they're now actively showing in more and more places


like Google Search, Google Images, and Discover.


If you use WordPress, you'll be happy to hear


there's a plugin in development which


makes it easier to create these directly in your own CMS.


The plugin has just been launched


as an early beta test.


It's still a bit rough, but it's a fantastic way


to give stories an early spin.


I've added the links in the description


if you're curious to find out more.


I find it fascinating to see this new take on public web


content, and I'm sure we'll have more updates


on this topic in the future.


Last episode, we talked about the Core Web Vitals,


a set of foundational metrics for website developers


to help build and improve online user experience.


Now, we've announced that they will


be a part of the Page Experience Benchmark, a future Google


Search ranking factor.


The Page Experience Benchmark uses the Core Web Vitals,


in other words, metrics for speed, interactivity,


and stability.


It also uses other search signals,


such as mobile friendliness, HTTPS security,


and the intrusive interstitial guidelines


to provide a holistic picture.


We've now updated various popular developer tools,


like Lighthouse and PageSpeed Insights,


to show these metrics, making it easier


to spot and improve web pages.


The thresholds we use for the Page Experience Benchmark


are based on high-quality user experience


and verified to be achievable with data from the Chrome User


Experience Report.


It's not yet decided when we'll start using these elements


as a ranking factor, but we'll let


you know at least six months before we


start using them in search.


If you want to check your site's performance already,


an overview graph is waiting for you in Google Search Console,


so check that out.


For structured data, we recently announced some new reports


and testable elements, as well as


that we're deprecating one of the two testing tools.


Google currently maintains two testing tools


for structured data.


One is a general tool that shows information


about all structured data.


We call it the Structured Data Testing Tool.


The other is specifically to diagnose issues


about what we show in the search results, which is


called the Rich Results Test.


In the past, these differences have led to confusion,


as there are many ways to implement structured data


without it being shown in search.


As people usually come to our testing tools


to improve in Google Search, we decided


to focus on the Rich Results Test.


So we're planning to deprecate the general Structured Data


Testing Tool.


Since structured data has been used on the web for many years


now, there are also general third-party testing tools


available, which can help folks who want a more general test.


And as always, we continue to listen to feedback


from our users and use that feedback as input


for our future plans.


So if there's something you'd like


to see added to the Rich Results Test, please let us know.


In more structured data news, we're


now showing special announcement, licensable


images, and AMP article markup in the Rich Results Test.


And in Search Console, we've added reports for recipe


and guided recipe markup, too.


We talked about mobile-first indexing last time.


An update there is that we recently


announced a change in the timeline,


extending it to the end of March 2021.


This will give websites a bit more time,


should they need to adjust parts of their pages


for mobile-first indexing.


Together with that, we published a number of best practices


to watch out for.


The link to that is in the description below.


In our search documentation, we've


expanded the guides for JavaScript sites,


adding information about links, the history API, URL fragments,


and 404 pages.


If you make JavaScript-based sites,


it's worth checking that out.


But wait, there's more.


While we've been working from home for some time,


we're still publishing a number of videos


that we recorded from before social distancing became


common.


For example, there are some cool episodes of SEO myth-busting


with fantastic guests being published,


such as recent ones about crawl budget and page speed.


Additionally, we've continued to expand our Search Off the Record


podcast, where we talk a bit more about what happens


behind the scenes at Google.


And since we're no longer going to physical events,


feel free to come and visit us virtually,


either in our regular SEO office hours


or at a number of virtual events run by third parties.


No mask is required for virtual events.


There's always more happening around search


than we can cover here.


If you'd like to stay up to date,


subscribe to this channel, check our blog,


and follow us on Twitter.


And that's all for now, folks.


I hope things are well for you and your family,


regardless of where you are.


Thank you for joining us here, and I look forward


to seeing you all again in one of the future episodes


of Google Search News.


If you have any questions or comments,


feel free to add them right here or to reach out to us


on Twitter.


And now, on to the weather.


I know, I know.


What good is a weather report if you can't go outside?


Well, hopefully, that's not the case where you are.


And it's nice to be able to escape outside from time


to time.


At any rate, I hope the information


in this video and the other insights that we publish


help you to make fantastic websites which


shine regardless of the current SEO weather.


English Google Webmaster Central office-hours from May 15, 2020

All right, welcome, everyone, to today's Webmaster Central


Office Hours Hangout.


My name is John Mueller.


I am a webmaster trends analyst at Google in Switzerland.


And part of what we do are these Office Hour Hangouts,


where folks can join in and ask their questions


around the website and web search,


and we can try to find answers for them.


A bunch of stuff was submitted on YouTube already.


Lots of questions, it looks like.


But if any of you want to get started


with the first question from here, feel free to jump on in.


If not.


SPEAKER 2


Hi, John.


Yeah, go for it.


Yeah, so one question was very much like, sometimes it happens.


However, it is not guideline.


So while we were dealing with our tech team, by mistake,


some redirection were made 404, or some live pages


which we wanted to remove.


Right?


So they were made 404.


Sorry, sorry, let me make it clear.


We wanted to redirect them.


We were doing site migration.


So we were planning to redirect those pages to new pages.


But because of tech team limitations,


they just made them 404.


And while the pages were 404, they


were just working on resolving the issue.


And I am sure that maybe Google would have just crawled them


and found them 404.


It happens sometimes.


In this case, when Google has crawled them and just dropped


them from indexing as 404, and then we


are setting up redirect to page move to new page,


what should we think?


The page is completely, I mean, content completely moved.


It was 301 genuine case, but the page was 404 for some time.


And then setting up redirect, how Google just


understands this kind of thing?


Yeah.


So I think, first of all, you're not alone.


I think pretty much everyone who has moved a few websites


has run into this, where you wanted to set up redirect,


and the team just essentially turns off the old website


instead of setting up redirect.


That happens every now and then.


And from our point of view, we will recrawl those URLs,


and we will see the redirects, and then we


will process the redirects.


And that should be more or less OK.


It's sometimes a question of how long the pages were 404


before you set up the redirect again.


And that's more a matter of how long it takes for everything


to be picked up again.


But in general, that happens.


And our systems have to live with it as well


and figure out how to catch that and clean it up again.


So in this case, in new page ranking,


the ranking may fluctuate.


What should we assume?


I think what will happen is, so usually what happens


is those pages drop out of the index


because of the 404.


And then when we start seeing the redirects again,


we can pick them up, and we can process them normally again.


So it's more a matter of, like, things


will be fluctuating with a site move in general.


And if you do a mistake like this,


then it fluctuates a lot more.


But it should catch up, and it should settle down normally


again.


So it means we can say that with the time the ranking will


be a new page.


There is nothing to worry.


Yeah.


I mean, it really depends on how long it was like that.


If you're talking about a few days,


I don't see much of a problem.


If you're talking about a few months,


then obviously that's a significant amount of time


where our systems may have thought, well,


this page is probably gone forever.


But it sounds like a couple of days,


then you notice the technical problem, you fixed it,


and then it's OK now.


So I don't see a big problem with that.


OK.


Cool.


Thanks.


Sure.


All right.


Any other questions before we jump into the submitted ones?


No?


Feel free to jump in in between if something comes up,


or we'll probably have time towards the end


again to do more open questions as well.


The first one is kind of an international sites question.


Googlebot usually comes to our website from the US.


At the moment, we don't fully block all of the US states.


We use alternative methods.


But let's say we have to block content for the US IPs.


Then if Googlebot comes from a blocked US state,


we have only two, in my opinion, bad options.


The first one, display a substituting


log-compliant content instead of displaying the full content


and risk that this substituting content will be indexed


for all users around the globe, or B,


identify Googlebot by agent header


and display the full content.


However, we risk cloaking as a user with the same IP


would get different content based on only the user agent.


What else could we do?


So you're right.


This is a tricky situation.


For the most part, we do crawl from the US.


So with regards to our webmaster guidelines,


our guideline is pretty clear that users


from the same location as the Googlebot


should have the same experience as Googlebot.


So in other words, when Google is crawling from the US,


and if you need to block all users from the US,


then you would also need to block Googlebot.


And in practice, what this means is


that your website or this content


will not be indexable because you're blocking Google


from actually accessing it.


And the two options that you mentioned


there are essentially the only ones


that we see from our side.


Essentially, displaying content that you're


allowed to display in the US, I think that's probably


the cleanest approach.


That's the approach that we usually


recommend because that makes sure that you have one


version of the content that's available to users


in that location.


They can go there.


If they want to sign up, and then the sign up


is blocked because they're from the wrong location,


then that's kind of something that's


more of a usability issue on your side.


From our point of view, that's OK.


From an indexing point of view, you're


correct that we would only be indexing this publicly


available content.


The second option, the one that you mentioned there,


is essentially cloaking to Googlebot,


which is not what we would recommend doing


because that would be cloaking.


That would be something where the web spam team would be


pretty unhappy about seeing that.


So essentially, if you're in the situation


where you need to block the US and everyone else can access


the page, then it's always going to be very hard.


It's easier the other way around or with other countries.


So if Googlebot is crawling from the US


and you need to block all users from Switzerland


because these Swiss people, they're, I don't know,


you don't like them, if you needed to do that,


then from Google's point of view,


that would be totally a non-issue


because we would crawl from the US.


We would see the content that users in the US


would be able to see.


We'd be able to index everything.


And it's just Swiss users, when they come to your site,


would be blocked.


So that's kind of the easier situation


is if it's not from a location where Googlebot is crawling.


But otherwise, that's kind of a tough situation


where you have to deal with that.


SPEAKER 2 Hi, John.


I'm sorry.


Can I have a follow-up?


JOHN MUELLER Sure.


SPEAKER 2 Yeah, the thing is that it doesn't concern, like,


whole US.


It concerns only a few states in the US.


So is there any option we can do, like, state-specific?


Or since it's in some states, we need to block users from all US?


JOHN MUELLER We don't specify the exact level


when it comes to kind of showing Googlebot the same content


as users from the same region would be.


So from that point of view, if you


can recognize where Googlebot's IP addresses are coming from,


which states, then you're welcome to kind of do it


on that level.


Most of the situations that I've run across


tend to be more on a country level, where


you need to block the whole country for maybe


legal reasons.


And that's where essentially, well, the US


is the whole country, and all of the crawling comes from the US,


or most of the crawling comes from the US.


So you would be blocking the whole country.


If there is differences on a per-state level,


and you can recognize where the IP addresses are coming from


to some degree of certainty, then that's fine, too.


SPEAKER 2 OK, so the solution would be to try to track


IP addresses of Googlebot.


And if they are coming from the same state,


we are not allowed to show the content,


then block the content for those states, right?


Yeah, yeah.


I suspect it will be kind of tricky,


because the mapping of the IP addresses


to a specific location is kind of an art, not really a science.


SPEAKER 2 Yeah, I can imagine.


But I would give it a try.


If that's an option that you have,


that you can do it on a state level,


then maybe that's a solution.


SPEAKER 3 OK, thank you.


SPEAKER 2 Sure.


SPEAKER 4 Hey, good morning, John.


How's it going?


Hi.


OK, I want to ask, can I backdate my posts?


Because I did a WordPress post to backdate.


Then I realized the search console


is showing excluded your pages.


I don't know why, because I have 5.4K pages, 5.4K posts,


but it is scrolling just 500.


And others 5K is showing excluded, currently not


indexed, from one month.


Why is that?


That can be a lot of reasons.


It's really hard to say.


It sounds like, from a technical point of view,


we can probably access your page as well.


So from that point of view, it's probably not a technical issue,


and more a matter of maybe the quality in general,


where our systems, they tend to try to focus


on higher quality content.


And if our systems recognize that, in general,


on this website, maybe the content is questionable,


then generally they wouldn't index all content.


But even, I think, outside of the quality questions,


it's completely normal that we don't index


all content on a website.


It can be because I backdated all the posts, or it's normal?


Because I backdated my posts.


I don't think backdating would be necessarily


a sign of low quality content.


But I don't know.


SPEAKER 2


I don't think it has any effect at all.


If these are nofollow links, and they're


coming to your site, then we don't pass any signals there,


essentially.


So that has no effect.


SPEAKER 3


OK, thank you.


Thank you.


SPEAKER 4


John, very small follow up on this.


So just wanted to confirm with you,


last updated in the page is really


considered by Googlebot.


I don't think so.


I think Google always gives the recommendation


that an XML sitemap last modified is recognized.


But what if the person does not have any XML sitemap


and just updates on the post that last updated two hours


back or three hours back?


Is it really understood by Google that, yeah,


the content is changed because this is mentioned on the page?


SPEAKER 2


Sometimes, yeah.


Yeah.


So we use the date in the sitemap


primarily to determine when we need to crawl the page.


But after we've crawled the page,


we look at the content primarily.


We see what the content says.


If there's maybe a date on the content,


if the date is marked up with structured data,


then all of these things are elements


that we would look at and we would try to pick up on


and use that appropriately for search.


OK.


All right.


Any pitfalls for using loading equals lazy for images


inside an article?


No, not from my point of view.


With regards to SEO, this is a really simple way


to add lazy loading to your images


within a page in the sense that these are normal image


tags that you keep in your HTML.


So when any search engine comes along,


they see the image tags, and they


can act on those image tags and index the images directly


like that.


However, for users, it's for the browsers that support it,


which I think are Chrome and maybe Firefox and Safari,


I'm not 100% sure.


For those browsers that recognize


this is a form of lazy loading, they


would load those images when that part of a page


becomes into the viewport.


So it makes it a lot faster to load those pages.


So from that point of view, I would strongly


recommend using this kind of lazy loading


or maybe another kind of lazy loading


if you have other preferences specifically for images,


because it can significantly increase the speed


of your pages being loaded.


In an ideal situation, we use hreflang


to make sure people from other countries


does not land on the EN US page.


But if we don't have an alternative page for,


say, English Canada, since the product doesn't sell


in Canada, should we just let the user land on the English US


page?


Or how should we handle this?


So essentially, that's up to you.


So there are two general trains of thought here.


On the one hand, you can have a page that says,


this product is not available here.


And you could just use hreflang to mark


that one as that version.


On the other hand, you can just say, well, this is the US page.


And when users try to put it in their cart,


for example, if it's a product, then you can say,


well, this product is unfortunately not


being sold in your country.


So those are essentially the approaches there.


In general, when it comes to hreflang,


I strongly recommend having fewer pages rather than


more pages.


So instead of just making landing pages


for every country and language version,


if you can simplify that to kind of just the languages


that you actually need, then that makes it a lot easier.


Then that might result in a situation


here where you just have one English landing page.


And depending on the country, the location of the user,


then you can respond to that individually


in a part of the page where you say, well, actually, we


don't sell this to you, but here's all of the information.


That also makes it a little bit easier


for users who are either recognized


as being in the wrong country or who might be traveling.


So they can still get the information that they need


and kind of make a call based on that.


We had a page that listed all of the products on our site.


We retired that page about two years ago,


and it currently returns 404.


It contained a lot of incorrect or missing schema.org


microdata for products.


Our problem has two parts.


We have a lot of search console errors


that are coming from crawls of this page that happened


around two to four years ago.


We can't issue requests for revalidation.


The UI currently shows validation passed


with no button to validate fix.


This is also compounded by the fact


that our site used to route users around the site


with a website.com question mark main page equals


product type of URL.


We've since changed that to a standard URL structure.


And in 99% of the cases, Google search shows those results.


But since the old style links redirect to working pages,


search console seems to treat them as discrete pages.


This has also been treating Google Ads URL parameters


as discrete pages on top of that, so the list of errors


can get into the thousands.


I looked into the removals tool, but those


require that the URL returns 404 for part of the URL


before URL parameters, which is our website group.


Should we be worried that these errors are affecting


our ability to get rich content results in Google search


for products on this retired page?


Are we getting errors for?


Is it possible that it would cause


other penalties on our site?


Long question.


OK.


So first of all, if you remove that page


and you don't want that page to be indexed in search,


then it's correct to return 404 for those URLs.


And it's completely normal to see errors in Search Console


for that because essentially, you remove this page,


we try to look at it, and we're telling you, well,


we saw an error when we tried to look at it.


And from a technical point of view,


that's what you want Google to see.


You want Google to see that this page no longer exists


so that it can be dropped out of the index.


So from that point of view, it's totally OK


to have 404 errors in Search Console.


That's essentially a sign that you're doing things right,


that you're using the proper response codes,


and everything is set up correctly.


So from that point of view, I don't


think you need to do anything specific here.


If these pages no longer exist, keeping them 404


is perfectly fine.


You don't need to do anything with validation.


You don't need to use the content removal


tools, anything like that.


Essentially, they drop out of the index.


We start crawling them less frequently,


but we will still check them over time.


And there is absolutely no penalty,


no manual action associated with that, no demotion.


It's not a sign that your website is of lower quality


if you have 404 pages.


Can we use jump links for an internal linking page?


Will Google crawl these links?


So jump links, I think, I'm not sure what the technical term is


for those.


But essentially, you have the hash in the URL,


and it just jumps to a particular part of the page.


Yes, you can absolutely use those.


We don't crawl those links because we would see


the same content as a normal page.


It's just jumping to a different location on that page.


But we do recognize them, and sometimes we


show them in search as well.


If we feel that a user is searching for something that's


on a particular part of your page,


and you have a link pointing to that part of the page,


then maybe it makes sense to help the user go to that part


of the page immediately.


So that's something I would definitely use those.


Over the past few weeks, I've noticed steady increasing


number of backlinks in Search Console from two domains


to my website.


From these two domains, every single page on my website


is linked about two to three times,


accumulating more than 500 backlinks over the past four


to eight weeks.


Looking at the originating URLs in Search Console,


I saw that these originating URLs from two domains


are empty pages, the home page on these two domains


is simply an empty page.


My discovery of this issue correlates


with a 50% drop in traffic.


These backlinks don't show up in other SEO tools,


only in Search Console.


As these originating URLs are empty pages,


do you have any idea why they would show up


in Search Console as referring pages?


And is this a scenario where the disavow tool makes sense,


or does Google detect them as unnatural and will ignore them


as a ranking signal?


It's really hard to say what you're seeing here.


It's certainly possible that there are pages out there


that show an empty page to users,


and then they show a full page to Googlebot.


From a practical point of view, there's


essentially no real reason why to do that.


So it seems more like a technical mistake


rather than someone trying to maliciously do something


with your website.


So from that point of view, I would just ignore those pages.


From my side, if you're actually looking at those pages,


maybe if you look at them with the mobile-friendly test,


you can kind of see what Google IP addresses would see.


If you're not seeing any content there,


then I wouldn't worry about that.


I don't think this is something that you need to disavow.


It probably looks weird in the links report,


but I really wouldn't worry about this.


With regards to the drop in traffic


that you're seeing, from my point of view,


that would probably be unrelated to these links.


There's no real, I don't know, situation


where I could imagine that essentially empty pages would


be causing an issue with regards to links.


So I would just ignore that.


If you decide to put them into disavow file anyway,


which you can certainly do if you wanted to do that,


just keep in mind that this would not


affect how we show the data in Search Console.


So the links report would continue to show those.


I don't think there's any reason to use a disavow file


in this particular case.


So I would just leave them be.


My coverage report shows a valid page has


been decreasing since January.


I don't know what happened.


Crawl currently not indexed and discovered currently not indexed


is slowly rising.


Most pages can be indexed when checked in the URL inspection


tool, but they're all listed as excluded.


I checked the robots text and meta tag.


Everything appears normal.


Can you give me some suggestion about the situation?


Is this an impact of algorithm update?


Yeah, I don't know.


It's really hard to say.


In general, we tend not to index all pages on our website.


So from that point of view, that can be completely normal.


As I mentioned before, one of the more common situations


where if a page is technically indexable,


but we just don't want to index it,


a common situation is really that our algorithms are just


not sure about the overall quality of the website.


And in cases like that, we might crawl the URL.


We might look at the content and say, I don't know.


Or we might know about the URL, which would be like,


it's discovered, but we haven't indexed it,


which is kind of a sign, well, we


know about this URL on your website,


but we're not really 100% convinced about your website.


So maybe it's not yet worth our effort


to go off and try to index the content of that URL.


So in that regard, what I try to recommend,


I try to do here is take a step back and think


about your website overall.


And think about ways that you can


make sure to significantly increase


the quality of your website overall.


That might be by working on your content.


It might be by working on everything around your content.


Because when we look at a website,


we don't just look at the words in the primary content.


We essentially try to look at everything


similar to how a user might look at everything on a website.


So that's kind of the direction I would go there.


I know this is never an easy situation, because obviously,


if you've been working on your website for such a long time,


then it's your baby, and you don't


want to accept that maybe there are things that you


could be doing differently.


But sometimes it's really worth taking a hard look


at everything and thinking about ways


that you can really significantly ramp things up.


One approach that some people take


is to kind of prune back pages that they


find were lower quality, or combine those pages


into other pages and make the existing pages higher quality.


That's certainly something that you can do.


It's not the case that you need to delete pages on your website,


because there is a limited number of pages


that can be indexed.


But it's really just the pages that we find


and the pages that we index, we want to make sure that they're


of high quality so that it makes sense for us


to show them to users.


Sorry, John.


Sorry to interrupt.


I have a question.


We have a client.


They are a solar panel company.


They sell solar panels, solar batteries, and solar inverter.


So they rank well for the LG Solar Panel Sydney,


LG Solar Panel keyword.


Now, the page ranked for this keyword,


that is actually not brand page.


That is one of the LG Solar product page, actually.


So what we did, we actually replaced the content


on that page with the new content.


And we added all of our product name on that page.


So we probably make that page LG Solar Panel brand page.


And then the content we had previously,


we removed it to another page.


But when we did this, we saw the ranking of that page dropped.


And the new page we created with the old content,


that actually ranked for the LG Solar Panel keyword again.


So is there something wrong?


Is this because of the new algorithm update?


Or is this because we changed the content?


I wouldn't see this as something that is


specific to an algorithm change.


It sounds more like you've made some changes on your website,


and now things are ranking slightly differently.


So I would see that more as an effect of those changes


that you made within the website.


And if this is something that you made,


kind of the changes that you made fairly recently,


then sometimes it just takes a while for things to settle down,


especially if you're moving content


between different pages.


Maybe if you're redirecting pages or folding pages together,


combining things slightly differently.


That's all something that sometimes takes a bit of time


to settle down, where in the beginning, it'll be a bit,


like our algorithms are not quite sure what to do,


how these pages are relevant between each other.


And over time, when we see how things fall into place


within your website, then we can rank those a lot better.


And one more question, John, this is about image alt-tech.


So this is just a graph, image of the graph,


and shows the efficiency dropped to 83% within 25 years.


So this is what I put in my alt-tech and caption


of the image.


So it looks like a little bit bigger description


or bigger alt-tech.


Is it OK?


That sounds perfectly OK.


Yeah.


The alt attribute should be something


that describes the image and should


be describing, essentially, the image


in the context of your page.


So it's not just that you have to say, like,


this is on the image, but rather it's like,


this is on the image, and this is the reason


why it's included here.


It's like a graph showing the efficiency of whatever


your products are there.


And that's essentially fine.


With regards to search, the thing to keep in mind there


as well is that in image search, people might search slightly


differently.


So if you're seeing that you're creating content


that you think would work well in image search,


then it probably makes sense to think


about what kind of keywords people would


use to try to find that content.


And it's not something that you need to stuff those keywords


into the alt text, but make sure that you have them


on the page somewhere at least so that when someone goes


to Google Images and searches for those keywords,


then we can recognize that your page is a good landing


page for that.


And this image is a proper image that fits


to what they were asking for.


Thank you, John.


Sure.


Let me just.


I have one more question.


I have a platform we are creating now.


We didn't launch it yet.


If someone will use copyright image, can we like,


I want to create a boot to send auto reply.


It's a copyright image.


You can't publish this content.


I don't want to hire someone to do all of these things.


Is Google supporting any kind of this application or API


because I'm using Node.js or Express?


I want to make something like this, which will like,


I will send a request and the reply comes back from Google


or any other service, which will tell it's a copyright image


or it's not.


Then on the base of that, I will make decision,


what should I do?


I don't think we have any kind of service


like that from Google's side.


I suspect there are other services that


do something like this, but I don't know of any offhand.


Which one is doing this kind of work?


I don't know.


OK.


Yeah.


I will find.


Thank you.


Let me grab some more questions that were submitted


and then we'll have some more time for all of the live


questions as well.


I have a lot of 301 and 404 pages on my site.


What do I need to do with them?


That's perfectly fine.


You don't need to do anything.


You don't need to block them in any particular way.


404s are perfectly fine.


Redirects are also perfectly fine.


I'm working for a travel portal.


I'd like to understand, due to COVID-19,


travel has taken a dip.


Will there be an organic ranking drop as well?


I don't think that would be related.


I mean, obviously, within the whole ecosystem,


things are all going in all kinds of weird directions.


So it's really hard to say what will be happening.


But it's not the case that we would say suddenly


travel portals should be ranked lower just because coronavirus


happens.


It might be more the case that things are just generally


different with coronavirus.


And maybe people aren't searching as much


for travel websites at the moment.


But maybe they'll search even more when things pick up again.


So that's really hard to say.


But there's no kind of coronavirus-related demotion


in search for travel sites.


We recently transferred our website from example.com


to multilingual one.


So for most language extensions, we have an ordinary setup,


like example.fr and LES.


But for Swedish, we kept the original domain,


like random.se.


The domain is within the same directory as our primary domain.


Let's see.


I was totally against this setup because I


don't believe Google will see this domain


extension as our main Swedish extension


and will treat it as a separate domain.


So in general, this is perfectly fine.


With regard to international websites,


sometimes you have brand.country code, top-level domain.


Sometimes you have different brand names


depending on the country.


Sometimes you have one kind of general website


and then some individual country code versions.


That's perfectly fine.


The connection with hreflang helps


us to figure out which one of these to show.


But essentially, these are separate websites.


So we will treat them as separate websites.


So from that point of view, you don't


need to match the part before the top-level domain.


The other problem is I try to block the domain in robots.txt


while it has been translated.


But some pages are still showing up in search.


I can't block the entire site as it shares the same directory


as a primary.


So that seems like more of a technical issue on your side


than something that we can really help with.


If you're using a server where you have the same robots.txt


file that are shared across multiple websites,


then that's something where you need


to find a way to kind of adjust your robots.txt file so


that it works well for the content that you have there.


And if you want to have a different robots.txt


file for different domains, then maybe that's


something that you can fix on the server side.


Most of the time, there are tricks to do this.


Even if you serve the website from the same directory


within the server where you can kind of map the host


name that was requested and then serve different URLs based


on that, there are various, I guess, technical workarounds


that you could do there.


So I would work together with your tech team


to find a solution there.


Would you say it's best to include image URLs for indexing


or to exclude them, especially in a recipe website


where there might be four to five images for posts that


is about one specific recipe?


I think it's always good to have images on a website,


and particularly on a recipe website


where people will probably use Google images to try to find


your content every now and then.


So I'd strongly recommend keeping those indexable.


And also, on a recipe website, you


can use the recipe structure data.


The recipe structure data highlights your pages


in a particularly nice way if we can recognize


that the recipes in the search results,


including an image or a thumbnail from your recipe.


So that's something we can only do if we can actually


index that image.


So from that point of view, I would strongly


recommend making those images indexable.


Obviously, you don't need to make every image indexable.


For example, if you have more decorative images


on the website, like fancy corners of buttons


or fancy parts of the UI or something like that,


then maybe that's something that you'd


need to make indexable.


But if these are really high-quality images that


are something that users might want


to search for in Google Images, then I would totally


make those indexable.


Let's see.


Some longer questions.


Let me jump over those.


Can URLs in a sitemap contain non-ASCII characters?


Yes, of course.


That's perfectly possible.


With regards to sitemap XML files, in the sitemap spec,


there is, I think, information on the encoding


that you need to use for the URLs.


So I would just watch out for using the right encoding.


But you can certainly use non-ASCII characters


in sitemap URLs.


I have a multi-regional website.


My hreflang and canonical are OK.


But in reports, there's traffic from other countries that


had their own segment landings.


Why does this happen?


So hreflang is not something that is absolutely exact


and that will block all users from going


to the wrong version of the site.


So from that point of view, I always


recommend having some kind of a backup mechanism on your site


when you can recognize that a user is coming


from a different location.


My usual recommendation is to have some kind of a banner


that you can show to users like that, something saying,


like, it looks like you're from this country


and we have special pages for our users in this country.


So maybe this is something that you would like to see.


By using a banner like that, there


are two things that you can do here.


On the one hand, users can still access the other version,


which sometimes helps because users sometimes travel,


or at least they used to travel.


And sometimes it's possible that you recognize


that they're in a different location than they actually


are, so the location lookup is not


perfectly exact all the time.


And on the other hand, when Google goes to those pages


and tries to index those pages, then we don't automatically


get redirected somewhere else.


So in that regard, if Google were


to send users to the wrong country


version of your pages, then that banner


would essentially guide them to the right version again.


So kind of like taking a step back and going back


to your question, sometimes it is completely normal


that you would see user traffic from other countries


to hreflang versions that you have marked up,


and that's just something where it's kind of due to the general


difficulty in understanding exactly what should


happen with those kind of users.


So I would just accept that this can happen and work


with a backup situation like a banner.


I have a multiple location website.


The content of the pages is similar,


so Google dropped the.com slash fr page


and said that the domain.com slash ca page is canonical


for us.


What we need is to have the French page and Canadian page.


We updated the location on the page,


created schema markup to show Google


that each page is connected to a country using hreflang,


local businesses, localized titles,


and then reindexed the page in Search Console.


Still, Google doesn't want to take this into account even


after crawling it.


Domain.com slash ca is still considered by Google


to be the canonical of the page.


Yes, this can sometimes happen, in particular,


if the content is in the same language


but for different countries.


And essentially what happens here


is our systems look at these two pages,


and they recognize that the content is either


identical or almost identical.


And then our systems try to make things easier on you


by picking one of these pages and using them as a canonical.


The canonical is the one that we will crawl a little bit more


often.


The canonical is the one that we would use as a basis


for indexing.


With hreflang, if you use that on these pages in addition,


then what will happen is we will still have that one canonical,


but we will swap out the URLs when users are searching.


So in a case like this, if we were


able to recognize the hreflang on these pages


and we had processed that, then we


would still index the Canadian version of this page.


And if users in France were to search,


we would show the domain.com slash fr version in the search


results, but it would still be based on the indexing


of the Canadian version.


So what you can do if you need to make sure


that these pages are absolutely indexed individually


is to make sure that the content of these pages


is significantly different.


So don't just tweak things like put a local phone number


on there or add structured data markup to say,


well, this is for this country.


This is for another country.


But really make sure that when our systems look


at these pages, they say, oh, this


looks completely different.


It makes sense for us to invest some more time


and to index this page individually.


From a ranking point of view, it doesn't change anything.


So it's not the case that one page will


be ranking higher or lower.


In a case like this, it's really just


like we use this page as a basis for the indexing,


or we use both of these pages as a basis for indexing.


It doesn't change anything with regards to how and when


it is shown in search.


So from that point of view, sometimes


it makes sense to just say, well, OK, Google


does this weird thing with the canonicalization,


but it saves me some trouble.


So I'll just leave it as is.


Sometimes there might be situations where you really


say, I absolutely need to make sure it's indexed individually


because maybe the company slogan is completely different


in Canada and in France.


So we need to make sure that our snippets are completely


different across those two versions.


And then it makes sense to try to split things up.


John, regarding this, there are some genuine cases


that the duration is also the same.


But in this case, I cannot target an online course


on campus page because I don't know for how long this online


will go on.


I'm not sure how you mean.


So my on-campus degree course page was ranking,


and everything was good.


After COVID-19, on-campus students


are not getting registered because of visa restrictions.


Correct?


And the company or the university


has moved to online courses because the company also


needs money.


So in this case, if I am going with a new page


for online courses, obviously the content would be the same.


The course structure is the same.


The duration is the same.


And I cannot suggest to them to play with on-campus page


because I don't know for how long COVID-19 will go on.


And again, they will be back to on-campus page.


Sometimes it is very difficult for us


to plan for if we want to target online now,


what more we should be doing because 80% of the content


genuinely is the same.


Yeah.


But I think in a case like that, the significant part


of the content would be significantly different.


If you're talking about on-campus versus online courses,


that's a very, very different setup.


In a case like that, I'm pretty sure we


would index both of those pages because they're targeting


very different things.


So I think.


Should we add some additional content just


to make different content on both pages


because syllabus would be same, exams kind of thing


would also be same?


I wouldn't artificially just add content


to make it look more different.


So I think in general, the online-offline variations


there, that should be significant on the page.


And that's something that we will pick up on.


I don't see any problems with that.


I think the trickier part, which you mentioned initially as well


is nobody knows how long this will last


and how much time you should be investing


into making those pages better.


That's something where it might also make sense to say,


I will work on the persistent version of this page


and just assume the course is kind of the main product


that's being sold and the variations in person or online


are more attributes of that main thing.


And just to keep your existing main page


and then add information about the online course there.


The advantage of using your existing pages


is that they're already pretty strong within your website.


It's already understood how they work within your website.


So ranking is going to be a lot easier.


Whereas if you create new pages for the online courses,


then that's something where we will


have to understand those new pages first


and understand how they fit in with your website


and understand how they work with the rest of the web.


I don't know.


Probably it's too late to make big changes like that.


But that's kind of the direction I


would tend to go if you need to make these kind of changes


where you're taking the same product


and providing it in a different way.


Yeah, OK.


OK, cool.


We're kind of running low on time.


I have a bit more time afterwards if any of you


want to hang around.


But feel free to ask any questions that


are still on your mind.


No questions.


OK.


I mean, I can grab more questions that were submitted.


Let's see.


We've had to adapt the information


on one of our holistic content pages due to COVID-19.


The changes will be temporary.


Concerning the meta description of the page,


I got the SEO tip to include a second meta description


and title concerning this topic info into the header.


Would this really make sense to Google?


So if you're including a second meta description tag on a page,


we will treat that the same as if you just extend the existing


meta tag on the page.


So there is no kind of bonus to using a second meta description


tag on a page compared to just adjusting your existing one.


With regards to the title, that's the same thing.


It's not that there's any kind of a bonus attached


to making a second title tag.


Essentially, you might as well just write a clear new title


using the existing title tag.


On some query in Thailand, the Facebook page


is indexed on number one.


And I noticed this for almost a year already.


Is it because Facebook.com is more authoritative


and leaves no chance for other websites


to be indexed on position one for this query?


Is there something called SEO for Facebook profiles as well?


I'm pretty sure there is something


called SEO for Facebook profiles.


I don't know what the name is, but I


am pretty sure for all of these big networks,


there are people that work on optimizing their profiles


and optimizing their pages to work well there.


So that part of the question is something


I can't really help with, but I'm pretty sure


that that's the case.


With regards to Facebook ranking for some queries in Thailand,


I think it's completely normal to have


a site like Facebook rank for some queries in Thailand.


It's not the case that we would say Facebook is the best


page in the world.


Therefore, it should rank for all queries.


But for some queries in some locations,


it probably makes sense to rank some of that content there.


The content there is essentially public web content


like anything else.


We can crawl and index it.


And if we think it's relevant for users,


we will try to show it in the search results.


So that's essentially pretty straightforward.


With regards to kind of competing


in a situation like this, it's essentially up to you


how you want to interact there.


It might make sense to look into those bigger platforms


and find ways to kind of highlight your content there


as well.


It might make sense to continue working on your own website


where you have a little bit more control over what you actually


do and to work on promoting that over time.


So I really can't give you advice one way or the other.


But essentially, there's nothing kind of magical about content


that's hosted on Facebook versus content that's


hosted on your own website with WordPress or maybe content


where you're using Wix or some other content management


system.


All of this content can rank highly in the search results.


And it's not that there's any kind of manual lever


on our side that says this domain has to be the one


that we show for this query.


Technical issues can cause a swing between normal rankings


and return to normal if we have temporary server errors


like 500.


Such events lasting a few hours.


And Search Console does not log these events as errors.


So this is something where it's hard to say


exactly what you mean there.


But in general, technical issues that happen temporarily


are things that we can work around for the most part.


So if your site is returning 500 errors or 503 errors


for maybe a few hours, then what will happen practically


is our systems will say, well, this


looks like something temporary is going wrong here.


And we will get back to this website


and check back with it later on.


And if later on we get to this website again


and we see that things are still broken,


then after a certain period of time,


we will drop those URLs from our index.


And at that point, we will highlight that in Search


Console and say, this page is no longer


indexed because of crawl issues.


Whereas temporary issues like this, which kind of come and go


and our systems are able to deal with them appropriately,


those are not things that we would necessarily


need to report in Search Console because they wouldn't


affect indexing until they do become something that happens


for a longer period of time.


So from that point of view, that's


kind of working as intended.


The one thing where these kind of errors


would have an effect a little bit faster than maybe


before the URLs drop out of the index


is with regards to the crawl rate of the website,


where if we see a lot of URLs suddenly returning server


errors, then usually our crawl system will say,


oh, we need to be careful here that we're not


the reason why these pages are returning errors.


And they will tend to kind of scale back the amount of


crawling that we do on this website.


And usually this happens over a couple of days.


And as we see that those errors go away again,


then we can ramp up the crawling again and crawl


as much as we did before.


This is essentially a kind of a protection mechanism


to avoid Google being the reason for causing issues


on a website.


All right.


I think we're kind of over time.


So let me pause the recording here.


If any of you want to stick around,


you're welcome to hang out a little bit longer.


Otherwise, I wish you all a great weekend.


Stay safe.


And hopefully see you again in one of the future Hangouts.


English Google Webmaster Central office-hours from February 21, 2020

OK, welcome, everyone, to today's Webmaster Central


Office Hours Hangouts.


My name is John Mueller.


I am a webmaster trends analyst here at Google in Switzerland.


And part of what we do are these Office Hour Hangouts


for webmasters, publishers, anyone


who's kind of interested around search and has a website.


A whole bunch of questions were submitted today,


but maybe I'll just start off with something from your side,


if anyone wants to go ahead and ask a first question.


I'll ask a question if no one else will.


Go for it.


Is there any plans to bring back the detail in Webmaster Tools


for where your 404s originate?


You can't see what the linked pages are at the moment.


So whereas you used to drill, be able to drill down and say,


well, actually, it's coming from here, linked from,


that's no longer an option.


So the report itself becomes almost pointless


because you can't fix what the actual error is.


Good point.


I know there was some discussion around that internally,


but I don't know what the current status is there.


I'll poke at it again.


OK.


Thank you.


I also have a question.


How would it affect deduplication


between multiple languages on a website?


If some of the pages that I have, hreflang,


if they are sort of like noindexed or redirected,


how would that affect, yeah, the deduplication?


OK, so with hreflang, it's always a connection


between multiple pages.


And if any of those pages don't work,


we just ignore those specific pages.


So if they're like individual language pages that


are noindex, or if they're redirecting somewhere else,


or if they don't have a return tag or anything like that,


then we would just ignore that one page.


So it's not that we would kind of say,


well, something is broken.


We will ignore everything, or we'll


ignore it for the whole website.


It's really just that individual page that we would ignore.


OK, great, thanks.


OK, there are tons of questions submitted.


So I thought maybe I would just run through them fairly quickly


so that we can try to get as many of them


covered as possible.


Some of the questions are kind of long,


so I'll try to simplify them.


Let's see, the first one I have here


is we're finding that Extended Stay America is the default


3-pack for any queries like Extended Stay and the city


name.


Kind of it feels unfair to other websites that


are offering similar things.


So I think two things here.


On the one hand, I don't know too much about the 3-pack,


about all of the Google My Business rankings


and how all of that works.


So that's kind of one thing there.


The other thing is I'm not completely aware of this query


and how specific that might be to that particular website.


So I don't know, it sounds a little bit


like people are trying to find that website,


so we are trying to show that website.


But I'll kind of pass this on to the team


here to take a look at as well, just


to make sure that they're happy with the kind of results


that we're showing there, or if there's something


that we need to improve on from an algorithmic point of view.


We generally add informative text content


below our product category page related to the specific product


and category.


Will it be OK if we add article schema with product breadcrumbs


and schema on product pages?


So you can definitely do this.


I don't know what advantage you would have by including


this kind of markup on a product page,


because when it comes to rich results,


we generally pick one type of rich result


and we show it like that.


So probably you're not going to see


a big advantage of adding article markup to a product


page.


The other thing I kind of caution against


is just arbitrarily adding text to a page


doesn't make it better.


So one thing we sometimes see is that people


take a product category page on an e-commerce website


and add five Wikipedia articles of text below the category


page, just to make sure that all of the possible keywords


are covered.


And that doesn't help us at all.


On the contrary, our kind of keyword stuffing algorithms


might pick that up and say, well, there's


just so much random stuff here on this page.


I don't know what we should rank it for at all.


So I would really focus on informative text


that people actually read and see


if you can provide value like that,


rather than just trying to dump as many keywords into a page


as possible.


A few months ago, I asked how to recover my site


from an algorithm change.


As per your suggestion, I removed 70%


of the non-performing, low-quality pages


from the index, and the process is still ongoing.


And the question goes on with, I would


like to add even more content to my website.


So I think that's something where


you have to figure out what the right balance is for you.


I'm still not completely convinced


that these kind of pages, like the one that you linked to,


are that valuable.


But that's ultimately up to you.


But the same thing applies to anything new


that you're adding.


It's like something like find a balance for you that works,


and pick the kind of pages where you would like


to add more value to the web, and kind of focus on doing


that.


What's the best way to contact you about an issue


or you ask more information about?


Usually, these office hour Hangouts are a good way,


or contacting me on Twitter is an option.


If you have more information kind of in a help forum thread,


that's also an option.


I see you linked to a forum thread there,


so I'll go take a look at that afterwards.


My website is still not showing up in Swedish Google Discover.


But outside of Sweden, it does.


Do you know if Google are able to block specific sites


from showing up just in one country?


I am not aware of anything manual like that


with regards to Discover.


Usually, that's a pretty algorithmic setup.


But I'll double check with the team here.


One thing that I kind of notice is


you have a bunch of websites that you mentioned there.


So I kind of wonder if you're really


kind of focusing your effort on one strong website,


or if these are websites that might be worth combining


into one stronger website.


I don't know if they're all your websites,


or if they're other people's websites, though.


But I'll pass that on to the team that


works on the Discover feed to see


what they might find there.


In general, Discover is an algorithmic, organic feature.


So it's not the case that every website in every country


would show up in Discover.


It's also based on what users are interested in.


So just because you have content doesn't necessarily


mean that we would show it there.


But I'm happy to pass it on to the team.


With regards to the For You articles in Chrome,


I don't know if that would be considered the same thing.


I imagine there are some aspects that we


share across these different kind of bubbling up information


platforms.


But they're pretty unique sometimes.


The new Google News Publisher Center


requires an RSS feed for including articles.


And it goes into a lot of detail on specific tags


within the RSS feed.


I am happy to pass that on to the Google News team.


I don't work on the Google News side,


so I don't really know what's all behind that.


But I'll pass that feedback on to the team.


I think you're also active in the forum.


So escalating it in the forums would also


be a good opportunity.


A question about alt text in photos.


I have a website for a cancer clinic


that has many pages and photos.


Let's say on a page about breast cancer,


there's a photo of a room with devices.


Should I write in the alt text only what's in the photo,


or can I use keywords about kind of cancer and treatment


and all of that?


So alt text is an alternative text for an image.


It's not kind of that catch-all place


where you dump all of your site's keywords in.


So I would really focus on kind of what


you're showing in those images or what's relevant


with regards to those images.


So don't just kind of blindly include all of your site's


keywords there, because that's not really


going to be useful neither for users nor for our algorithms.


So we use alt attributes to understand the images


a little bit better.


And it doesn't really help you that much


if you include all kinds of keywords specific to an image,


in the sense that in the best possible case,


we would show that image to people


searching for those keywords.


But if that image is not relevant for their interests,


they're not going to go to your site.


So just appearing in the search results


isn't something that is really that useful


when it comes to specifically around image traffic.


This is a follow-up to some questions


regarding a big drop in rankings with our website.


I've seen you and your website around for a while now.


And it's kind of frustrating to see that it's not picking up


from your side.


You list a whole bunch of things that sound like good ideas


that you've been working on.


So I'll definitely pass this on to the team


to double-check to see if there's anything more specific


that we can point at.


So I think your idea at the moment


is to change domain names again, which


is a pretty drastic measure.


So that's something where I'll double-check with the team


here to see if we can point out something more specific that


might be worth looking at, or if it's more a matter of, well,


the web has just changed significantly over time,


and things that used to be very, very relevant in the search


results are maybe not so relevant anymore


for specific queries.


But I'll double-check with the team here.


Purchases made by consumers on e-commerce sites


are a sign of Google ranking.


No, we don't know what people purchase on e-commerce sites.


So it's kind of hard to use that as a ranking factor.


In Search Console, it looks like I've


had little traffic from Discover in Sweden in the last weeks,


while Google Analytics said I haven't had any traffic


from Google Discover.


Which of these sources should I trust?


I would focus on Search Console.


So specifically for Google Discover


and the different kind of feeds that we have like that,


from my understanding is there is no specific refer


that we would use for that.


So looking at Analytics, you probably


wouldn't see much information there.


Whereas in Search Console, it's really


focused on giving you information


about the clicks and impressions where we actually


showed your site, which is generally a little bit more


information for Discover.


When it comes to Discover, I believe


the refer that we send with the user traffic


is more like a generic Google refer


because it comes from one of the many Google surfaces.


But in Search Console, you would see it more specifically.


So sometime we had the same hreflink


present multiple times, up to 20 times on category pages.


Can Google penalize us for this, or should we


mark them as nofollowed when they're repeated?


No, Google will not penalize you for having


the same link multiple times on a category page.


Sometimes that just happens for technical reasons.


Sometimes that's just the way your website is built.


That's not something that we would


consider to be kind of malicious or spammy.


Should we mark them as nofollowed?


No, you don't need to mark them as nofollowed.


Again, that's perfectly fine.


We see that link on the page.


If we see it multiple times on a page,


it doesn't change anything.


With regards to seeing things multiple times on a page,


the one thing I would watch out for


is more like keyword stuffing.


So if you're mentioning the same keywords not like 10 or 20


times, but 200, 300, 400 times on a page,


then that's something where we might say, well,


this is a lot of duplication.


And we don't know if perhaps that's something


that we need to watch out for.


But just having something like a link 20 times on a page,


that's absolutely no problem.


Can breadcrumb schema on home page


give any benefits over non-breadcrumbs home page


schema?


I don't think you'd have much advantage when


it comes to the home page of putting breadcrumb structured


data on that page, specifically because there


is no kind of breadcrumb trail that leads to your home page.


It's like the start of all breadcrumbs, right?


So from that regard, I don't think


there is any advantage there.


Some sites just use breadcrumb schema on all pages.


And if it's easier to put it on all pages


rather than to individually pick and choose,


then having it on the home page is perfectly fine.


But I don't think you'd see an advantage.


How can we change the content, website name,


and the Apply button in Google for Jobs when it's wrong?


Adoption of the title tag, image tag, taking jobs


ads in and out with the API was not successful.


I honestly don't know how that works in Google Jobs.


But if you want to drop in a link to the search result


or the site where you're seeing this,


then I'm happy to pass it on to the team


to have them take a look as well.


We've been dropping in Search Console average position


in general and for relevant keywords at a rate of 0.24


and 0.1 each month since 2017.


That's pretty exact.


I looked at the backlinks, and we have 50%


of the total backlinks come from a subdomain generated


by a tool we used since 2012 and stopped using three years ago.


This tool allowed us to publish help guide material


for our users.


Despite eliminating all of the generator posts


one year ago with this tool, all of the URLs remain active.


Could this be directly related to a dropdown?


What would you suggest to do?


Does disavowing a domain with half of our backrank,


will that affect our rankings?


So I think starting with the last one,


if you disavow a significant part of your natural links


to your site, then yes, that can negatively affect your ranking.


But that would be negatively affecting your ranking,


probably not what you're looking for.


In general, if this is content that you published,


which is kind of help documentation,


extra material about your website,


about the product or services that you're offering,


then that seems like perfectly fine content to keep indexed,


and not something where I'd say you need to block that


from being shown in search, or you'd


need to block those links from being picked up on.


So from that point of view, I don't


think any of the changes that you're seeing


are related to this content and those particular links,


but it's probably a more general thing.


So in particular, when you're seeing a gradual drop


over a longer period of time, then that, to me,


points at natural ranking changes,


where things in the ecosystem have changed,


things in our algorithm are slightly changing,


maybe users are searching in different ways


or expecting different kinds of content in the search results.


And that generally wouldn't be a sign


that there is this one thing that you're doing wrong, which


kind of made everything blow up, if you're


seeing these kind of granular, step-by-step changes


over a longer period of time.


So that, to me, would be something


where I'd try to take a step back


and try to look at the website in general overall,


and find maybe areas where you can make significant


improvements to kind of turn the tide around a little bit


and make sure that your site becomes more relevant


or becomes significantly more relevant for the kind of users


that you're trying to target.


What's the difference between index pages, so site colon


and URL on google.com, versus Search Console coverage,


which one is more accurate?


So in general, I'd say the Search Console index status


report is extremely accurate.


That's the one that's really based on pretty much all


of the indexing that we have.


A site colon query is a restricted query,


which means we would try to only show results


from that particular site.


But it's not an inclusive query in the sense


that it includes everything that we


have indexed from that website.


So sometimes you will see things like you do a site colon query,


and it shows you, I don't know, a couple hundred results.


And you look in Search Console, and it says you have 100,000


pages indexed.


And from our point of view, that can be completely normal.


So that's not necessarily something


we would consider to be wrong.


The other thing is that if you have a larger site,


and you look at the count that is shown in a site query,


then that's something that is generated


with a focus on speed, not with a focus on accuracy.


So that as well can be something where the number where


we show of about, I don't know, 100,000 pages,


that's something where that number can


be significantly different.


I've seen cases where it's off by a factor of 10,


or 100, or even more.


So that's something where if you want


to have a number to focus on with regards to the pages that


are indexed for your site, then definitely


use the Search Console coverage report,


and don't use a site colon query.


And the same thing applies to in URL queries


if you're kind of doing it the other way around.


On an e-commerce site with around 100,000 products


categorized in many different attributes,


Googlebot will be dealing with 100 million plus pages


generated from all sorts of filters.


Google is setting its own canonical instead of ours.


This is giving us lots of low quality URLs indexed.


Is it OK to disable filter links that


don't have a search intent only for Googlebot


to eliminate a number of URLs for Google to process?


It's perfectly fine to put no index on these kind of pages.


So that's something where from my point of view,


really focusing on the pages that you want to have indexed


is kind of what I would do there.


In general, when it comes to category pages,


I would try to find one kind of sorting


order and a general filtering that makes sense overall


and let us kind of crawl and index


all of those paginated pages from that setup.


And then past that, if you want to have individual facets


that you think are relevant and useful in search,


then that's fine.


But everything else, I would find a way


to block, which the easiest way to do that is just


use a no index on those pages.


On a product page with a load more JavaScript button,


generally, how many times do you trigger the load more


before you stop?


How do you detect the load more button if it's an image button


or has an unusual text other than load more?


So my understanding is we would not click on this at all.


I believe in the past, we would try to figure this out


and try to trigger some of these to see what would happen.


But that's extremely expensive when it comes


to rendering the whole web.


So I believe we don't do that at all.


What we do instead is we do something


we call frame expansion, which is we take the page that you


serve us and we try to render it on an extremely long viewport.


And we see if the page does anything like infinite loading


where it expands the viewport.


And that's something where we would expand that viewport once


and see what all loads and try to index all of that.


But we would not do this multiple times in a row


because, especially when it comes to infinite scrolling,


like the name says, some websites actually


do have pretty much an infinite scrolling where you could


expand this a whole bunch of times


and you wouldn't necessarily find new content.


You just find kind of repetitive sections


across the whole website.


So if you're using a Load More button with JavaScript,


then that's something we probably wouldn't use.


That's something I'd try to reconsider.


If you're using infinite scrolling,


then double check our documentation


to make sure that you're implementing it


in a way that works well for Google for indexing.


Let's say ranking factors could be measured in bananas.


That's crazy.


How many bananas would you give URL structure in comparison


to a title tag?


And what is your future prediction about this?


So I would give it seven bananas.


I think that's a good number of bananas.


Actually, that's a lot of bananas.


No, I don't know.


I think you're generally asking, is the URL structure


more important than a title tag or not?


And these are vastly different things.


So that's not something where I'd


say you should focus on either one or the other,


but rather, if you want to work on your site


in a holistic way, then try to make sure


that everything aligns.


And I don't think that's something that will


significantly change in the future.


We will definitely try to understand


which parts are relevant and treat that appropriately.


But sometimes, we can pick up a little bit


from the URL structure.


Usually, we try to pick up most of our information


about our page from the page's content itself, which


would include the title tag.


But just blindly focusing on URL structure or title tag,


I think, is way too simplistic.


We operate one of the largest stock photo agencies


with millions of visitors from Google Monthly.


We have experienced the issue of Google folding together


localized versions of our website.


I described the issue in the forum thread.


We implemented hreflang and canonical for many years


without any issue.


What's the issue now?


I don't know.


It sounds like something where I'd


have to take a look at the details.


And with the forum thread, I'm happy to pass that


on to the team here to double check to see


what might be happening there.


One of the common, I guess, sources of confusion


with regards to hreflang is if you


have exactly the same content, such as the same language


content, but for different countries,


then that's something where our indexing systems will often


say, well, this is a duplicate of the other page


that you have, which would be correct because it's


the same language content.


And in cases like that, we would understand


that this is a duplicate.


We'd fold it together for our indexing,


and then we'd try to fold it apart again


when it comes to showing it in the search results.


So in the search results, we try to do the right thing.


However, since in indexing, we fold it together,


in Search Console, the reporting is quite confusing there