on it, then over time, we'll learn that it's a new website


English Google SEO office-hours from February 4, 2022

and we'll treat it accordingly.


But there is no specific time frame for that.


So from that point of view, I don't


have any date or any kind of duration


that you need to watch out for.


It's been a year, sir.


Yeah.


I mean, it sounds like you're doing the right thing.


see anything kind of problematic that you have to do differently.


I guess the two aspects I would watch out for


are if the previous website was involved


with a lot of shady practices, especially


around external links, then that might be something


that you want to clean up.


And the other aspect is if there's


any web spam manual action, then obviously that's


something that you'd want to clean up so that you can kind


of start from a clean state.


It's never going to be completely clean


if something else was already hosted on the website before,


but at least you're kind of in a reasonable stage


and you don't have to kind of drag that baggage along.


So is it more like to develop all those backlinks which


we are getting for the travel business kind of thing


from many countries?


Does that mean that we should develop all those links


from different countries for the travel significance


and then only focus for the commerce one,


or doing a lot of VR and all sorts of activities


and getting the news for our e-commerce business?


You don't need to disallow or disavow


the links that are related to the other business.


But if there are problematic links related


to the previous business, then that's


something that you want to disavow.


So if, I don't know, the previous business


bought a lot of links or did a lot of weird things


with regards to links, then that's


something you want to clean up.


And that's something you'd want to clean up regardless


if it's a previous business or just the previous SEO that


was working on it.


So that's kind of the direction I would head.


Just because it's a different topic


or maybe from a different country,


that's not a reason to disavow.


I don't think that's problematic.


So thank you so much.


Sure.


John, I have a follow up on the same.


OK.


So does it make also any sense if for links,


I understand you told that you should get the links updated


if there is any problem.


But for content, suppose earlier it was travel domain.


So all the content was about travel.


So site owner, if he moves to e-commerce,


should he also approach the old website owners


where they have published travel content about travel?


And should he ask that kindly remove travel related content?


No.


I don't think that's, I mean, like you can.


But I don't really think that's necessary.


Because websites evolve over time.


And what worked as a link in the past


might not work as a link in the future.


It's not that this will cause you any problems if someone


wrote about kind of like what the previous owner


of a domain did.


So I was only thinking that if, as you told,


that it may take time to understand the new business


of the domain name.


So I was thinking if removing old content may expedite


Google to understand that now this is not travel anymore.


I doubt you would have to go that far.


No, I don't think so.


OK, John.


Thank you.


Sure.


OK.


And next up, I can't read your name.


I'm sorry.


Hi, John.


I think, I don't know, I think you might be muted.


He's a Korean, I think.


Yes.


Hi.


The person with the mask.


Yeah, he's talking about a.


Oh, I think you're still muted.


Sorry.


OK, can you hear me?


Yes, perfect.


OK, I'm, yeah, that's Korean.


I have a question about relationship


between failure from robot TXT and server and connectivity


failure and search traffic from Google.


Because I saw more than 5% of robot TXT and server


connectivity failure recently in Search Console.


So we had seven days failures in November.


And it happened once in December.


So our server team blocked some IP addresses from Googlebot.


So since November, our website traffic from Google


has dropped about 20%.


So if nothing changed during the date, is it reasonable?


If I blame the server team for the decrease of their traffic,


I'm just wondering.


So how much effect on the problems


will be from the robot and server connectivity


to Google search?


Is the problem fixed now?


Or?


They kind of opened their IP address.


But we had one more problem in December.


And they don't know what is the reason.


Because they all opened their IP address.


And they don't know what is the reason from their Google search


concern.


So I think there are two things maybe to watch out for.


On the one hand, if you have server connectivity issues,


we would not see that as a quality problem.


So it wouldn't be that the ranking for your pages


would drop.


So I think maybe that's the first step.


So if you see that the ranking of your pages is dropping,


then that would not be from the technical issue.


On the other hand, what does happen with these kind


of server connectivity problems is


that if we can't reach your robot's text file for a while,


then we will assume that we can't crawl anything


on the website.


And that can result in some of the pages from your website


being dropped from our index.


So that's kind of a simple way to figure out,


is it from a technical problem or not?


Are the pages gone from the index?


And if so, that's probably from a technical problem.


Or are the pages just ranking lower?


Then that's not necessarily from a technical problem.


And if it is from a technical problem,


like if these pages are gone, then usually we


will retry those missing pages after a couple of days maybe.


And we will try to index them again.


So if the issue was in November and you fixed it in November,


then I assume you should not see any effect from that anymore,


just because we should have re-indexed


those pages in the meantime.


If you do still see that we're not indexing the pages properly


from your website, I would double check the crawl error


section in Search Console to see if there is still perhaps


a technical issue where sometimes maybe Googlebot


is blocked.


SPEAKER 2 OK.


Thank you, sir.


JOHN MUELLER Sure.


All right.


I'm mayor SEO.


SPEAKER 3 I'm also muted.


OK, now.


Hi.


How are you?


JOHN MUELLER Oh, very, very hard to hear you.


SPEAKER 3 Great.


Just a second.


Maybe I should change the microphone.


Is it better now?


JOHN MUELLER Very faint.


SPEAKER 3 Like this.


Like this.


It's getting better.


OK.


We'll try.


So I have small questions.


First, should the title be with the company name


like we are in the news industry?


And should we add to each title the company name,


like we are like Test the News or Dash News?


This is the first.


You can answer it if you can't.


It's small yes or no, I think.


I think it makes sense.


So when we show a title link for a website,


if it's for a lower level page, we'll


try to show the name of the website


as well in the title that we show.


So if you add it yourself, then you already


include that information.


So I think it's a good sign.


OK.


The second thing, some of our competitors


is like copying the article and base it again on their website.


But they rewrite the main header or the title


inside the article.


So they are showing before us in the search results.


Is it a good practice to do that,


like rewrite the title, the header title inside the article


or the main title inside the article?


I don't know.


I mean, if they're copying your content,


then I don't think that's a good practice.


So from that point of view, I don't


think just rewriting the headline


would make it a good article.


If they're really copying your full content,


I might also look into the DMCA process


to see if that applies there.


Actually, I applied the three or four copywriter removal


and Google removed their articles.


But for the first two or three days,


they are showing up before us in our article.


Because they just three are rewriting.


So I thought, should we do that?


So I think it's a no.


We shouldn't do that.


OK.


I mean, you can change the headlines of your articles


anytime.


That's up to you.


But I don't think you would see an advantage of changing


the headlines, just purely from changing the headlines.


We have a comment section under the article.


But it's not showing in Google.


If I searched for a comment, like someone is typing a comment


and I search for it, I don't find it like the article


is not showing there.


Is there any good or best practice


for the comment section I should do?


So can Google see more of our comments?


Because comments is like discussion about the news


it's happening.


Is what the good practice of this?


It's up to you.


You can decide if you want to have that shown or not.


But it's essentially a technical element on the page.


So it's not that there's a setting in Search Console


to turn it on or off.


It's basically there are different ways


of integrating comments on web pages.


And some of those ways are blocked from indexing.


And some of those ways are easy to index.


So if you want to have your comments indexed,


then make sure that you kind of implement them


in a way that is easy to index.


And the inspect URL tool in Search Console


will show you a little bit what we find on the page.


So you can test to see, can Google index the comments?


But it sounds like even when you search in Google directly,


you already see that we can't index them.


So that's.


AUDIENCE 1 Yes, so there's some problem, I think, in structure.


OK, should we add the title in the, not the title,


the dates in the title?


We have some prices of currency.


So our currency today is like all the currencies for today.


Should we add the title?


Or should we add the date inside the title?


Is it good practice or no?


It's not basic.


JOHN MUELLER I don't think it changes anything.


So for news articles, I do think it makes sense


to include the date in various places on the page.


And that can include the title.


Just because with news articles, we


try to understand what the primary date is of the page.


And we do that by looking at all of the mentions and things


that you have on the page.


And if we can confirm that date with these mentions


on the page, then it's easier for us to pick that up.


But I think for a page that is changing constantly,


like currency prices, like you mentioned,


or previously stock prices, I don't


think it's critical to have the date in the title.


AUDIENCE 1 We are creating each day a new article


with a different, that why I said maybe we


should list the dates or not.


Each day there is a new article, not changing the main article.


Like the static page.


JOHN MUELLER I suspect that will make it really hard for us


to crawl and index your website.


So that's kind of the other aspect there.


If you're automatically generating these pages


on a daily basis for all of the currency.


AUDIENCE 1 We are not automatically manually,


actually.


We are writing because, yeah, the dollar


rise because of one, two, three, the dollar.


JOHN MUELLER Oh, OK.


Yeah, OK.


I mean, from my point of view, then those


would be normal news articles.


So if you want to include the date in places like that,


I think that's perfectly fine.


I don't think you would get any magical SEO


bonus from adding the date.


But it's OK.


Sure.


OK, last one, should we add like our URL now is like website


slash news slash ID of the news.


Should we add the category like maybe USA or Arab states


or something slash then the news?


Is it like affecting the category


of the news inside that URL?


I don't think you would notice any visible change there.


I think for tracking purposes, sometimes it


makes it easier because then you can look at analytics


and say, oh, this category is very popular


or this category is not working so well.


But from an SEO point of view, I don't


think there would be any noticeable difference.


Thank you very much.


All right, Saidu.


Hi, John.


I have a question about Google crawling and indexing.


So recently, we are facing some issue


for some of our client website.


So we are writing content and posting content on their blog.


What happens last four months, we


are posting content on the website.


But most of the content are not getting indexed by Google.


And when we inspect the URL, we usually get two messages.


One is the URL is crawled but not indexed yet.


Or URL is discovered but not indexed yet.


So my first question is about if it's a URL scroll but not


indexed, does it mean Google thinks


this content is not perfect for indexing?


Or what does that mean?


I don't think it means anything in particular.


So I think that's always an easy early assumption to say,


oh, Google looked at it but decided not to index it.


Most of the time, when we still crawl something,


it doesn't necessarily mean that we will automatically index it.


So I would almost treat those two category of not


indexed as a similar thing.


And it's tricky because we don't index everything.


So that can happen.


If you think this is a good website where


we should be indexing more, I would


love to have some example URLs so that we


can pass that on to the team.


If you want to drop some in the chat,


that would be super useful.


OK.


And the next one is, in this scenario,


it is not only for one client.


We are facing these four multiple clients, five


or six clients at the same moment.


So is there any other way?


We are trying to inspect the URL,


then requesting for indexing.


We are trying to use Google's SaaS console.


Is there any other way to make it faster?


Or is there any other thing if we do this, maybe that


will help for getting indexed?


I mean, there are different things, which perhaps you're


already doing.


On the one hand, making sure that it's


easy for us to recognize the important content on a website


is really good, which sometimes means making less content


and making better content, so having fewer pages


that you want to have indexed.


The other thing is internal linking


is very important for us to understand


what you would consider to be important on a website.


So things, for example, that are linked from the home page


are usually a sign that you care about these pages,


so maybe we should care about them more.


Things with external links, they also


go into that category of where we see other people think


that these pages are important, then maybe we


will see them as important too.


And then site maps and RSS feeds from a technical point of view


also help us a little bit better to understand


these pages are new or they have changed recently.


We should check them out again and see what is there.


But all of these things kind of come together,


and it's something where it's rarely


that there's one trick that you're


missing to get these pages indexed.


And the last question, so what happens when we saw that those


blog posts?


Now, we are using those blog posts for interlinking as well


to other pages.


So getting the link to those blog posts, which


are not indexed by Google, will have any impact on the website


ranking or will add any value from a SEO point of view?


I mean, if we find external links to those pages,


then chances are we might crawl and index that page.


It's like a little bit higher, I guess.


It depends a little bit on what kind of external links,


of course.


There are links from social media


directly usually have no follower count,


so we don't really forward any signals there.


And if it's something where we can recognize,


well, these are maybe problematic links


or not that useful links, maybe we will ignore those too.


But obviously, if we can tell that something


is seen as being important, we'll


probably go off and crawl and index that page more likely.


What you generally won't see is that we


will kind of forward value to the rest of the website


if we don't actually index that page.


Because if we decide not to index that page,


then it's still that situation that, well, we


don't have a destination for those links,


so we can't do anything with those links


for the rest of the website.


Thank you, John.


Sure.


All right.


Meet.


Yeah, hi, John.


So our website is based out of India mainly,


and our users are also from India.


So like a few days back, what we did


was we moved all our dynamic traffic behind the CDN.


But what we have been observing is


that in the search console under crawl stats,


our response times have gone up from like 300 milliseconds


to around a second.


So we are not understanding.


And also, the crawl rate has dropped.


It dropped from around 2 million requests a day


to around 80k requests a day.


So we are not understanding does moving to a CDN


impact how Google crawls our website?


And could the network latency that


is introduced due to the CDN servers being


present close to from where Google is crawling our site,


does that impact how the crawl rate as well as our search


ranking as well?


And also, yeah, so Prateek from my team


is also there if he wants to add something.


Yeah, so from a ranking point of view,


this would not change anything.


So maybe that's kind of the first question ahead.


But it can.


If you change your hosting significantly,


what will happen on our side is the crawling rate


will move into a more conservative area first,


where we'll say we'll crawl a little bit less first


because we saw a bigger change in hosting, which


could be a move to a CDN or a move from a CDN


or from one CDN to a different CDN.


And then over time, over, I don't know, a couple of weeks,


maybe a month or so, we will increase the crawl rate again


to kind of see where we think it will settle down.


So essentially, that drop in crawl rate overall for move


to a CDN or change of a CDN, that can be normal.


The crawl rate itself doesn't necessarily


mean that there's a problem.


Because if we were crawling 2 million pages of your website


before, it's unlikely, I assume, that you would have 2 million


pages that change every day.


So it's not necessarily the case that we


would miss all of the new content on your website.


We would just try to prioritize again and figure out


which of these pages we actually need to recrawl


on a day-to-day basis.


So just because the crawl rate drops


is not necessarily a sign for concern.


What wouldn't worry me more is the change


in the average response time.


Because the crawl rate that we choose


is based on the average response time.


It's also based on server errors, those kind of things.


And if the average response time goes up significantly,


we will kind of stick to a lower crawl rate.


So that's something where I assume


if the average is around one second per URL on your website,


I feel that's quite a problem.


So actually, about that, the average response time


has actually improved for our users.


It has only increased for the Google crawler.


So we are not able to explain that, actually.


Yeah, I don't know.


It depends on how you have your CDN set up.


So that's, I think, always a tricky part


if you focus your website on users in one country


and Google box crawling usually happens from the US


and you don't kind of see what users in the US would see,


then that's sometimes a little bit tricky to diagnose


and to figure out what exactly is causing that delay.


But from our point of view, the response time


is critical for the amount of crawls that we do per day


because we want to limit the number of active connections


that we have to your server.


And if the response time is high,


then we run into that limit fairly quickly.


Whereas if we can crawl quickly, then we


don't have that much of a problem


with the number of concurrent connections.


SPEAKER 2 We tried running tests from US locations.


With CDN, without CDN, the latency


that we are observing are same.


So is there anything else that we can look into?


SPEAKER 1 I don't know.


It's hard to say.


I would use maybe some testing tools that are running


within Google's network.


So if, for example, you have something,


I don't know, if you can set up a VM on Google Cloud,


for example, and use that to test the latency


to your server, that might be an approach


that you could take.


What you could also do is look at your server logs


to see which URLs Googlebot is explicitly requesting


and check to see if there is a general pattern there.


So in particular, it might be happening


that Googlebot is crawling a lot of URLs


that take a lot of resources on your side, which


are not representative of the rest of your website.


And because of that, it might be that the URLs that Googlebot


is requesting, they take a lot of time.


And maybe there are ways for you to either optimize those URLs


or make it so that Googlebot finds fewer of those URLs


or maybe even block them completely


if they're not useful for your website.


SPEAKER 2 So John, I have Prateek here.


Sam with me.


So what we observed was the time it took for a request


to be served from US.


That did not change at all before or after we


moved our URLs over there.


So could it be that previously our origin servers


were directly requested by Google, and they were in India?


And Google is considering the network latency over there


to be very high.


So it is reducing that network latency


and showing a reduced response time in the crawl rate.


And for the CDN, our servers, the site


would be served from US location,


as the CDN servers would also be present in US.


So for Google, probably the server


is now present in US instead of India.


And it is not reducing that network latency as such.


That shouldn't be the case, no.


We really use the actual latency that we see, not anything


where we say, oh, it might be like this.


Because the report there is less about usability for users


and really more about the practical crawling


from Google's side.


So that's something where we really


report exactly what we saw.


And like I said, what might be happening


is that there is something along the chain of that request that


is slowing things down.


It might also be that on average, the requests


that Google makes are just slower because


of the URLs that are accessed.


So I've seen this, for example, when a website has


a lot of very large PDF files.


Then obviously, per request, the amount


of bytes that need to be transferred is very high.


But that doesn't mean that on average,


when you look at other URLs, that it's a very long time.


But it's that large number due to the size of the bytes that


is pulling things down.


It can also be that, for example, we


crawl a lot of search pages.


And maybe the search functionality on your site


goes directly to the origin servers


and takes a long time to be processed.


Then again, it could look like the average response


time is quite high.


So I would try to look into the details there.


Another thing is perhaps something


around the lines of your DNS connection


or anything in the line of the connection


there that might be slower for Googlebot to process.


One last question.


The increase in latency for the crawler,


would it affect the rankings?


No.


So we are seeing a constant drop over there since we moved.


So that was one question that we had.


Yeah, that would not be affecting the ranking.


So the technical side of things is really purely


about getting all of your pages as quickly as possible,


getting the new pages indexed.


And if we aren't able to crawl as much,


we'll try to prioritize more.


But that wouldn't be affecting the rankings.


Thank you.


Thanks.


Thanks a lot.


Sure.


OK.


Let me go through some of the submitted questions


so that we don't lose track of them completely.


And I'll go through more of the live questions


as well afterwards.


Don't worry.


I think the first one was the CDN question, which we just had.


The next one I have is we've undertaken a substantial code


change of our website to JavaScript with server-side


rendering to static HTML.


It's been four months, and our search rankings


are not recovering after a drop.


We're also noticing strange activity in the search results


with vastly inconsistent rankings between target keyword


and subtle variations.


It feels like Google is struggling


to understand the broader context of the content, which


has not changed after the upgrade.


Are you able to shed any light on why this is occurring?


It's really hard to say without seeing the website itself.


But essentially, if nothing on the website has changed,


if you just move from client-side rendering


to server-side rendering, then I would not


expect to see any change in rankings or visibility


on the website.


So I would assume they're either one or two things are


happening.


On the one hand, it might be that with the change


of your infrastructure, your website layout and structure


has changed as well.


So that could include things like internal linking, maybe


even the URLs that are findable on the website.


And those kind of things, they can affect ranking.


The other thing could be that maybe there


were just changes in ranking overall that were happening,


and they just happened to coincide


with when you made the technical changes on your website.


And that's something where I think


you can kind of try to figure out


if that might be happening by really comparing


your previous website to your current website


to figure out whether really significant structural changes


are not.


If there are no structural changes


and you're seeing a change in rankings,


then I would assume this is more of a general ranking change


rather than something specific to this infrastructure change


that you made.


Within some industries, would it be unfair for Google Shopping


Ads to run alongside the ads displayed


on the top of the search as it pushes organic search


results in smaller organizations further and further


down the page?


Do smaller organizations have a chance


in competing with larger companies?


I don't know.


I think this is almost like a philosophical question.


From our point of view, it's definitely not the case


that we're trying to kind of focus on big websites


or anything like that.


But from a purely practical point of view,


obviously, if you're a small company


and you're trying to compete with larger companies,


then it's always going to be hard.


And especially on the web, one of the things


that I've noticed over time is that in the beginning,


a lot of large companies were essentially incompetent


with regards to the web.


And they made terrible websites.


And the visibility in the search results was really bad.


And it was easy for small websites


to get in and kind of say, well, here's


my small website or my small bookstore.


And suddenly, your content is visible to a large amount


of users.


And you can kind of have that success moment early on.


But over time, as large companies also


see the value of search and of the web overall,


they've kind of grown their websites.


They have really competent teams.


They work really hard on making a fantastic web experience.


And that kind of means for smaller companies


that it's a lot harder to kind of gain a foothold there,


especially if there is a very competitive existing market


out there.


And it's less about large companies or small companies.


It's really more about kind of the competitive environment


in general.


And that's something where you kind of as a small company,


you should probably focus more on your strengths


and the weaknesses of the competitors


and try to find an angle where you can shine,


where other people kind of don't have the ability to shine


as well, which could be specific kinds of content


or specific audiences or anything along those lines,


kind of like how you would do that with a normal physical


business as well.


If an href tag contains both a text and an image,


which one helps Google to better understand what the link


page is about, the alt text of the image


or the visible anchor text?


Does the order in which they occur within the tag


also play a role?


So with regards to an image itself,


we would probably not find a lot of value


in that as an anchor text.


If you have an alt text associated with the image,


then we would treat that essentially the same


as any anchor text that you have associated with the link


directly.


So from our point of view, the alt text


would essentially be converted into a text on the page


and be treated in the same way.


So it's not that one or the other


would have more value or not.


They're essentially equivalent from our side.


The order doesn't matter as much.


I don't think it matters at all.


It's essentially just both on the page.


However, one thing I would not do


is purely for usability reasons just say, oh, well,


if the visible text doesn't matter as much


or is the same as an alt text, I'll


just remove the visible text.


Because especially other search engines


might not see it that way.


And it might also be for accessibility reasons


that it actually makes sense to have a visible text as well.


So I wouldn't just blindly remove it to a minimum,


but rather at least know that you're not losing anything


by having both of those there.


We're relaunching a customer website


and moving it from domain A to domain B.


We're reconfiguring redirects from domain A to domain B.


The question now is whether the page authority and page ranking


will be negatively affected if there are many backlinks


to domain A. And also future backlinks,


which will point back to domain A,


is the domain authority from the old domain


automatically inherited by the new domain,


even if there are lots of links?


So I think there are two aspects here.


On the one hand, if you're moving from one website


to another and you use the redirects to move things over


and you use the various tools that we have,


like the change of address tool in Search Console,


then that helps us to really understand everything


from the old domain should just be forwarded to the new one.


So that's essentially a situation


where as much as possible we'll take everything


from the old domain just applied to the new one.


The other aspect there is on a per-page basis,


we also try to look at canonicalization.


And for canonicalization, we look


at a number of different factors that come in.


On the one hand, redirects play a role,


things like internal linking play a role,


the rel canonical on the pages play a role.


But external links also play a role.


So what could happen in probably more edge cases, I guess,


is that if we see a lot of external links going


to the old URL and maybe even some internal links going


to the old URL, that we actually index the old URL instead


of the new one.


Because from our point of view, it starts to look like, well,


actually the old URL is the right one to show.


And the new one is maybe more of a temporary thing.


And because of this, what we recommend


when you do a migration from one domain to another


is not only set up the redirect and not only use


the change of address tool, but also go off


and try to find the larger websites that we're linking


to your previous domain and see if they can update those links


to your new domain.


Just making sure that everything that you do


is aligned with making sure that we


can focus on the new domain rather than on the old one.


OK.


We are having a problem with e-commerce facets, filters


that are getting indexed even though they're


blocked by robots.txt and have a canonical tag that point.


Is there a point in adding no index tags too?


So probably not.


The short answer, I guess, is if the URL


is blocked by robots.txt, we don't see any of those meta


tags on the page.


We don't see the rel canonical on the page


because we don't crawl that page at all.


So if you want us to take into account the rel canonical


or take into account a no index that you put on a page,


you need to make sure that we can crawl the actual page


itself.


The other aspect here is that oftentimes these pages


may get indexed if they're blocked by robots.txt,


but they're indexed without any of the content


because we can't crawl it.


And usually that means that these pages don't show up


in the search results anyway.


So if someone is searching for some kind of product


that you sell on your website, then we're


not going to dig and see if there's also


a page that is blocked by robots.txt, which


would be relevant because we already


have really good pages from your website


that we can crawl and index normally and that we can show.


On the other hand, if you do a site query


for that specific URL, then maybe you'll


still see that URL in the search results without any content.


So a lot of times what I notice is


that this is more of a theoretical problem


than a practical problem, in that theoretically these URLs


can get indexed without content, but in practice they're not


going to cause any problems in search.


And if you do see them showing up


for practical queries on your website, then most of the time


that's more a sign that, well, the rest of your website


is really hard to understand.


So if someone searches for one of your product types


and we show one of these roboted category or facet pages,


then from my point of view, that would be a sign that, well,


actually the visible content on your website


is not sufficient for us to understand


that the normal pages that you could have indexed


are actually relevant here.


So that would be kind of my first step


there is to try to figure out, do normal users see these pages


when they search normally?


And if they don't see them, then that's fine.


You can just ignore them.


If they do see these pages when they search normally,


then that's a sign that maybe you


should be focusing on other things


on the rest of your website.


Then a quick question about the Google Fabicon user agent.


Does it only crawl the home page?


How does it respond if the home page redirects to a subfolder


within the same domain?


Will it still download the Fabicon from the subfolder?


Yes, it should.


So the Fabicon that we pull should,


I believe we only do that kind of one Fabicon per domain.


I'm not 100% sure, but I think that's the case.


But if you redirect your home page


or if you redirect the Fabicon file


to a different part of your website,


we should be able to still pick that up.


Because practically what would happen here


is we would follow that redirect,


but we would probably still index it as your home page


anyway.


So from a practical point of view,


if you search for your name, probably


we would show the root URL, even though it redirects


to a lower level page.


Wow, lots of questions left, but also a bunch of hands.


Maybe I'll just go through some of the folks


with the raised hands.


And I have some more time afterwards as well,


so we can go through more questions too.


Let's see, Yogesh.


Hey, hi, John.


So the question is regarding the sitemaps.


And I'm working on the project of e-commerce.


So it has a huge number of URLs, let's say, lakhs of URLs.


So we have segmented with compression,


with sitemaps of sitemaps.


So from past month, I'm seeing that Search Console is not


able to fetch it, but the twist is the main sitemap.


The root sitemap is able to index,


and it's showing status success.


But the nested sitemap, sitemap 1, 2, 3, that is showing


couldn't fetch.


And the discovered URL is 0.


And I tried doing inspect of each sitemap.


They were showing the fetch is successful.


So just want your inputs on that.


What would be the scenario in this case?


I don't know.


It's hard to say.


So I think one of the things I have heard,


I haven't actually looked at it in detail,


is in Search Console, if you have an index sitemap, which


is kind of that sitemap of sitemap thing that you said,


then we don't show the details of the detail pages


directly in Search Console.


So essentially, to get that information,


you might need to submit the individual sitemaps


individually just so that you can look up


the reports for that.


That probably is something that you


don't need to do for all of these


and just maybe pick a sample of them


and see how that's going there.


The other thing is that what might help here


is to just change the URL of some of these detail sitemaps


and see if that works.


Because it might end up that we try


to recrawl some of these sitemaps.


And for whatever reason, we think we can't do it.


And we kind of remember that state.


So if you use a new URL for the sitemap file,


maybe it'll help us to understand, oh, we should


double check the sitemap again.


And that might help us to kind of reprocess those.


So I would try that for maybe a handful of them first.


And if you see that it works, then maybe it


makes sense to change the naming convention that you have.


I can't promise that it'll kind of result


in all of these sitemaps being crawled and indexed.


Because we do sometimes just not crawl individual sitemap files


where we think, oh, we're not sure what the actual value is


of crawling and indexing the rest of the content there.


So two things comes in my mind.


So one thing is like it's one month before everything


was working properly so that sitemap was able to crawl.


And the nested sitemaps were also working properly.


And if, let's say, I'm testing for one batch of sitemap


and checking how it is work.


But if I change that to, let's say, one sitemap,


I'm changing to the next folder and testing out,


isn't it the same data is already there with the Google


and just the same iteration I'm resubmitting and creating


confusion?


How do you mean?


It means, let's say, sitemap1 and sitemap2, sitemap3


is already there in the sitemap index file.


And then I'm just creating, let's say,


x5z.com slash sitemap1 and trying to test.


But in that sitemap1, it's already like 20,000


URLs are there if I'm testing out that already


that data is available with the Google, the previous sitemap.


I haven't actually tested it in Search Console,


or at least not recently.


But my understanding is in Search Console,


we don't show the details of the sitemap index file.


So if you submit them individually, then yes,


it's the same sitemap file.


And probably we've already processed that.


But because of the way that the Search Console reporting works,


we would be able to show you that information directly.


So it's more a matter of so that you see the information right


away rather than Google sees it.


And seeing it twice is not a problem.


OK.


Then it's fine.


I will test it out then.


Cool.


Thanks.


Joy.


Hi, John.


How you doing?


Hi.


Oh, yes.


I have a question, maybe a quick one, regarding product reviews.


We want to revamp our image guidelines.


And we want to be able to distinguish ourselves


in this space.


So we're thinking about different ways


that we can add original imagery,


some more scalable than others.


So I'm wondering from Google's perspective with vision


algorithms that you may use, is it OK or enough for us


to just use photoshopped images from product


that we find online?


Or do we need to go in and automatically update


all our images to include photos from our studios?


I think the guidelines that we have for reviews


or the recommendations that we have should really


be focused on unique photos that you create of these products,


so not like artificial review photos.


I don't think our systems would automatically recognize that.


But it's probably something that we


would look at, at least on a manual basis, from time to time.


So I don't know.


Just looking at the recommendations


that we have with regards to reviews,


it feels like it's not really in line what we're


trying to do there, where we're trying to really bubble up


reviews, where we can tell that someone is actually


testing this product in, I don't know, real life.


OK, so you're saying best option is


to just go with original studio photos from ourselves?


I think that would be ideal, yeah.


Awesome, thank you.


Sure.


OK, let me take a break here with the recording.


And I have more time for you folks with your hands raised.


So we'll get to you as well.


If you're watching this on YouTube,


thanks for sticking around to the end.


If you'd like to join us for one of these sessions


in the future, I post them in the community section


of the channel, usually a couple of days


before we do the sessions themselves.


Yeah, thanks for all of the questions.


And hopefully, we'll see each other again


in one of the future episodes.


Make your React web apps discoverable - JavaScript SEO

If you are building single-page apps, you might be using a JavaScript framework to do so.


One of the most popular frameworks is React.


React influenced a series of other frameworks and is also known for introducing a DOM extraction


layer and a domain-specific language to write component templates called JSX.


The virtual DOM that React ships with will come in handy when we are trying to make sure our


users get to the content as fast as possible, as we will later see. So, what do you need to do


to make sure your React apps are discoverable in search, and how do you handle SEO?


Let's start looking at a typical React app. While your React projects might be different in the


libraries and components you use and their content and complexity, there's a lot that


all React applications roughly have in common. For this video, we will start by creating a fresh


app using the create-react-app helper utility. In this video, we are using create-react-app version


2.0.2. And first, we create the application skeleton using this command. This created a


very basic React app for us to work with for this video. So, let's take a look at it by running the


suggested development server we can install via npm as well. So far, our application only has one


piece of content. Let's make this more interesting by adding another route and another piece of


content. First, we install react-router-dom. Now that we have a router to handle different pages,


we can add a component for another page. We will be using a placeholder API to fetch a few posts,


so we will call this the posts list component. When this component is ready to be displayed,


we fetch the data from the API endpoint and store it, truncate it a little bit, and save it in the


component's local state. In the render method, we map the posts to list items containing the


post title and then render all the posts in a list. Moving on to our app component, we import


the router along with the switch and route components and change the render function so


that we render the posts list component instead of the header and logo. This is a good time to


throw out the default CSS as well and then check our app in the browser. We now see a list of post


titles as we would expect. So, similar to the posts list component, we can now add a component


called post detail to show one individual article with the title as the headline and the content


below it. It is very similar to the post list except we only fetch one post from the API and


we take the id from the post from the URL parameters. Let's add this component to our router


setup. Last but not least, we will set up our post list component so that it provides links to each


of these article pages. We will use the link component from react router for this. Now we build


our project and deploy it somewhere. With our sample application working, we can look into


improving the discoverability of it so users can find our content more easily on the web.


A good first stop to check how we are doing is the mobile friendly test. We just put the URL


into the field and we see what Googlebot sees when crawling our pages. We see that all our content


shows up and that the links are a tags with an href attribute which is what Googlebot will pick


up and crawl. So, our content will be indexed but we can do better here. First of all, we want to


make sure to give meaningful descriptive titles and descriptions to all our pages. For the home


page, that can be the name of the blog as the title and what this blog is about as the description.


For the individual post pages, we should set the title to the post title and the description to


the most important takeaways or summary. Well, we don't have that information here, so instead,


for the description, we will just use the first 100 characters from the blog post.


Note that this is an arbitrary length. The meta description does not have any length restrictions


but putting the entire post content into it isn't very useful either. To do this in React, we can


use the helmet component that we installed from npm or Yarn and then add it to the render method


in our components. We can configure the title, the canonical URL, as well as any meta tags.


Now, if we check one of the post pages in the mobile friendly test, we see the title is the


same as the blog post title. This is great because it helps users searching for a specific topic


to see how the blog post is relevant for their goal. The same is true for the description. We


show a snippet from the description in the search results to help users make an informed decision


on which result is the best for their needs. Alright, let's look at a more complex topic.


As we explained in a previous video, when we crawl your pages, we defer the execution of


JavaScript to a later moment when resources for rendering are available. This leads to two ways


of indexing. The first one that isn't running JavaScript and then potentially a second one


where we execute JavaScript to see the content that depends on it. If your content changes very


frequently or your site is very large, you might want to make sure to have your content be available


in the first wave of indexing, so without JavaScript. It is also a good idea to not depend


on JavaScript too much as other bots might not execute it at all and you might find that you gain


a better performance for users as well if you do not rely on JavaScript to display your content.


Luckily, React has a few ways of supporting us with reducing the reliance on JavaScript


by doing server-side, hybrid, or dynamic rendering. In our example, the content won't change that often,


so we can use a solution like React Snap to pre-render our content and deploy the resulting


static files. Let's install React Snap and take it for a spin. Here it is crawling all our post pages.


If we now look at the source of our page in the browser, we can see that these files contain our


content even if JavaScript hasn't been executed. Setups like this are not always this easy to


accomplish, but there's plenty of documentation available to find out more about the other


possibilities for server-side or pre-rendering with React. We could also decide to only serve


the pre-rendered version to bots like Googlebot. That would be called dynamic rendering, in which


case do take a look at our dynamic rendering documentation as well. I suggest you also take


a look at the other videos of this series for more great tips on making your app visible and


successful with Google search. We have a couple more episodes about JavaScript SEO coming out,


so stay tuned and please tell us what you think so far in the comments below.


HTML tag for site ownership verification - Google Search Console Training

In this video, I'll go through the steps you need to follow


to verify your website in Search Console using the HTML tag method.


In order to verify your website with an HTML tag,


you'll need permission to edit the source code of your website's homepage.


After you click Add property, add your website URL


under the URL prefix option and click Continue.


Expand the HTML tag section to find the meta HTML tag with a personalized key.


Copy the tag and paste it into your homepage head tag.


Make sure not to change the text.


Verify that the tag is present on your live page by visiting your homepage


and checking the page source text to confirm that the tag is present.


Once you complete this step, click Verify in Search Console.


If you perform the steps above correctly, verification should work immediately.


If it doesn't, click the full details link to learn about potential errors.


You can find out more about those errors and how to handle them in the Help Center.


I hope you found this video useful.


For more videos like this, check out the Google Webmasters YouTube channel.


Google Search News (June ‘21) - Search Console Insights (BETA), RSS, and more!

Hi, everyone, and welcome back to the Google Search News.


I hope life is treating you reasonably well wherever you are.


I'm your host today, John Mueller, here from Google Switzerland.


With this show, we want to give you a regular summary of what's been happening around Google


Search, specifically for website owners, publishers, and SEOs.


If you find these useful, which I hope you do, and if you'd like to stay up to date,


make sure to subscribe to the channel.


It's good to see you all again.


I have a bunch of updates for you today.


They include Core Web Vitals, Search Console, RSS, and a lot more.


So without further ado, let's jump right in.


Let's start off with Core Web Vitals and the Page Experience Ranking Change.


We mentioned this last time too, so I'll be brief.


This is a new ranking change which takes into account the user experience of web pages.


It includes elements like HTTPS, as well as aspects around speed and responsiveness.


The announced changes, such as the updated Cumulative Layout Shift, or CLS for short,


are now live in all of the testing tools and in Search Console.


Keep in mind that the data in Search Console is based on the Chrome User Experience report


data, which is aggregated over 28 days.


Another update is that the ranking change has just recently started rolling out.


It's aimed at being fully rolled out for all URLs by the end of August this year.


In practice, this lower update means that you would usually not see your whole site


shift in search visibility from one day to the next.


Instead, you'd see a gradual change over time.


This also means you can continue to work on improving things if your site isn't doing


as well as you'd like in terms of these metrics.


There are a lot of details and specifics that folks might be interested in.


For that, we have an expanded FAQ in our help forum.


I'll link to all of this below.


And now over to Search Console.


Google Search Console is a fantastic way of finding out how your website is doing with


regards to Google Search.


After verifying your website there, you can get lots of information.


For example, where and how are people finding your website?


Are there any issues you could resolve to make your site easier to find?


All of that and more.


Last time, we talked about regex.


That functionality was so well received that the team expanded on it and has recently launched


negative regex.


It gives you all of the information that doesn't match what you provided.


And that's a positive thing.


For example, you could use a regex to cover all variations of your brand's name and then


see all queries that didn't mention the name.


This is a way of splitting branded search terms from unbranded ones.


So plus one for negative regex, if you ask me.


Then we recently launched Search Console Insights.


Search Console and Google Analytics give a fantastic amount of information, but it can't


be a bit too much if you don't have time to regularly understand what it all means.


Search Console Insights distills this information down and gives site owners a nice and easy


overview of what's been happening with regards to their site.


If you work as an SEO or for an agency, this is a neat way of helping any less savvy site


owners to see how your work has helped their site.


It's still super fresh, so I'd recommend checking it out.


Think about how you might use it and point folks there who don't want to research numbers


in the mountains of data in Search Console and Analytics.


Let's move on to RSS.


RSS is a name of a specially formatted XML file or feed.


It's commonly used for syndicating information.


It's been traditionally used to provide information about new blog posts and articles to subscribers.


Some might say RSS is not new, but it was recently mentioned at Google I.O., the big


Google Developer Conference, so let's take a brief look.


We use RSS feeds similarly to sitemap files to help us optimize our crawling and to make


sure we don't miss any updates on a website.


This remains unchanged.


Additionally, at I.O., we mentioned that we're going to start using them in the context of


a follow button in Chrome.


This would allow users to receive updates from your website directly in their browsers.


This is independent of Search.


This Chrome experiment will first be visible in English in the US in Chrome on Android.


If your website is already producing RSS feeds, you may already be prepared.


As usual, these feeds should be linked from the head of your pages.


One aspect to mention is that the title of your feed may be visible to users, so it's


a good practice to check that it's a useful representation of what the feed provides.


To speed up delivery of updates, we recommend using WebSub, which we also support for Search.


I'm curious to see how this works out.


I hope we have more news to share on this once it expands from the experimental stages.


So many things to think about when it comes to making websites.


If you're just getting started on SEO or if you're keen on diving into specific topics,


I recently ran across a fantastic website by Aleta Solis, which can help you out.


It's called learningseo.io.


It's an amazingly comprehensive way of getting an overview of all of the information already


published about SEO.


You can find a learning path that works for you or just be amazed at how many fantastic


articles folks have written on certain topics.


Aleta does a fantastic job of curating this, so the information you find there will be


useful and relevant.


I recommend bookmarking that and checking in when you're keen on working yourself into


another facet of SEO.


And now for a bunch of smaller updates.


The latest core update was announced early June, and there's another planned for July.


These are updates to the core of our search system, so you may see changes in how your


site is shown in search.


We have a blog post with more information about these updates, which I'll link to below.


Google IO took place at the end of May.


There were lots of sessions with videos.


You can find them all on the Google IO website.


At IO, we also announced MUM and other machine learning technologies.


SEOs have asked if this is the end of SEO.


The shorter answer is no.


These modern tools make it easier for you to do your jobs, and the specific activities


will evolve.


For example, there will be less, fewer, or reduced need to include all possible synonyms


or words that mean the same or similar.


We recently launched a shared location for all content policies around search.


This should make it easier for you to find any specific policy that you're looking for.


And then we recently unlaunched the generic rich results search appearance type in Search


Console.


This makes it a bit easier for you to focus on the bigger picture.


And last but definitely not least, we're doing a survey.


You should find the link right here.


The surveys are about our video content, and I'd really appreciate to hear from you to


find out more about what we could do to better support you and your work.


That's all for now, folks.


Thank you for watching this episode.


It always feels good to me to look back at the last two months and pull together the


things that really stood out.


I hope you found it useful and insightful as well.


If there are elements you'd like to have covered differently, please let me know in the comments


below or drop me a note on Twitter.


Looking around, it seems like summer is finally coming.


Well, at least on this side of the planet.


While it's impossible to know for sure, it feels like things are also slowly getting


a little bit better in terms of the coronavirus.


I hope the outlook is looking positive for you, your family, and those around you as


well.


If you'd like to see more of these episodes, make sure to subscribe to the channel.


We're regularly publishing new videos here.


I'm sure there's something coming up which you'll appreciate as well.


And as always, I look forward to seeing you all again in one of the future episodes of


Google Search News.


English Google Webmaster Central office-hours from June 23, 2020

All right, welcome, everyone, to today's Webmaster Central


Office Hour Hangouts.


My name is John Mueller.


I am a webmaster trends analyst at Google in Switzerland.


And part of what we do are these Office Hour Hangouts,


where people can jump in and ask their questions


around their website and web search.


Looks like a bunch of questions were submitted already.


But as always, if any of you want


to get started with a question of your own,


feel free to jump on it.


Or not, that's fine, too.


Maybe something will come up along the way.


Hey, John.


Hi.


Go for it.


Go ahead, Sanjay.


OK, so John, actually, I have a question.


So I am seeing that still my site primary caller is a desktop.


So it's still not changed.


We have a huge number of pages, around 20 million traffic.


So it's a big concern for me.


So how to solve that?


And what is the problem that you're seeing?


In my primary caller in the Webmaster tool,


it's showing as desktop.


Index as desktop?


Yes.


Yeah.


So I mean, from a practical point of view,


that doesn't mean that there's anything broken.


We're indexing the site as it is.


I think what I would really look into


is whether or not the mobile and the desktop pages


are really the same with regards to the content,


and that the embedded content that you have on those pages


is similar.


So some of the things we've sometimes


seen when people check for the text, if the text is OK,


if the structured data is OK, then sometimes there's


still an issue that maybe the headings are not


actually visible on mobile.


Like if you don't have any headings on mobile,


if they're just marked up to be more visible text,


but not using the heading tags, that


might be something that we notice.


Another thing that's really common


is if you have thumbnails that you use as kind of links


to related items on your pages.


If you have a larger number of thumbnails on your desktop


pages than on your mobile pages, then our systems


might think that you're missing some important images.


Probably those thumbnails don't matter so much,


but it's something where our systems might say,


well, there are 20 images on desktop,


and there are only 10 on mobile.


Maybe there's something that the server would


want to take care of first.


So those are, I guess, kind of the more tricky topics


that are kind of hard to spot automatically.


There's also the issue of sometimes


if we think that a page should be a landing


page for an image or a video, and on mobile that image


and the video is not so prominent,


then we might think that actually this


is not such a good video or image landing page anymore.


And that's also something that our mobile first indexing


systems will look at and say, well,


it looks like this would be problematic for image search


or problematic for video search.


Therefore, we want to be a bit more cautious here.


So those are kind of the things I would take a look at.


I don't think it's the case that you urgently need to move


or that you will have any kind of a ranking or search


advantage if you move or you get your site moved


to mobile first indexing.


But it's good to take care of these things.


SPEAKER 2


So I know there is no magic formula for ranking.


However, should we be concerned about the search console's


error and be obsessed with it or still focus


on the user experience or the content on the site


as one of the factors?


So how do you mean focus on the search console?


So we are going through a major redesign on the site,


and we probably have 3,000 pages or so discontinued.


And we are running through a list of 404 errors and all.


And then there is also core vitals showing up


a little bit where there is a serious issue and stuff


like that.


So should we be obsessed with it?


Or it's like, yeah, eventually it's going to flesh out


and kind of hope that things will settle in terms of ranking.


It's really hard to say offhand because there


are so many different kinds of issues


that can be reported on in Search Console.


And not all of them are immediately critical issues


that you need to take care of right away.


So for example, if we can't reach your site,


if we can't index your pages, that's pretty critical.


However, if it's a matter of, well, your page is not


as fast as it could be, then that's


something where it's like, it's OK.


It's good to take care of these things


because they are things that users might see.


But it's not the same level of criticalness


as we can't index your pages at all.


So with regards to that, it's something where you almost,


I guess, need to be able to take a step back and think about


what is the effect of this issue that I'm seeing reported?


And is that effect something that


is critical for my web presence at the moment?


Or is that kind of a nice to have?


Or is that something that's maybe worthwhile cleaning up,


I don't know, when you have a bit of downtime


and you don't need to worry about urgent issues anymore?


So that's kind of the way that I would look at it there.


I don't think there is, by definition, any automatic way


to recognize which of these issues


are the most critical ones.


We try to bubble up the things that we think are critical.


But there are so many different things that can go wrong.


So it's kind of hard to say what the most critical thing is


you should take care of first.


OK, thanks.


OK, let me look at some of the questions that were submitted.


And as always, if any of you have questions


along the way, or comments, or anything to add,


feel free to jump on in.


The first one is, let's say I have two strong URLs


about cheese in my website.


One is an e-commerce page where you can buy cheese.


The other is a complete guide about cheese.


So two different pages talking about the same topic,


but both really relevant.


What's the best practice for internal linking?


Is it OK to link both pages using the same anchor text


cheese?


Or should one be linked differently?


What are some suggestions?


So essentially, internal linking helps us, on the one hand,


to find pages.


So that's really important.


It also helps us to get a bit of context


about that specific page.


And we get some of that through the anchor text


from the internal linking, and some, of course,


from understanding where these pages are


linked within your website.


So with regards to that, thinking specifically


about the anchor text here, I don't


think you need to do anything specific there.


If you're already linking to those pages,


if you're using a reasonable anchor text for cheese,


in this case, that sounds perfectly fine.


I don't think you need to change the anchor text to be,


buy your cheese online here.


And it's like the ultimate guide to all types of cheese here.


It's something you could do if you wanted to,


if you think it makes sense for your users.


But it's not something where I think


you would see a visible effect in search.


So I hope that kind of helps in that regard.


I think it's always kind of a tricky situation when


you have multiple pages on the same topic,


or multiple pages for the same keyword,


in that people often worry about cannibalization, which


essentially means that you have multiple pages ranking


for the same term.


And maybe you would be ranking better


if you had just had one page ranking for that term.


That's definitely something legitimate to think about.


And from my point of view, I generally


prefer to have fewer pages rather than more pages.


But these are completely different topics, completely


different types of pages.


So it feels perfectly fine from my point of view


to have two pages that are kind of different like this.


And oftentimes, you will have this situation


where you have informational pages on a topic,


and maybe transactional pages on a topic, maybe something


in between, maybe some category pages that


are also on this topic.


And that kind of setup is completely normal.


And that's something that our systems try to work with,


where we try to show the relevant page in the search


results.


OK, the best experts at a country level


review our articles from the Help section of our website.


We want to be sure that not only users, but also Google


appreciates it.


What advice can you give us?


So essentially, when it comes to things like this,


where you're saying, well, we do a lot of background work


to make sure that the information that you get


is really valid, it's correct, it's relevant,


then that's something that you essentially


need to show to users primarily.


And our systems will be able to pick up on that over time.


It's not that there's a specific structured data element


that you can put on a page and say, well, my page is correct,


or my information is correct.


It seems like you should trust me.


I did a lot of work.


But really, who you want to kind of persuade is the users.


And if we can tell that there's a lot of background


information there, if our systems recognize


that this is clear to users as well, then that's something


that we will try to reflect in search.


So from that point of view, I wouldn't


focus so much on structured data or meta tags


or anything like that.


But really, make sure that it's clear for users.


And then we should be able to pick that up too.


I think the next question is kind of related.


We have a health section on our news website.


We want to implement a review by schema markup


on the pages.


It looks like this parameter is only for web page descendants,


medical page, for example.


Should we implement review by in the news article that


works right now, or should we change news article


to medical page for all articles in the health rubric?


So I think this kind of goes into the same direction.


Essentially, you need to make sure that these things are


clear for users and not primarily focus


on what search engines would see with regards to structured


data.


In a case like this, if you have news content,


then I would mark it up as news content.


That's kind of the technically correct way


to do that with the schema.org markup.


I believe there are also some elements in Google Search


that specifically watch out for news article markup


to recognize that actually this is a news article, which


from my point of view would kind of make sense.


If there is no way to add reviewed by schema markup


to news article pages, I don't know,


then there is no way to do that.


So that's something where if you have news articles


and you can't add another kind of markup specifically


for something else, then that's something


you can't do purely from the schema.org setup.


It might be that there are alternate variations that you


can do for this kind of site.


I don't think when it comes to search


that you would see any difference if you kind of


implemented the markup in a hacky way


or didn't implement it at all.


Most probably our systems just watch out for those elements


that we explicitly have documented in the developer


documentation, where we have all of the different types kind


of documented that we use in search.


That's specifically what we watch out for.


And everything else is a little bit more like, well,


you're providing a bit more context.


So in a case like this, I wouldn't worry too much


about being able to add reviewed by markup


if that's technically not something


you could do when it comes to news article pages.


Is it against Google guidelines if you cloak


but show almost the same thing?


Let's see.


For context, due to a data contract,


I can't show an exact results page due to scrapers,


so they have to be noindex.


However, I can show cached results.


So if you're looking for a four-bedroom house as a user,


you'll technically be served a different page than a search


engine.


So I think when talking with the engineers or the quality


team about topics like this, they generally say,


well, it should be exactly the same.


From a practical point of view, there


will almost always be differences


with regards to what Google sees and what users see,


especially when it comes to content that's very dynamic.


So for example, it might be that you have a news page,


and when Google crawls, it has one article on top.


And then an hour later, when a user goes to that page,


there's a different article on top.


Technically, it's slightly different content.


From a practical point of view, the reason for the page


is essentially just the same.


So it's not something where I think


our systems would watch out for that and say,


oh, this is against the webmaster guidelines.


With regards to the web spam team,


they would also look at this and say, well,


it's essentially the same page.


Maybe it's kind of the data that's


shown is slightly different with regards to what is cached


and what is served live.


But it's essentially the same page.


So that generally would be fine.


Usually, the place where we run into more problems


with this kind of a setup is when


you have a technical issue on the part of your site


that is only visible to Googlebot.


Then it happens every now and then


that something kind of severely goes wrong there.


Maybe there's a server error showing,


or maybe the cache is super stale,


and it's from last year and not from this week.


Those are the kind of things where, as a site owner,


you might not notice them immediately.


Because when you look at the page,


when you kind of use your monitoring tools


to access those pages, you see the live version


that a user would see.


So often, the issues that come up here


are more about, well, if you implement it in a bad way,


or if something breaks along the way and you don't realize it,


then you could be shooting yourself in the foot there.


It's not so much that from kind of the web spam guidelines


or the web spam team that they would look at this and say,


oh, the text is slightly different on this version


than on that version.


Therefore, we have to take a manual action.


I'm getting this error in Search Console.


Reference, AMP URL is not an AMP.


What's causing this error, and how can I fix it?


It's essentially impossible to say without knowing the URLs.


This is one of those cases where I would recommend going


to the Webmaster Help Forum and getting some help


from other folks who have a little bit of experience


in Search Console.


Explicitly mention the URLs that you're seeing.


Maybe take a look at the URL.


The URLs that you're seeing, maybe take a screenshot


as well so that people can take a look and see,


is there really an issue here, or is this something


where maybe Search Console has a weird error that's


a bit confusing?


Maybe it's also something where there's an error on our side.


And the folks in the Webmaster Help Forum,


they can escalate these issues to us.


So that's kind of the direction I would head there.


My guess is that, just purely from the text,


that you have a page that looks like an AMP page,


but for whatever technical or theoretical reasons,


it's not a valid AMP page.


And that's something that happens every now and then.


It could be that maybe there's a script tag that's


kind of lost on the page.


Maybe something briefly went wrong with a plugin


that you're using.


All of these things can happen.


But the folks in the Webmaster Help Forum


can generally help to kind of narrow things down.


Is there a concept in Google of types of pages in a site?


In other words, on an e-commerce store,


for example, would a category listing page be identified,


product detail type pages be identified?


Is there a concept of this, perhaps


for understanding crawling or canonicalization?


I don't know if we'd have an explicit kind of concept


for this.


It's very likely that we would have different kind of signals


that we try to compile for different types of pages


to figure out how and when we should show this page


appropriately.


But I don't know if we would have something kind of as


explicit as this, where we'd say, oh, this


is a category page on an e-commerce site,


and it lists shoes in the category,


or if it's something that's kind of more abstracted and more


useful just for our algorithms.


Part two is, if there is such a thing,


does it pay to keep the basic template of common page types


similar to help Google to understand, for instance,


products in a div, ID equals my product list,


or whatever the structure is, common across category pages?


So sometimes that does help us to understand


which pages belong together.


So that is something that we try to do


when we crawl and index a website,


is to figure out what pages are kind


of a part of the same group.


And for the most part, we do that with regards


to crawling, to understand, well, these


are all product pages.


I mean, not specifically that we'd


say these are product pages, but kind of this URL pattern


looks like a kind of set of pages that are all


very similar.


And based on that, we can prioritize our crawling


a little bit.


So that's kind of the direction that we go there.


It's less about kind of the ranking side


and understanding exactly what page type it is,


but more understanding, well, this set of pages


belongs together in one group.


And maybe we've seen 90% of the pages in this group


are noindex, for example.


Then if we see new pages that belong


into the same group with a similar URL pattern,


then probably we can say, well, it's


very likely that these new pages will also be noindex,


even without us crawling them.


So we could theoretically deprioritize


those pages a little bit with regards to crawling.


And similarly, if we recognize that actually this group of


pages is really important for this website,


if we find new pages that fit into that group,


then that's something we can say, well,


maybe we should prioritize these a little bit with regards


to crawling and pick them up a little bit faster.


It's less about the HTML exactly on the page


and really more about the general kind of bucket of pages.


And I think for the most part, we organize this by URL patterns


where our systems automatically create


a little bit of an understanding of the URL patterns


within a website and understand which parameters are relevant,


where part of that you see in Search Console and the parameter


handling tool.


I started a blog last year following all your tips


on that blog.


Within a few months, I was getting 800 daily visitors


from Google.


And that's huge for me.


But sadly to say, for some reasons,


I was inconsistent on the blog for three months.


After my inconsistency on the blog,


I was getting zero visitors from Google.


But after realizing my mistake, I started again in January


and migrated to a new hosting provider.


Now I'm writing posts and building links daily.


I'm still only getting 10 to 30 visitors from Google.


What should I do?


So it's really hard to say what exactly you've been seeing.


My general sense is that the differences in the traffic


that you're seeing from the search results


is not due to your consistency on the blog


or inconsistency on the blog.


But rather, that's something where our systems have


tried to figure out how relevant this page is


in the context of the overall web.


And that can change over time.


And it can be such that you might be getting 800 visitors


a day at one point, and our algorithms update,


and we understand, well, maybe this


isn't as relevant as we thought it was,


and you'd get fewer visitors afterwards.


It can also happen that our algorithms get updated


and say, well, this is a really fantastic page.


Why are we only sending 800 people here?


We should be sending thousands of people here.


And that can also happen.


So that's kind of where my guess is this is more of an algorithm


change thing and less a matter of you being consistent or not.


The other thing that I'm kind of reading between the lines


here is you're writing posts daily


and you're building links daily.


And it kind of makes me wonder what you consider


to be building links, for example.


It's very easy to get sucked up into the link-building world,


and essentially you're sending emails to everyone,


or you're dropping comments on other people's blogs,


or you're trying to find those sites where


you can drop your link and it's not a no-follow


and everyone goes there or automatically goes there


and drops their links there.


It's very easy to get sucked into that


and to spend a lot of time building links, in a sense,


but building a lot of links that essentially our systems ignore.


And that could be time that you could be spending


to improve your content overall, which


is something that our systems will kind of appreciate more


over time.


So that's something where I think it's important


that you find a balance between promoting your work


and doing your work, but primarily you


should be doing your work really well


and not spending that much time promoting it altogether.


Let's see.


I think the Search Console Inspector tool,


we talked about that briefly.


What makes posts eligible for Google Discover?


According to my Search Console reports,


my posts were getting featured for some time,


but then it completely stopped.


So Discover, I think, is pretty cool in that it shows


your content to people who are not explicitly looking for it,


but who we think might have interest in that topic.


So overall, I think that's pretty cool.


On the other hand, that also means


that our systems have to be a little bit more


cautious with regards to the content


that we promote there, because we


realize that people aren't explicitly


looking for that content.


So we should maybe be a little bit more cautious there


than we otherwise would elsewhere in Search.


With regards to eligibility in Discover,


I believe the Google News publisher guidelines


apply to Discover.


Not 100% sure.


I think that's what we link from the Discover Help Center page.


So that's kind of the baseline eligibility.


But as I mentioned, we do try to make sure


that the content that we provide through Google Discover,


or that we link to, is something that we can really


fully stand behind.


So my guess is if for a while your posts were being shown


in Discover, maybe they're not being shown as much


anymore, then our systems are a little bit more cautious


with regards to whether or not this is really fantastic


content that we should be promoting in Discover.


That also means that probably you just


need to step up a little bit more to kind of reach


that target again.


It is really hard to say what exactly you should


be doing differently, because there is no query assigned


to Discover.


But it is something where working with your audience,


the people who are going to your site already,


and figuring out what is it that really interests them,


and how can you provide that in a way that


gives a lot of value to everyone who looks at that content,


that's something you can try to tweak over time.


Is using schema to link between other related schemas


on the same page, between schemas on pages of the website,


basically creating a network just like internally linking,


does that have any benefit for Google's understanding


of a website or any benefit to SEO?


I don't think so.


So primarily, when it comes to structured data,


we try to use the structured data


that we can use for specific search features.


We have that explicitly documented in the developer's


guide.


So that's kind of our primary use case of structured data.


Additionally, we do try to look at the structured data that's


otherwise on the website to understand a little bit better,


is there any context that we're perhaps missing here?


Is there any additional information


that helps us to better understand


how this particular page stands within the website?


But at the point where you're referring to referring


from one type of not directly supported structured data


to other pages with not directly supported structured data,


at that point, I think you're probably in the area of, well,


our systems see this, but we can't really


do anything useful with it.


Even if we can understand the relationships


between those specific items that are marked up


with structured data, it's probably so vague


that it's really not something that


would have any visible effect in ranking at all.


Do you think that errors in a domain


might affect the web positioning of subpages,


for instance, in relevance?


It's really hard to say without knowing what kind of errors


you're looking at there.


So when it comes to technical errors, for the most part,


I don't think that would apply.


If you have some pages that are accidentally 404,


I don't think that would apply at all to anything else


on the website.


The one exception that I can think of


is if it looks to us like this domain is not in use at all,


then that might be something that would apply


to the other subdomains there.


So for instance, if the www and the non-www version


of the domain, if they both just don't resolve


to a normal web page, then our systems might think,


well, maybe this website went down.


Maybe it's no longer accessible.


And at that point, they might apply that


to the other subdomains as well and say,


well, it looks like the whole domain is gone.


Maybe we don't need to index all of these subdomains either.


But that's an extremely rare situation


that I ever see something like that.


So that's usually not something that people worry about.


Because if it looks like your website is down,


then that seems like a more general problem


than is like, how does it affect my relevance in the search


results?


Other types of errors, like for example,


if you have structured data issues on a website,


if you have issues with regards to speed


or with regards to even simple things like, I don't know,


grammar or spelling on a website,


technically those are errors.


But that's not something that would


affect the other pages or the other domains,


subdomains on the website.


So that's generally not something


that I would see as being an issue.


So if you have a setup with different subdomains,


obviously making sure that your website doesn't


look like it's down is kind of a good thing.


But otherwise, I wouldn't worry too much


about how you managed the main domain versus the subdomains.


What could be the cause behind Google activating a website's


review snippets enhancement followed by massive upwards


and downwards fluctuations in clicks and impression


and a deactivation one month later?


We've been experiencing such quick switches since last year


after having your review snippets activated constantly


for years.


I don't know.


It's hard to say from just this question.


It sounds like there might be some kind


of a technical issue involved here.


It might also just be that our algorithms are


a bit on the edge with regards to how to treat this website.


But if you've been having this kind of on and off


the situation kind of on a weekly or monthly level,


then that feels kind of weird, like something


that probably shouldn't be happening,


where it might be useful to have your domain or the queries


where you're seeing this.


What you could do is maybe post in the Webmaster Help


Forum to see if other people have anything obvious


that they can spot there.


Otherwise, you can also send me a note on Twitter,


and I can take a look at that too.


I'm doing SEO on and off page for some packers and mover


websites approximately two months.


Backlinks and domain authority have increasing right now.


I don't know.


I've been increasing, I'm guessing.


But pages are not ranking.


What should I do to rank all of my pages?


So I think, first of all, we don't use domain authority


in search.


I think that's kind of a misconception that's


out there that comes up every now and then.


I think it's really cool to have these kind of tools


out there that help you to understand


how my site kind of fits in with regards


to the rest of the web.


But we don't use domain authority.


So it's not something where I would focus on domain authority


and say, this is what I need to do, but rather think


about your website on a whole and think


about what you should be doing overall to kind of improve


your website.


The same thing applies to backlinks here,


as I mentioned before, with the other case of someone who


is just building backlinks.


It's something where it's very easy to get kind of pulled down


into this world of building backlinks, where you have


to drop your links everywhere to increase the number of links


to your website.


And that's definitely not what our algorithms are looking for.


If you just drop random links on other people's blogs


or drop links in forums or in other places


where it might get picked up, that's not something


that we would look at and say, oh, this


is a sign of a high-quality website


that we should be showing more.


That's not how it works.


So that's something where, if you've


been working on this website for approximately two months,


then I would recommend maybe spending a bit more time


on the website overall, especially if you're


active in a very competitive area.


So I don't know how packers and mover websites


are where you are located, but I've


seen that as one of the more competitive areas.


And coming in for two months and spending a bit of time


to build up a website and drop some links in various places


and focus on domain authority, I don't


think that would be enough to actually make a dent there.


So that seems more like something


where you'd have to come up with a long-term strategy


and figure out ways that you can differentiate yourself


from other people, from all of the other competitors that


are out there, and really show the additional value


that you provide and to make sure


that Google and other search engines


are able to recognize that additional value.


And we would recognize that not by random backlinks that


are dropped or domain authority, but rather by looking


at the whole picture.


All right.


Can you share any more information


about the possible bug with Disqus comments


not being indexed?


I can see some pages are definitely


having their comments indexed while others aren't.


I've seen this for a long time with Disqus,


and I thought Google was selectively


indexing comments from certain pages based on quality.


Martin split explained the other day that there might be a bug.


Since many sites use Disqus, I would


be great to know if more Disqus comments are going


to be indexed moving forward.


Martin said this could happen.


Well, I guess if Martin says it could happen,


then that seems like a pretty strong case.


I personally haven't looked into this.


I don't know exactly what is happening there.


I know some sites use Disqus comments extensively.


Some sites implement them in a way


that they're cached within the static HTML that's


served with the website.


It makes it, I suspect, a bit hard to understand exactly what


is happening there if there are so many different ways


that you can embed these comments within a website.


So that, I don't know.


It sounds like from this comment and from, I think,


the brief exchange I saw on Twitter,


that there is something that we could


be doing to improve indexing of these comments.


But I don't know how big of an effect


that would be if that's something where we can just


index a little bit more or if it's kind of a really big jump


that we can suddenly index a lot more.


I don't know.


We'll see.


Barry, you have Disqus comments on your site, right?


BARRY SCHWARTZMAN-SMITH Yes, I do.


It seems to index fine.


So I don't know.


OK.


But Glenn did a blog post months and months ago


saying how some sites Google had no problem indexing


these comments on and some sites they don't.


And he has a real long story on that.


I'm not sure.


I haven't looked into it either.


But maybe there's just the way that those publishers have


implemented Disqus that it's kind of being blocked.


I kind of want to block mine.


I used to block it.


Then you guys got better at indexing.


So now I have to find another way to block it.


Yeah.


That's always a struggle.


A bunch of people come to us and it's like,


how do I get myself indexed?


And other people come to us, how do I get it unindexed?


Well, you can't win, right?


Always something to do.


It's good.


Yeah, I don't know.


We'll see what Martin comes up with.


I know he's been in touch with the rendering and the indexing


team on that.


So it sounds like there is something.


But I don't know how big of an issue it is.


It was just a weird response because Martin was like, yeah,


maybe there's an issue on our end.


So he basically said there is an issue on our end.


So I mean, unless he just took a quick look


and was like, maybe there is an issue, but I wasn't sure.


So if you could follow up Martin and have him tweet something


out that would make for a good headline, that would be great.


OK.


Something that people can comment on, yeah.


Yeah.


I think it's sometimes tricky with a lot of these JavaScript


implementations in that when you test them manually,


it looks like things are working or it can


look like things are not working.


But our indexing systems try to be a little bit more resilient


and catch things that even the offhand testing


tools don't notice.


So it's sometimes a bit tricky in that regard.


But we'll see what comes out.


Yeah, I mean, I wrote about it.


And then I put some weird long phrase in my comments area,


waited, I don't know, 20 minutes,


did a search for that weird long phrase.


And you guys found it.


So clearly, at least on my site, you


seem to be indexing it well and fast.


So if you could slow that down, that's something to know.


I mean, clearly, it's a sign that the comments on your site


are high quality.


And we need to take those up quickly, right?


Yes.


OK.


We'll see what happens there.


Stay tuned.


Is it normal for the last crawl of a page


to be the first time Googlebot has indexed it?


I'm using React plus Next.js with server-side rendering.


And I think there might be a problem for Googlebot


in reaching my site.


When I request a recall, it gets a spike in impressions


the first day, and then it falls off.


This happens for all pages.


Am I missing something from a technical point of view?


I don't know if there's anything kind of by default that's


wrong in a case like this.


It's something where just looking


at the relevance or the visibility of a page in search,


that can be kind of normal in that when you publish something


new, then suddenly it's very relevant and new for everyone


to be shown in search.


And then over time, that kind of drops off.


So that could be kind of normal.


However, it sounds like you have some very specific technical


specifics that you're seeing there.


And that feels like something where


maybe it would be worthwhile to post some clear example URLs


and maybe even some screenshots of what exactly you're seeing


there with regards to the date of the crawling


or the last crawl or the first crawl of these pages.


So just purely judging from the names


that you're using there with React, Next.js,


server-side rendering, that seems totally unproblematic.


But it's really hard to say what exactly we're seeing there.


What I would recommend doing there


is maybe posting in the Webmaster Help Forum


if you think this is a kind of a general crawling,


indexing type of confusion.


If you think that it's based on your JavaScript setup,


so maybe server-side rendering isn't working well


or we're doing client-side rendering for your pages


for whatever reason, then we have a special JavaScript sites


in Search working group, which is kind of a semi-private forum


that you can just join and you can ask your JavaScript sites


related questions there.


That's something where we tend not to index that content from


that.


I don't think we index it at all from that forum.


So it's a little bit more of a private area


where you can discuss things that


are specific to JavaScript sites just because we want


to make sure that these kind of newer technologies


are indexable reasonably well as well.


Posted in this Friday's central office hours as well,


but maybe we can get to it sooner,


we're facing a strange issue with canonical


selected by Googlebot.


We have two pages that, from our point of view,


have clearly distinct content.


But Googlebot sees one as a canonical for the other.


And then the two URLs, this affects all such similar pages.


So whether for a specific location,


we initially believe that this is because Googlebot was not


reliably fetching the CSS and have since


inlined the critical CSS.


But we're still facing the issue.


Are we doing something clearly wrong that we've missed?


Could this be because of a shared path


in the URL structure?


I don't know.


I need to take a look at the specific setup.


It's kind of hard to say offhand.


In general, I wouldn't assume that there


is a CSS issue that would cause these pages to be


as canonical.


So I'm guessing just from the URLs


alone that the content there is significantly different.


So just purely not having access to the CSS


would not make us think that the content is the same.


So just because of CSS reasons, I


don't think that would be a reason why we would think


that these pages are the same.


What I have seen in some situations, especially


with sites, I'm guessing, kind of like this,


where you have a city name in the URL itself,


is that sometimes we see that you can specify


any name as a city name.


And we'll see essentially the same content.


We won't get a 404 for cities that don't exist, for example.


And in a case like that, we might


think that actually this part of the path or the parameter that


has a city name is irrelevant for this page


because we can enter kind of random words there


and it leads us to the same content.


So that's something that we've sometimes seen there.


MARTIN SCHMIDT-GALANINI Just to add on that, basically, no,


we do 404 cities that are not available.


And what led us to believe that this would be a CSS issue


is when we're doing live testing, in some instances,


we saw a properly rendered page and in some instances,


we didn't.


And when we were checking our logs on server side,


we were actually not seeing any requests for the CSS


from the Googlebot.


And so this is what led us to believe that this might be a CSS


issue.


But anyhow, now the critical CSS is


delivered with the initial HTML, so it should load properly.


And we still can't figure out how to solve this.


The majority of our pages are marked as non-canonical,


and so we've lost basically all of our search traffic because


of that.


JOHN MUELLER OK, and these are all city names?


Is it?


MARTIN SCHMIDT-GALANINI Yeah, basically, the idea of such


a page is that we give you the weather


for the specific location.


So all of those are going to be city names,


and basically, most of them are going


to be in the region of Austria.


What we did find is a very similar challenge


that IKEA posted here, where they are also getting


improperly selected canonicals.


And they seem to have resolved the issue


because you can still find things,


but we've never seen any resolution, anything


in the forums.


Some people have raised similar issues,


but we couldn't figure out anything that solves it.


OK, and you have kind of the country or the language code


in the URL as well, right?


Do you have the same issue with other languages,


or is it just German?


It's actually just for German that we see this issue.


And it's weirdly enough because we


are differentiating between DE, which is our main language,


and DE minus DE for German German.


And this should be proper, actually,


if you read the specifications, but it only


affects the pages with the UL DE so far as we can tell.


But it's a weird issue.


Do you have the same content in other languages then?


Or it's like these are the only kind of pages for Germany


that you have there?


But we've basically prepared ourselves


to offer multi-language, specifically


in the Swiss region, for example.


And that's why we also have this.


But really, we are not utilizing it right now.


So there's no different languages


being offered for these pages.


OK.


I'll check with the team to see what might be happening there.


It might be that our systems are confused with regards


to that URL path and somehow thought that maybe this path is


irrelevant.


But if that's the case, then that's usually something


that we can resolve on our side.


But I'll check.