Because from my point of view, if you're deleting your website


English Google SEO office-hours from August 20, 2021

and you kind of say, well, I think my website is bad


and I want to start a new one, then you


wouldn't be republishing the same content.


Whereas if you did want to kind of keep some of the content


because you think that content is actually good,


then I would just delete the bad content


and then focus on building up a website with the good content


instead and not kind of do this thing where you delete


everything, wait a couple of months,


and then hope that Google doesn't realize that actually


you're just republishing the same thing as before.


So I would kind of shy away from this approach


and try to think of it more as building up something of value


over time.


And if over time you realize that there


are some things in there that weren't that great, then just


kind of improve those things or remove them


and focus on continuously building up


the overall value of the website,


especially compared to kind of this almost


like slash and burn strategies that spammers sometimes


use where they just build up a big website.


And if it doesn't work, they tear it down and start over


completely again because it's just so much work.


And you lose so much energy on kind of like this continuously


building up and taking down thing


rather than just focusing on building something


really strong and really good and improving that over time.


I do a query on Google, and the results are not good.


The first result only has one video and thousands of comments.


Why does that one site rank?


I end up writing quality content on the same topic.


Even not only me, other results that are also great


are still below that result. I look up the website,


and I found there are a lot of authoritative in their niche.


But as much as I know, Google's priority


is on content and relevancy.


And the query was programming.


So I think, first of all, a query like programming


is so ambiguous that there is no absolute right or wrong when it


comes to ranking something there.


So that's something where I would assume


that the results that you see there


are going to be kind of mixed.


And it's going to be hard to just say, well,


I'm going to create a piece of content


on the topic of programming, and Google will rank it number one.


So my recommendation here is, especially


if you're starting out, don't focus


on queries like programming.


Instead, focus on something that is really strong, something


that you can do really well, and something


that doesn't have as much competition


or doesn't have as much other content out there already,


so that you can kind of build up some experience over time


and understand how things work, understand how users actually


react to your content, understand which kind of content


works well for search, which kind of content


works well for users, and over time,


keep building that up.


And that can result, in the end, with you focusing on things


that are more head terms, so things that are shorter queries


that users search for a lot.


But that also gives you a little bit of a foundation


to kind of build on, where you kind of know that, well, I


get a lot of questions on this specific aspect of programming,


and that's why I have a lot of great content on,


and that's where I rank really well.


And then over time, maybe it makes sense


to expand from there into a more broader topic.


Or maybe you find other topic areas


where it's equally the case that there's actually not


a lot of content out there, but there


are enough people that are searching for this information


that it makes it worth your while


to actually create that content and maintain it,


keep it running on your website.


Let's see.


I have a question about branded names and site diversity


update.


In Turkey, we have a case about this one website.


In Turkish, it means directly from the owner,


or I guess it's a Turkish word.


And we can say that it's a generic keyword in our sector.


So our problem when searching for a generic and critical word


like this, the search results of our companies


is operating in the used car listing and real estate sectors


do not appear on Google search results for that page.


On the same page, Google argues that users can only


get the best service when they can access


very different content from a variety of sources.


However, this situation is also in contradiction


with the fact that showing dozens of results belonging


to a single site and queries that are made with this word.


I don't know.


It sounds like something very specific,


where I probably need to see some of the example search


results pages.


But in general, while we do try to show a variety of sources


in the search results, it can definitely happen


that we show a bunch of results from the same website.


So it's not the case that we have a hard limit


on the number of results that we show per website,


but more a case that we kind of try to focus on a broader


variety of sources.


But that still kind of means that it's


possible that we end up showing multiple results


from the same website.


And we wouldn't necessarily see that as a bug.


But it's also really hard for me to judge this,


because on the one hand, I don't know Turkish.


And I don't know what the results could


be for some of those queries.


So it's tricky for me to, I don't know,


figure out what the best approach here would be.


My recommendation here would be, if you can send me


some examples, maybe on Twitter or somewhere else,


on what you're exactly seeing here


and why you think this is bad and ideally why


the top results are bad for these kind of queries,


then I'd love to take that over to the search quality team


and let them take a look to see if there's something that we


can tweak in our algorithms.


But this isn't something where we


would take individual queries and individual websites


and kind of hand-tune them.


That just doesn't work at scale.


But it is something where sometimes this kind of feedback


helps us to improve our algorithms for the long run.


We deleted a disavow file because it


was made in the wrong way.


How long will it take to affect the site results?


There is no fixed time for kind of deleting and reprocessing


disavow files.


In practice, what happens is we pick up the disavow file


immediately.


And over time, as we recrawl the links that used to be in there,


we double check the disavow file.


We recognize, oh, these links are no longer


blocked by disavow.


And then we take those into account.


So my recommendation here would be not to delete the file


and just wait until it's clean and then update a new file,


but rather to just update the disavow file


to what you want to be taken into account for the disavow


and kind of work from there and kind of see it as something


that over time will be taken into account.


Cool.


OK.


We're kind of running into the last 10 minutes.


So maybe I'll switch back to questions from you all.


I also have a bit more time afterwards.


So if we can't get through everything,


we can do more of that later on.


Let's see, Michael, you're up next.


Good morning.


Three questions, one following your disavow one.


If somebody had put a disavow file


and since then they've passed away and stuff like that,


we've taken over the website, but we don't have access


to that shirts console account.


If we upload a blank or a different disavow file,


would this one supersede the previous one?


Yes.


But I mean, then you would have to have access to the Search


Console, the verified site.


Yeah, so we've got access to that.


So we've got access with the new partners looking


after the business, the previous owner, one of the partners


has passed away.


And obviously, all of those access we don't have,


but we've set up new accounts with new access and stuff


like that.


OK.


Well, in that case, you should also


be able to download the disavow file.


And then you can take the old file into account


and update that.


What, from our new creation of the Search Console?


Yeah.


Even with a different account, if it's


for the same verified site, then you


should be able to download the disavow file.


All right, I'll have a look for that then.


The next question is, want core vitals?


We've got static websites done years ago,


and they perform really well.


In the Search Console, we've got good URLs all across the board,


But on the Lighthouse, it comes up as a performance pool.


Now, normally, if somebody comes to this website,


they're browsing generally about seven pages.


They're looking at limousines.


So they're looking at different cars and different events.


And then they convert maybe after going


through a couple of pages.


So it's not the initial page.


But obviously, when they first land,


all the relevant assets, the CSS and any JavaScript and that.


So is it sending the customers, look,


don't worry about the headline figures on Lighthouse


or just look at your Search Console for 152 good URLs?


A bit of both.


So let's see.


Essentially, what you're looking at


is the different kinds of data that we


have around speed, which is the field data, so what users are


actually seeing, which is in Search Console,


and the lab test, which is what Lighthouse would be showing.


And the lab tests try to approximate


what users would see.


And they're very useful for things


like recognizing low-hanging fruit, things


that you can improve on your website.


And they're useful for this iterative debugging


and optimization process.


But lastly, what we take into account


is what users actually see.


And if users are seeing good results and Lighthouse


is saying, oh, you could improve something,


then you could still improve those things.


But users are actually seeing something good.


So essentially, you're already in a good state.


And because Lighthouse tries to approximate what users see,


sometimes that doesn't match what your actual users see.


So in particular, we make certain assumptions


with regards to device connections, the capacity


of the devices that they're using.


And it might be that your users don't match those assumptions.


And because of that, maybe they're


seeing a really good experience versus what our lab tools say,


well, perhaps they might be seeing.


AUDIENCE 2 OK.


OK, and the last question is, we're


reworking a number plate registration site.


So these aren't number plates you make up.


These are pretty old number plates.


So you've got 500 number plates.


So the actual, now, we want people to find those.


The only problem is each number plate's obviously


on their own page and all that kind of stuff.


But the only thing different is the number.


And you can't write 1,500 words about a registration plate.


So what do I do with them?


JOHN MUELLER I think that's tricky, yeah.


I think, on the one hand, it makes it really hard


to make interesting pages.


And on the other hand, I assume lots of other sites


already have similar information.


So we see that, for example, with all of the phone number


sites, where essentially people write a script


and they iterate through all possible numbers,


and they just create pages for those.


And on the one hand, there's a lot of competition


because those numbers are on different places on the web.


On the other hand, there's very little actual content


on the individual pages.


So you have to kind of find a balance


between making something that is actually useful for users


there and making something that has enough value that it


can stand on its own.


So maybe it would make sense to combine some of these


and say, like, I don't know, 500-er number plates


or 600-er number plates, something like that.


But I suspect if you just create individual pages


for every number plate possible, then our systems would kind


of struggle with that and not really know what to show where.


So potentially just a page with 500 numbers on there


and no index the individual pages.


I don't know.


I mean, it depends on what you're trying


to achieve with those numbers.


Yeah, so these are like numbers going back


since day dot in the UK.


So these are previously owned.


So for example, A1 on a number plate.


They can't be purchased anywhere else.


So if you wanted A1 on your car, the only place to get it from


is this particular website.


And that's it.


So if somebody is going to search for it,


that page, that number needs to come up somewhere.


Oh, OK.


So these are essentially products that you're offering.


And the name of the product is essentially just


that number plate thing.


Now, I don't know.


I think that'll be tricky.


But it's something where my guess is the website overall


will just need to be seen as kind of an authoritative place


for these kind of number plates.


And my assumption is you'll have a hard time ranking


for most of these number plate queries directly.


And you would end up ranking more


for things like unique number plates or something like that.


OK, Doki.


All right, thanks for your time.


Sure.


All right, apps, I think.


Hi, John.


Hi.


We, it's quite a specific question.


We run a e-commerce platform where anybody can come along


and create products, stores, have a small blog area,


and so on.


And all of this is under our domain.


So think Shopify, but all their stores under one domain.


And we've got about 100k pages indexed.


And we think these pages are best described


as user-generated content, or UGC.


And we've watched previous Office Hours of yours.


And we learned that UGC content and our content


is treated the same, and that we should filter out


low-quality content.


And we're doing our best to do that.


My question is, and hopefully it's not a silly one,


we'd like to rank higher for our own target keywords.


But we're worried that all this varied content and all


this noise is holding us back.


Is there something else we should be doing to kind


of differentiate our content?


We have our own blog and our own marketing pages.


But then there's a ton more e-commerce pages.


Yeah, I think the cases where I've seen something


like this in the past, the best approach


tends to be to separate things out.


So essentially, if you're taking your own content


and it's ranking amongst a lot of user-generated content


or content generated by other people that is very similar,


then we would see that as one website.


And then we would struggle to figure out


like which part of this website is relevant for which queries.


So the ideal situation there, at least in cases


that I've looked at in the past, where you're offering blogs


to other people, for example.


I don't know if that kind of matches the model that you


have for your e-commerce site.


But when you essentially offer individual websites


that people can create and host on your platform,


then it makes more sense to put that


into something like a subdomain so that each user has


their own subdomain, or maybe even make it


so that it's easy for them to put it on their own domain,


and for you to kind of take your kind of infrastructure


information and also put that on a separate site.


So essentially something like, I don't know,


your provider as a separate website


from all of the hosted content that you have.


And using subdomains is sometimes


an easy way to do that.


Sometimes it makes more sense to separate that out even more.


Kind of similar, I guess, to how a blogger does it,


in that you go to blogger.com for the blogger information,


and the subdomains on blogspot.com


are the individual user sites.


I think that kind of model makes it a lot easier.


That also helps to isolate things a little bit when


individual providers post a lot of low-quality content,


because then we can recognize, actually,


this subdomain is low-quality, not the website overall.


Or sometimes we run into situations


where someone includes malware or spam or something


on their site.


And if we see that all of the part of your whole site


itself, then we'd say, well, actually, this whole site


has a lot of spam or malware elements attached to it.


We need to take action on that.


And that would include your kind of the infrastructure content


as well, or all of the other providers


that you're hosting there, which don't have anything


to do with that spamming content.


So by isolating things a little bit,


usually it makes it easier for us to understand


these are individual elements.


And we need to take each of these into account


individually, rather than as one combined element.


AUDIENCE MEMBER 1 Got it.


That makes sense.


It's interesting you mentioned the malware,


because it was actually a second part to my question, where


we, in Search Console, we've been getting uncommon downloads


as a warning.


And what we've done with that is we've scanned our website.


It's easy for us to destroy infrastructure and recreate it.


And then we submit a review.


But it's been coming back once a month.


We get a success message, but it comes back again.


I guess it's really hard also to ask for a specific URL that's


causing that issue.


I don't know what we could do with that.


Yeah.


I mean, usually the uncommon downloads


comes from something where you're hosting something that


is almost unique per user.


So that's kind of a general situation where, for example,


you can sign up and you can download some software tool,


and it generates a zip file or an executable specifically


for that user.


Then because those files are unique to each user,


we can't scan those files.


And that's why we flag them as uncommon downloads.


And if you do the review request,


then we do go off and scan those files that we found


and double check that they're not actually malware.


But that's kind of why we don't have individual URLs that we


can show you there.


But if you do the review, we will double check,


and we will let you know if there's anything


that was actually problematic.


And if these are files that individual users are uploading,


then it's kind of tricky.


And that's also one of those cases


where isolating the individual users from each other,


I think, makes a lot of sense.


Because then it's just one subdomain or one domain


that is getting this warning, and they have to figure it out.


And it's not something that you have


to figure out for everyone.


SPEAKER 1 Got it.


Just to clarify the subdomain, do you


mean the subdomain on our domain?


Because wouldn't that still fall under our property?


Yeah, I think a subdomain on your domain


is kind of the minimal step I would take.


I would even kind of move your primary content


to a different domain.


Just so that it's like the infrastructure domain


and the different hosted websites are separate.


Perfect.


Thank you, John.


Super helpful.


Sure.


OK.


Time-wise, I think we're at time.


But like I mentioned, I still have more time


for the people that still have raised hands.


Thank you all for all of your questions so far.


It's been really interesting.


Lots of good questions.


And I hope you found this useful.


If you're watching this on YouTube,


feel free to join in on one of the future office hours.


I do these English ones about once a week,


alternating between early and late times


to try to get more time zones in.


But you can also drop your questions in the YouTube


community section as well.


All right.


And with that, I will pause the recording.


English Google Webmaster Central office-hours from October 18, 2019

All right.


Welcome, everyone, to today's Webmaster Central Office Hours


Hangouts.


My name is John Mueller.


I am a Webmaster Trends Analyst here at Google in Switzerland.


And part of what we do are these Office Hour Hangouts,


where anyone can join in and we can talk about their website


and search.


As always, it looks like a ton of questions were submitted.


But if any of you want to get started with the question first,


you're welcome to jump in now.


Did the messages get approved?


Our messages seem to be vanishing from the list.


Is that normal?


Vanishing from the list.


So how do you mean messages in Search Console?


It was just a question on the YouTube community comments


area.


Oh.


I don't know.


I guess it was upvoted quite a bit.


So oh well.


I mean, I think there is some kind of spam filtering that


happens.


And partially, I get some of those listed


that I have to manually approve.


But I thought they had them all approved.


That's probably it.


But maybe too many people like this question.


Therefore, it must be something wrong.


Feel free to ask it if it's something that you submitted


that's missing now.


OK.


I've been at this for 15 years full time.


And I've seen it all.


But I think I have a pretty good case for a legacy


penalty on a domain registered and promoted heavily


by a client in the millions of dollars of branding effort.


And the issue that I have is that we can't lift above


position 23 for a very unique branded search.


There is no competition on Google for the brand.


Bing took it to number one in two days.


And we have a completely scrubbed link profile,


great content, good inbound links.


This was in May that the site was launched.


And the link profile scrubbed.


Wayback Machine shows the had a porn history in 2014.


The main was expired and then re-registered.


They didn't check its history and then called me later.


So what we are desperate to find out


is other than doing the normal great content, good link


profile, this kind of thing, I've never seen it before.


Yeah.


Wow, there's a lot of noise.


Let me mute some of you.


Feel free to unmute if you have some questions.


Now, it's hard to say where to start.


It sounds like you have pretty much the basics of that


covered.


So things like making sure that the content is good,


cleaning up old issues with regards to links,


those are generally the bigger things.


Sometimes there's also a matter of geotargeting,


which can play a role.


So if something, for example, was highly


targeted to one country, and then the next owner essentially


takes it and wants to target a different country,


sometimes that takes a while to settle down again.


If it's a global website, that's probably less of an issue.


But sometimes it can be a bit tricky.


If you want, you can send me the URL.


Maybe drop it in the chat or send it somewhere else.


And then I can take a look at that.


But especially if there's, I guess,


a long history behind a domain, then


it can take a while for things to settle down.


And it sounds like you've had this since May


or have it set up since May, and that


seems like it should be enough time.


But sometimes things take a little bit longer,


and it's good to double check.


I put it in the chat for you.


All right, great.


Thanks.


Hey, John.


It's Amos here.


Hi.


I joined the Hangout, I think, about a month ago,


and I was asking about recent migration we've done,


which involved Google for Jobs.


And we lost a whole bunch of our visibility.


We're still nowhere.


We're not recovering, essentially, from this.


And I shared.


I emailed you about it, too.


I was wondering if you had any updates on that.


I did hear something back from the team,


but I'd have to look it up.


Let me see if I can find your email afterwards,


and then I'll ping you there.


Great, thank you.


Hey, John.


Oh, go ahead.


Oh, thank you.


Hey, John, how are you going?


Pretty good.


How are you?


I'm very well, thank you.


It's 6 PM in Australia here, so I'm


staying up on a Friday just because I'm


so keen to chat to you.


Oh, man.


So I just have a question about featured snippets


for a particular client that I'm working with at the moment.


So they're a very, very large site,


one of the largest sites on the web.


So they rank for a lot of different queries.


But they don't rank for any featured snippets


at all for the site.


But there's no technical issues on the site,


or they're not using that no snippet script.


And I have a good feeling that it is an algorithmic penalty


that's been put in place for this domain.


It does fit the criteria for one of the categories


in the Google Support documentation


for why featured snippets might not show for a site.


So I have a good feeling that that is why.


Not sure if that's algorithmic or a manual thing put in place


by Google there.


So basically what I wanted to ask you,


is there any chance that I could send you this site privately


and get you to have a look if there are any sort of lists


at all?


Yeah, sure.


I can take a look.


But especially if you're saying that it's


one of the kind of sites or the kind of content


that we might not show as a featured snippet,


then I don't know how much we can do there.


Because a lot of that is, like you mentioned, algorithmic


in that we try to figure out where our site fits in


and where it makes sense to show these kind of snippets.


So that's a bit tricky.


But you're welcome to send it to me directly if you want.


Let me just drop my email address here.


Yeah, I'm not getting in contact to ask you,


like, why is this happening?


I want this site to rank for featured snippets.


It's more so just wanting to know, is it on a list?


And is that the reason why they're not ranking for any?


I looked at quite a few other sites


that are competitors to this particular website.


And probably 80% of them don't have featured snippets showing


for their websites.


But there is a couple that do have featured snippets showing.


And they seem to rank for similar keywords.


So there's a bit of ambiguity there.


Yeah.


I mean, with a lot of these algorithmic factors,


it's always the case that we can't catch everything


or catch it perfectly.


So that might be something where you might see, well,


in this general niche or this general type of content,


Google says they don't want to do it.


But they still show this one particular one.


And then that's usually not because we're


trying to exclude them from that group,


but rather because our algorithms just


aren't picking that up.


So I don't know what kind of site it is.


So it's really hard to say.


But I'm happy to take a look.


Usually, if there is something manual that's


blocking something specific, then that


would be shown as a manual action in Search Console.


So you'd be able to do reconsideration requests there


to have someone review it.


But there are lots of features in Search


that have different kind of triggering algorithms


to figure out when we should be showing something


or when we shouldn't be showing something.


So that's fine.


I'm happy to take a look.


Yeah, one thing I recall seeing on Twitter a little while ago


that Danny Sullivan said that there isn't any notification


that's sent through Search Console at the moment


for a featured snippet penalty.


And that's something that Google is looking into at the moment.


But if you know something that I don't know,


that would be awesome.


I'll send you an email anyway.


Sure, OK.


Great.


Thanks, John.


Hey, John.


Sure.


Hi.


So my question is mostly, so we have


a website which has an m version and a desktop version


separately on the web.


So right now, because of mobile-first indexing,


should we change the canonical and the rel equal


to alternative form?


Because before it was like in desktop,


we put alternative to mobile and mobile canonical


to the desktop.


Should we keep same way, or should we reverse it now?


It should stay the same way.


So you should still point the canonical


to the desktop version and the alternative


to the mobile version.


What will happen internally is we


will use that to understand which of these pages


belong together, not necessarily that you're saying,


like, this is the canonical.


Because internally, we'll pick the mobile version


as canonical.


But we need to understand that they belong together.


And we understand that through this set of links.


So another question is, we have this indexing API


for indexing a job base or maybe a live streaming pages.


So in that also, we have to keep desktop version of the page


or the mobile version.


Good question.


I don't know.


I would assume that the mobile version would


be the right one to show there.


Because that's what we would index.


OK.


So another question.


So right now, we have lots of issues going on


in the desktop as well as mobile version.


Because it's a completely kind of new website.


So my question is, how to tackle it?


We should go ahead and clear the issues with the mobile first


and then desktop, or vice versa?


Go ahead and just do the changes in the desktop


first and then mobile.


So they should be equivalent.


So the mobile version should be the same as a desktop version.


So it's something where, if you ask me which one to do first,


it's basically, you should do both first.


But I think one of the things that we've


been seeing quite a bit, especially


with mobile first indexing, is that when


you have a separate mobile URL for the setup with m-dot version


and desktop version, then that just


makes everything a lot harder and a lot more confusing.


So we've really started recommending


not using separate mobile URLs and instead using


one single URL for both desktop and mobile version.


Because then you don't have all of these problems.


You don't have to worry about which URL to set up,


where to have the content.


It's like all one URL.


Well, we have asked our product team to do that.


But they are kind of setting to do this.


So yeah, we are focused on doing that.


I know.


It's a lot of work to redesign a site like that.


So it's not that we're saying you shouldn't do it like this.


But we really recommend, especially


if you're doing a bigger redesign at some point,


then try to move from separate mobile URLs to just one URL.


So is dynamic serving will help in that case?


Like, let's suppose if the URL will change the URL format.


So it will be one URL, but in back end, it will be different.


So will it be different?


Dynamic serving is perfectly fine.


Using responsive web design is fine.


We usually recommend responsive web design


because then you don't have to worry about which version you


show.


All of the content is there.


But if you want to go with dynamic serving,


that works well.


Because product team is kind of not ready to move


into the responsive site because of some other reasons.


Yeah.


I know.


It's always a lot of work to make these kind of changes.


Yeah, that's all I have for now.


Thanks.


Cool.


OK.


Let me run through some of the submitted questions.


And if you all have any questions or comments


along the way, feel free to kind of speak up.


And I try to make sure that we have a bit of time


towards the end as well for anything new that comes in.


I have a client who has quite a few pages, no index.


However, Google is grabbing these no index pages


and assigning them as canonicals.


It's a real estate site.


And they're ascribing zip code searches


to static neighborhood URLs, which is, in fact, resulting


in duplicate content.


I suggested they stop and engineer a better solution.


But kind of like, what could be happening here?


It's really hard to say what exactly is happening there.


In general, when we see a URL with a no index,


we try to avoid making that a canonical URL.


But what can happen is that our systems


try to understand broader patterns on a website


and accidentally pick up things like that


with regards to duplicate content and canonicalization.


So a really common setup is, for example, you


have www and non-www.


And we understand that these versions are essentially


the same.


So if we find one that is not the canonical version,


we'll kind of automatically shift that over


to the other version.


Other examples that I've seen are, for example,


if you have a bunch of websites that essentially show


the same content.


Maybe you have an e-commerce site,


and it's all the same products on two or three


different domains.


We might recognize that these URLs are all the same.


And we'll say, at that point, our systems might say,


well, any URL on this domain is the same as a URL


on that domain.


And it can also kind of play internally within a website


in that we might find this URL pattern within the website


almost always leads to exactly the same content


with a different URL pattern.


And we might pick one of those patterns kind of as a canonical.


And then it can happen that we say,


well, this URL, it fits the pattern that we know.


We haven't had a chance to double-check what the content


actually is on this URL.


But almost all of the URLs with this pattern


map to this pattern.


Therefore, we'll map that as a canonical.


And then it could happen that you


have some pages that lead to a canonical that are no index.


Because we think, well, almost all of these URLs


kind of canonicalize like this.


So we'll do that here in this case as well.


The real way to kind of clean this up


is to convince our systems that you


don't have this kind of systemic duplicate content


or kind of canonicalization issue on your website.


And that's something that our systems do kind of relearn


over time.


So if, for example, you have a bunch of search results pages


and they're all canonical to one version,


or they all show the same content as a different version,


then make sure that you don't have


all of these duplicate URLs that lead to the same content.


So that's kind of the general approach


that I would take there.


If that's something that you've already cleaned up,


and it sounds from the question like you're still discussing


this kind of a cleanup, then that


would also be something that we'd be able to double-check


manually on our side.


You could either post in the Webmaster Help forum


or ping us on Twitter or something


so that we can double-check to see what is happening here


if we have to do something manual


or if this is just a matter of our systems kind


of relearning your site again.


SPEAKER 2 I have an issue that's somewhat similar.


OK.


You want to jump in?


Sure.


So we see the SEO of an e-comm site.


And we have four properties, Canada, US, Australia,


and New Zealand.


They all share the same technical infrastructure.


What I'm seeing is that in New Zealand,


our website gets canonicalized to the AU site every two weeks.


So the NZ site just disappears from Google Search


every two weeks for around two weeks and then reappears.


And when I look at the URLs that disappear,


I see that the user canonical is the NZ canonical.


And then the canonical picked by Google is the AU one.


And I'm trying to understand why the hreflang is set up


properly, not having any restrictions on the server


side.


So I was thinking that it might be a crawl issue.


It's not.


So I'm a bit clueless as to what it should be.


Yeah, I think it leads into a similar situation.


And it's something that is fairly common.


But it's fairly complicated.


So let's see.


So I think primarily what happens


is our systems look at these pages


and say these are the same.


So probably, I mean, I don't know your site.


But in a lot of cases, you'll have exactly the same product


on the UK version as well as on the Australian version.


And the difference might be just the price or something


like that if you have different currencies.


And our systems would look at that and say,


well, these are essentially the same page.


We can save the webmaster some trouble


by just picking one of these pages and indexing that one.


And then through the hreflang links,


we still understand that there are multiple URLs


that lead to this page.


So in the search results with hreflang,


we can still swap out the URLs to show the right one.


But indexed is just one of these versions.


And this is further complicated by Search Console, which


sometimes makes your life a little bit harder


than it needs to.


In Search Console, we try to report on the canonical URL.


So in the search results, we might show the Australian one


because we are able to swap that out.


But if we have the New Zealand one as canonical,


we'll show that one in the performance report.


So if you look at the indexing report and the performance


report, it'll look like, well, New Zealand


is the only one that's getting traffic


and the only one that's indexed.


But actually, in the search results,


we would still be showing the Australian URL.


Usually, you can try that out by just searching


for a product name and then changing the URL parameter


in the search query with, I think,


GL for the geographic location, I guess,


and then equals the country code.


So that's usually how I double check that.


And you should be able to see that while we have the New


Zealand version as canonical, we do show the Australian URL as


well.


So that's only true in some instances.


NZ, in this case, disappears, right?


But I don't see AU showing up instead.


So our site, our domain just completely disappears.


And on branded queries, we see AU, but only on branded queries.


So we do that bit of detail together for AU and NZ.


OK, that shouldn't be happening.


That sounds like something that maybe I should take a look at.


If you want, maybe drop some of the URLs in the chat


and I can double check that afterwards.


But especially if the content is exactly the same,


then all of these things kind of play a role.


And it makes it really hard to figure out


what is actually happening.


The real way from talking with the indexing teams


to kind of fix this is to make sure


that the content is different, which for an e-commerce site,


when it's specific to the product,


it's like that's really hard unless you really


spend a lot of time to make sure that the template


of these pages is significantly different across


the different country versions.


The other thing from a practical point of view


that I usually recommend is to use some kind of a banner


on your pages when you recognize that the user is probably


from the wrong version.


So if you can recognize that a user is in New Zealand


and they go to the Australian version,


then use JavaScript and show kind of like a banner on top,


saying, hey, we have a local version.


Yeah, so what we did, we did something similar,


which is a geo redirect, which you probably won't like.


But for any NZ customers landing on AU,


we're redirecting them to NZ.


Yeah, I think if you're doing that across Australia


and New Zealand, we probably wouldn't see that.


But if you're doing that for users in the US as well,


then you might also get a redirect.


And then we would just follow that redirect.


That's why we're only doing it there, because I thought


that you might only be crawling from the US.


Yeah, I would double check where we crawl from.


But I think even not fairly, we crawl from the US.


Yeah, I'll send you the details by email, if that's OK.


OK, sure.


Cool, thank you.


Yeah, these international sites can be quite tricky.


We have the same problem in Germany, Austria,


and Switzerland, where everything also is in German.


And usually in the German hangouts,


it's like one of those topics that comes up every time.


Hey, John.


Hi.


Can I get a question in here?


Sure.


So we have a lot of user-generated content.


We have a listing site where communities list.


And is spelling errors and stuff like that


going to hurt us and content and stuff like that


that the users generate?


Not necessarily.


But if you provide this content in a way that we can index it,


we will assume that it's content that you


want to have published.


So it's not that our systems will look at your site


and say, oh, this was submitted by a user.


Therefore, the site owner has no control


over what's happening here.


But rather, we look at it and say,


well, this is your website.


This is what you want to have indexed.


You kind of stand for the content


that you're providing there.


So if you're providing low-quality user-generated


content for indexing, then we'll think, well,


this website is about low-quality content.


And spelling errors don't necessarily


mean that it's low-quality.


But obviously, it can go in all kinds of weird directions


with user-generated content.


SPEAKER 2


All right.


That's good.


What about not safe work?


So some of these communities are not safe work.


Is there anything that Google does in that kind


of situation?


So if you have a site or which sites


need to be filtered by Safe Search,


and depending on how you have that set up


with the communities, if that's subdomains or subdirectories


or just different domains completely,


then that makes it a little bit easier.


If this kind of adult content is mixed


in with the rest of the website and that from a URL pattern


level, it's really hard to tell which parts are adult content


or essentially just general content,


then that would make it a little bit trickier for us


to figure out how to deal with Safe Search there.


So the clearer you can separate it,


the easier it will be for us to pick that up and just say,


well, this is the part that needs


to be looked at from a Safe Search point of view.


SPEAKER 3


Would a not safe work tag on the page, just a simple tag,


would they pick that up NSFW or something?


Probably not.


So there is an adult content meta tag


that you can add, which I think we specifically


use for image content.


But I don't know how easy that would be to implement.


OK.


Thank you.


Sure.


All right, we're going to move to the submitted questions.


And we'll go to that.


I'm a little noise here.


Sorry.


And we'll try to make some room for more questions from you


all as well.


Wondering if Google checks status codes


before anything else, like for rendering content.


Yes, we do.


We do.


And it goes, are you going to index the content


or render the content?


In particular, if it's a status code 200,


then that's like a sign that there is something here


that we might be able to index.


And if it's a 400 or 500 error or redirect,


then obviously those are things we wouldn't render.


So if you have a really nice 404 page,


then that's not something that we would see for indexing.


Similarly, if your page returns 404 by default,


and actually you can't change the status code.


Well, if it returns 404, then we just


won't render anything there anyway.


So it should return 200.


I have an issue with Google cache and dynamic rendering


of a page with JavaScript.


I always try to see to index the totality of web pages


so that the cache can see what my users see.


If the two are not aligned, I'm always a bit worried.


So one thing about the cache page


is it's not exactly tied to indexing.


So just because you see something in a cache page


doesn't mean that's exactly what we use for indexing.


In particular, when there's JavaScript on a page,


then it can be a bit hit and miss


if that JavaScript is executed when you look at the cache


page, because JavaScript has certain requirements


with regards to when it's allowed to run.


And if you're pulling in further resources,


if that's happening across domain,


then JavaScript might not be able to do that.


So the cache page really just looks at the static HTML


version that we see when we try to crawl that page.


If you have a website that uses JavaScript,


then the cache page won't necessarily


reflect what we use for indexing, because for indexing,


we would try to render that page.


We want to implement a dark mode for our website.


Do we have to consider anything in relation to SEO?


I think this is totally cool.


I really like the trend to dark mode websites or dark mode apps


in general.


It's something, I don't know, that I wouldn't have expected


like a few years back that people


would want to have kind of this light mode, dark mode setup.


With regards to SEO, that's absolutely not a problem.


Usually, people implement this using CSS,


and the way that you implement things in CSS


doesn't affect how we would pick things up for indexing.


So go for it.


I think that's really awesome.


I don't think dark mode would be a ranking factor.


So maybe at some point in the future,


if dark mode is really, really popular,


then maybe we would need to highlight dark mode sites


in search when people have their phone set up to dark mode.


But I don't know if that would actually happen


or if that will really go that far.


Can I publish multi-language versions of the main content


without subdomain or subdirectory?


Yes, you can.


So different language content should be on different URLs,


but you don't need to put it in different subdirectories


or subdomains.


It can be with URL parameters even.


Earlier this year, Google announced all new domains


would be on mobile-first indexing.


We're not really seeing that.


Sometimes it says mobile.


Sometimes it says desktop.


Did this actually happen?


Yes, it did happen.


But kind of like the name mentions, it's for new domains.


So if this is a domain that we've seen before in the past,


then we'll do the usual process of trying


to figure out when the domain is ready for mobile-first


indexing.


If it's something that we haven't seen before,


then we will use that for mobile-first indexing.


At some point, the next step will


be that we move everything over to mobile-first indexing


just to kind of keep things simpler.


But I don't know when that would happen.


We'd definitely announce that ahead of time.


If I build 200 backlinks in two days


and didn't perform any link building for years,


will Google still see this as black hat and penalize me?


So what about link velocity?


So from my point of view, if you're


jumping in with a question like this


and you're saying, I am going to get 200 backlinks in two days,


then that sounds a lot like you're not


getting natural backlinks.


That sounds a lot like you're going off and just buying them


or having someone buy them for you.


And that itself would be kind of the thing


that we would not be happy with.


So it's not so much a matter of how many links


you get in which time period.


It's really just, well, if these are links that are unnatural


or, from our point of view, problematic,


then they would be problematic.


It doesn't really matter how many or in which time.


Is there a way to show the last updated date instead


of the published date in the search results?


No.


Or rather, it's not that we show the published date in the search


results.


We try to figure out when a date is appropriate to show


in the search results and which date would


be appropriate to show there.


And sometimes that can be a date on the page.


Sometimes that can be a date from when we originally


saw the page or when we saw that the page was


last updated.


So the dates that we show can be kind of different.


And in a lot of cases, we don't show dates at all


because we don't think that the date is an important part


there.


But you can't toggle between which date should Google


show in the search results.


We have a website with separate desktop and m-dot versions.


So we use a canonical.


Since mobile-first indexing, should we reverse that?


I think we touched on this before, and the answer is no.


Kind of leave those links the way that they are.


We looked into this quite a bit with the team involved.


And our recommendation was really


people shouldn't have to worry too much about the mobile-first


indexing change.


In particular, they shouldn't need


to watch out for when this change happens


and then change everything on their website.


So we use this connection to understand


that the pages belong together, even


if we pick the mobile version as canonical in the end.


So John, sorry for dispelling.


So for the sitemap also, we'll do the same thing, right?


It will be as it is.


Sure, yes.


Perfect.


Thank you.


Does a no-index follow header directive


on an index sitemap containing links to other sitemaps


block sitemap retrieval on Search Console?


So first of all, normal meta tags,


you can't put them into sitemap files


because sitemap file is an XML file and not a web page.


You could use HTTP headers to give us


this information on no index.


And that would only apply if that URL


were shown in normal web search.


So it doesn't affect the processing of the sitemap file


at all.


You can put a no index xRobots tag HTTP header


on an XML sitemap.


You can put it on a robots.txt file.


You can put it on your CSS files if you want to.


All that does is prevent that URL from showing up


in normal web search.


In almost all cases, we have enough good content


to show from a website.


So we would not show a robots.txt file or a sitemap


file in the search results.


It can happen that for whatever reason,


we weren't able to understand the normal web pages properly.


And then suddenly, the sitemap file shows up.


The more common scenario is that people explicitly


search for a sitemap file using a site query or in URL query.


And then in that case, we don't really


have an understanding of the relevance.


And then we might show that in search.


But usually, that's not something that you need to suppress.


So unless you really see your sitemap file showing up


for normal search queries, then I


would not bother with putting a no index xRobots tag HTTP


header on that.


And if you do see your sitemap file showing up


for normal web search queries, then I


would strongly recommend figuring out


why we think your sitemap file is more relevant


than your normal web pages.


Because that seems to be a sign of a more general problem


on your website, that we don't understand your website well


enough to understand that actually your content pages are


much better for users than the sitemap file.


I have a valid FAQ schema.


But I was wondering about the guidelines around not


for advertising clause.


Could being overpromotional hurt my rankings?


This would not affect your rankings.


So that's kind of the first thing.


What can happen is that our algorithms look at this


and say, well, we wouldn't show the FAQ search results


type for this page.


It definitely wouldn't affect the rankings.


So on the one hand, I'd love if everyone just


sticks to the guidelines and kind of does it the way


that we have it recommended.


On the other hand, if you're not sure


if it's recommended, on the other hand,


if you want to play around a little bit with that


to see how far it makes sense to go


and maybe how far is too much or looks kind of weird


in the search results, then you're welcome to try that out.


In the worst case, what would happen


is that we would not show the FAQ schema for those pages.


I guess if you really, really overdo it,


then what could happen is that someone from the Web Spam team


looks at that and says, well, all of the structured data


on this website is really spammy or really problematic,


and we would just not show structured data,


rich results from this website.


But again, that would not change your rankings.


It's really just the presentation in search.


What's the best way to embed relevant Google reviews?


Is the API the same as simply copying and pasting text?


So if these are reviews for your company


that you want to host on your company website,


we would not show them in search.


So you can embed them however you want to do that.


If you're using an API that uses JavaScript,


keep in mind that we need to be able to render this output


if you want to have that indexed.


That might make it a little bit trickier.


I don't know how that API in particular works with regards


to can Googlebot crawl the JavaScript files and the server


responses to get those pages.


You can check that with the Inspect URL tool


in Search Console.


Copy and pasting text is something you can always do.


If it's in static HTML, we'll be able to index that.


I don't think this would change anything for ranking,


and we do try to prevent these kind of reviews


for your business when they're hosted on your business


from showing up in search.


But if you want to put them on your pages


and you want to have them indexed,


users to see them, that's totally up to you.


Is there a benefit for a site to have reviews for product


on completely separate URLs other than the product?


I don't really think so.


Sometimes I see sites do that with reviews


because they have a lot of reviews


that they put the kind of like paginated versions


on separate URLs.


But in general, it helps us to understand


that this is the product, and these are the reviews that


are available for that.


So ideally, have that together.


What are the main fundamental SEO activities


to grow website traffic?


I noticed that SEO has changed quite a bit,


and consumers change.


They search on Quora, and they go to Amazon directly to buy,


and they go to video sites for entertainment.


In general, I don't know.


The web keeps changing, and I don't


think there's one magic trick that I can offer you that will


make sure that your website stays


relevant in the ever-changing world of the web.


So that's something where you'll kind of


have to monitor that on your side and work out


what makes sense for your site, for your users,


for your business.


Hi, John.


Let me run through a few more questions, and then.


John, can you hear me?


Yes, but let me run through a few more questions,


and I'll get back to you.


I have one query.


I'll get back to you, just a second.


OK, we are ready.


All right, are snippets in more languages coming any time soon?


I don't think so.


So we try to understand what language a page is in,


and then we show the snippets or the kind of text preview


in that particular language.


So it wouldn't be the case that we'd


recognize the pages in multiple languages


and try to show multiple snippets.


So in particular, if you have multilingual content,


make sure it's on separate URLs so that we


can show it appropriately.


Search Console says I have a content injection problem.


After multiple analysis with multiple browsers,


I still can't find it.


All the content is legitimate.


I requested a new review, but I'm still


getting the same error.


What can I do?


So in general, the content injection message


is something from the, I believe, from the security


side in Search Console, which is basically


that our algorithms are picking something up that


looks a lot like hacked content that perhaps someone


has placed on your page.


And one of the things or one of the techniques


that website hackers like to use is they cloak this content


to Googlebot in particular or to search engines


so that when you, as a site owner, look at it,


it's like, it looks fine.


But when search engines look at it,


they see all of these links or all of these kind


of promotional material leading somewhere else.


And that's something that would be problematic


and might be tricky to find.


So I'd recommend double checking using tools like Inspect URL


in Search Console and maybe getting some help


from other people who've run into similar problems like this


so that you can be fairly certain that you're checking


all of the right versions.


And sometimes we do get this wrong.


So using the review feature is a great way


to make sure that someone manually looks at your pages


and make sure that this is really something that


is problematic and is probably from a hacker


rather than something that is really on your web page.


For example, I've sometimes seen cases


where you have a medical website that talks


about certain pharmaceuticals.


And we might look at that website


and say, well, this is talking about the same pharmaceuticals


as spammers talk about all the time.


So maybe this is a hacked site as well.


And that's something where when we look at that manually,


the Web Spam team will be able to clean that up.


We're seeing highly unusual activity in our Search Console


data.


Impressions drop suddenly and rise up again.


We believe there's some bug in Search Console here.


Can you let us know what's going on?


I took a quick look at this thread.


And it looks like someone else has replied already.


So maybe that's kind of OK.


It sounds like what you're seeing there


is something similar to the first question


in that you have a country version


and you have different country versions.


And if the same content is available on multiple country


versions, then we might pick one of these as canonical.


And that can shift over time.


The other thing that I noticed when looking at these pages


is you're automatically doing a geo redirect


for individual countries and regions.


So if Googlebot sees that redirect as well,


then that could also result in this kind of shift.


So I'd really recommend trying to stay away


from geo redirects like that and instead doing a banner type


thing, like I mentioned in the beginning, or perhaps also,


or and, I guess, also trying to make sure


that the content itself is actually


different across the different country versions.


We have a large price comparison site.


And we have structured this in a way


that whenever there is zero offers per merchant,


the page gets noindex automatically.


Is it possible that this indexing, noindexing


might affect my rankings?


It shouldn't affect your rankings


in that it's completely normal for sites


to have some content that's indexable.


And then maybe later on it's noindexable.


And maybe then it's indexable again.


So that wouldn't be affecting your site's overall ranking.


However, when we see pages that are persistently noindex,


then we might see them as kind of soft 404 pages.


And what would happen then is we would just


recrawl those pages a little bit less frequently.


So with a normal page, maybe we would recrawl it


every couple of days.


If we see that it's always been a noindex for a longer time


now, then we might say, well, maybe we'll


look at this page every couple of weeks


instead of every day.


So that's something that you might be seeing there.


If you're changing things back and forth fairly quickly,


then sometimes it can happen that we


don't recognize that the change has happened that quickly.


You can let us know about this on a per page basis


using the inspect URL and submit your indexing tool.


You can also let us know about this using the sitemap file


with the last modification date.


All right.


We're into the last 10 minutes.


So maybe I'll just shift over to more questions from you all.


Yeah, hi, John.


Hi.


Hi, this is Sanjay.


I am from India today.


We have a URL called indiatoday.in.


We are facing some issue from last three days


where our site is not being crawled.


So there is a zero indexing of any page on Google Search.


So can you just find out?


OK.


Is that something that you are seeing across Google News


and Google Search?


I think you also tweeted about this, right?


Yes, correct.


So it's indiatoday.in, which is Google Search.


I'm talking purely Google Search.


News still we are still there.


Some of our articles are there.


One of the things I noticed with your site


yesterday while looking at it is we're


having a lot of trouble crawling things from your website.


So that might be something to kind of look into.


In particular, we were seeing a lot of errors


trying to access your website, which makes it really hard


for us to crawl.


I think because of that, we've significantly scaled back


the crawling from your website.


So that's something where I would double check


from a technical point of view that you're not blocking us


in any particular way.


Maybe check with the log files if you have access to that


to make sure that there's no special response


codes or no kind of geoblocking or anything like that


is happening.


It looks like we're still able to crawl a little bit.


We're just not able to crawl nearly as much


as we used to be able to.


Yeah, yesterday there was some crawling happened.


But since last night, again, everything has gone out.


So whatever was crawled already.


So what you're saying is right.


Either we have blocked something,


we have checked using third-party tools,


or also maybe through the web console, Webmaster console,


we have tried to individually, URL,


we tried to crawl it manually.


It's showing no error.


So then it becomes very difficult to identify also.


OK.


I'll double check with the crawling team on that.


Yeah, please.


How can we reach out to you?


How can we talk on this particular issue?


I think Twitter is probably best.


OK, maybe I'll ping you on Twitter.


Maybe I'll DM you.


OK, cool.


Sure.


We are a news company.


Just to inform you also, we are a news company.


We publish roughly around 250 to 300 articles


per day on this particular website.


None of them are reflecting over there.


Yeah, yeah.


I saw that in your tweet.


What you can also do is, as a Google News publisher,


there is a link to, I believe Danny posted this


to the contact form specifically for Google News publishers.


So if there's something specific to Google News,


then they would be able to help with that.


If there's something just general


that we can't crawl as much from your website,


maybe they wouldn't be able to help.


So we have submitted that also, but we did not


get any feedback from that.


So we have submitted the ticket also.


So we hope that we will get to hear from them.


Yeah, we also had some indexing issues, I believe,


yesterday or the day before.


But that was fairly short.


So if you're still seeing issues with regards to indexing,


then that wouldn't be related to those indexing issues.


John, we already got response from a news support team,


news support team from Google.


They added and they sent the mail.


We did not find any technical issue in your website.


After that, I tweeted to Danny and you.


I already filled that form provided by Danny.


I didn't get any response, any mail from them after that.


I don't know particularly how they're organized


on the Google News site, but I'm happy to take a look.


And like I mentioned from what I looked at yesterday,


it did look like we were just generally not


able to crawl that much content from your site.


So it's not, I don't think it's specific to Google News.


It's just, in general, we're not able to crawl as much


from your site as we used to.


So is it generally with our website only, or do you see?


Because I've seen a lot of people also


tweeted the same thing, that they are also


facing similar issues where the indexing is very slow


or almost not.


From what I can tell, it's specific to your website.


So it's not a general change on our side.


We still crawl as much as we can.


And for pretty much the other websites


that I checked, all of the crawling was normal.


It's really just, from what I saw in the diagnostics,


we're not able to crawl as much for your site.


You probably already also see this


in Search Console in the Crawl Stats report,


where we show the number of pages


that we crawl from a website.


And I don't know exactly what it would look like to you,


but my feeling is that you would see there's


a reasonably high level.


And then over, I don't know, the last couple of weeks


or last couple of days, it goes down.


But that's usually a sign that there is something technical


somewhere on the hosting side, on your side,


where we're not able to crawl as much as we'd


like to be able to crawl.


And so we slow down the crawling.


And sometimes we slow down so much


that if you publish a lot of new content,


we don't have time to pick up all of that new content.


SPEAKER 2


Sounds good.


Thanks so much.


SPEAKER 3


Hi, John.


One question, if I may.


I saw that now you have the snippet control feature where


you can shorten the snippet.


Can we expect anytime soon to have a control


over the site links?


I don't think so.


So the main reason there is that site links, at some point,


we had site links set up in a way


that they were kind of a unique way of showing content


in search.


And for that, we had a control in Search Console


where you could say, show this one or show that one.


But now, site links are essentially just a separate


search results type.


So it's not a rich result that we would show like this,


but rather it's like we have multiple results


from your website, and we want to show them in search.


So we present them either as kind


of multiple results in the same search results page


or as one main result with multiple site links in that.


So we see them as essentially just normal multiple results


from the same site.


OK, the question is because for some of the clients we work


with, we see that in the site links,


we get pages that are 404.


For example, they remove the content,


but Google didn't crawl it to see that there's a 404 there.


OK.


Is there any way of making sure that doesn't happen?


I don't think there's ever a way of making


sure it doesn't happen because what usually this means


is we've indexed the page, and we haven't


seen that it returns 404 yet.


So what you can do there is when you remove a page,


you can also still include that in the sitemap file


for a bit longer with a last modification date of when


you remove that page so that the next time when we crawl


that page, we see a clear 404 status code,


and then we can drop it from the index.


The other thing that I sometimes see with regards to this


is that when a website doesn't use a 404 status


code for pages that are removed, then


for a certain period of time, we might


continue to index that page because we


think there's content here.


And then it takes a bit longer for us


to realize that, oh, actually there's an error message here.


It's not content.


That's my final point.


Yeah.


OK, thank you.


Sure.


Hey, John.


I wanted to second the last speaker that said that they


saw a drop in their crawl stats.


And we also actually have seen a big drop quite like it.


It's clear that the Google Watch activity


has dropped significantly during the past weeks.


So we've checked a lot of technical sites.


We've talked to our hosting.


We've talked to our CDN.


And everything looks just fine.


Like the site map hasn't been read for a week now.


And a lot of times we have to manually request the index.


I'm not sure if this is a bug or something.


Like the last ones that we've seen the past few months.


Or if this is something on our end,


I think we've pretty much looked everywhere


for some kind of issue to track it down and fix it.


Send me the URL and I can double check.


Maybe either here in the chat or on Twitter somewhere.


So I can pick a few.


I think yesterday with a screenshot of the crawl stats.


OK, I can put it here.


My question is about Google selected canonical URL.


We have two pages that are targeting the same, I guess,


user intent.


So the keywords are the same.


But one is a landing page that is like 700 words long.


Another one is a long form blog for studies


like 2,400 words long.


And so the landing page was published first.


And the blog post was published later.


So the blog post, Google thinks that it's


the duplicate of the landing page


and is not indexing the blog post.


Even though the set of keywords are the same,


but this log is a bit different.


Obviously, that one is landing page.


This is a blog post.


So what do you suggest we do in this case?


What are our options?


So I guess one thing I would double check


is to make sure that this is not a systemic issue


across the website.


If it's really just those two pages within the whole website,


then that's more a matter of us trying


to crawl and index the content there


and seeing that it's identical.


Sometimes what can happen is if you're


using JavaScript to load your content,


then maybe we're not able to pick that up properly.


If you're using some kind of lazy loading


to expand the landing page into the blog post,


if it's just the top part, then that


could be something that's happening there


where we index the same content on both of those URLs.


Otherwise, if we're able to see that the content is different


on both of these URLs, then we shouldn't


be treating them as canonicals.


And that's something, especially if it's significantly


different content, then that should be fairly straightforward


on our side.


If it's really a one-to-one match,


then that gets kind of tricky now.


So I'm happy to double check that.


The error is saying that it's a duplicate.


But when I even inspect it and look at the HTML


that Search Console shows, of course,


you're loading the whole thing.


It's clear.


And I can see the different headings and the text.


I can actually put both URLs here so you can have a look.


Sounds good.


All right.


Let me take a break here with the recording.


And I can stick around a little bit longer if any of you


want to hang around, but kind of just to lock in the recording


so that we don't make it all too long.


Thanks for joining in.


Thanks, all of you, for submitting questions and asking


questions live in person.


It's always good to see some of the activity here.


And I hope I'll see you all in one of the future Hangouts.


AJAX Crawling & Hash-bang URLs #AskGoogleWebmasters

Today, let's talk about AJAX crawling.


Oleg is asking, what's the current status of hashbang


AJAX crawling?


And how do I set up redirects?


Allow me to dwell in the myths of history a little bit.


The AJAX crawling scheme was something


we proposed in the early days of JavaScript sites,


way back in 2009.


So about 10 years ago, or one third of the way


back towards the invention of the web back in 2009,


that was a really long time ago.


And apparently, people are still using the AJAX crawling scheme.


Who would have thought?


Well, this worked great for a number of years.


But over time, it became kind of redundant.


Search engines, or at least Google,


had learned how to render most pages like a browser would.


And in the meantime, were even using


a special version of Chrome for crawling and rendering.


In short, for Google, you would have


no longer needed to do anything special for hashbang URLs.


We'll just try to render them directly.


In order to move to a different URL structure,


you need to use JavaScript on these pages


to create the redirects.


It's not possible to use server-side redirects,


since everything after a hash, so the number symbol,


is not sent to the server, but rather processed in the browser.


Once you've set up those redirects,


as Googlebot redirects, you can use the same redirects


to create those redirects.


As Googlebot reprocesses the hashbang URLs on the site,


it'll spot the redirect and follow it appropriately.


I hope this has answered your question, Oleg.


There are more episodes of Ask Google Webmasters coming


to the Google Webmasters YouTube channel,


so I'd recommend subscribing to the channel


to get notified whenever there's a new video.


Till the next episode.


Trailblazer Certainly needs to this day to be shown


once they've materials added.


One of my favorite things about Google is the awesome


view of these papers already.


And while we haven't covered the shelves yet,


these paper facts have shown Google and Werner


Structured Data Properties (Google Developers Site vs Schema.org) #AskGoogleWebmasters

Today's question comes from Abhishek. Abhishek is asking if you need to use structured data


exactly as specified in the Google Developer documentation, or if you can add more properties


from schema.org. Google uses structured data in search for both supporting rich results


types as well as for better understanding the pages themselves. With that in mind, in


order to be eligible to be shown as a rich result, you need to make sure that the page


uses the right structured data and that it complies with the appropriate policies on


our side. However, even without the structured data leading to rich results, our systems


profit by understanding the pages better when they use structured data. Also keep in mind


that it's not guaranteed for us to show these rich results just because a page uses


the appropriate structured data. So in short, if you're aiming to be eligible for a specific


rich results type in Google Search, then you should definitely watch out for the requirements.


Independently you're always welcome to use structured data to provide better machine-readable


context for your pages, which may not always result in visible changes but can still help


our systems to show your pages for relevant queries. For more information about structured


data in general, check out the links in the description below and don't forget to submit


Location-specific websites #AskGooglebot

Data Lasts


Gaitl asked us about creating versions of a website


for regions within a country.


First of all, this is a pretty complex topic,


and it can cause headaches depending on how it's implemented,


so I'd recommend getting help from experts


or reaching out to the Search Central help forum for advice.


Take this video as a starting point.


Googlebot


The most important aspect here is that Googlebot


usually crawls and indexes a website just from a single location.


If you're showing different content by location,


only one version will be indexed for search.


Because of that, if there's something you consider important on your website,


make sure it's on the default content that's shown to all users.


And finally, if there's any page that you want to have findable in search,


make sure that you're not blocking users in other locations from reaching that page.


I hope that's a good starting point.


As I mentioned, it's good to get help on this topic since it can get a bit complex.


And that's it for this episode of Ask Googlebot.


Drop us a note with the hashtag AskGooglebot if there's something on your mind.


And of course, don't forget to subscribe to stay up to date.


Let’s talk sitemaps


Let’s talk sitemaps

Hello, and welcome to another episode of Search Off the Record,


Let’s talk sitemaps

a podcast coming to you from the Google Search team,


Let’s talk sitemaps

discussing all things search and having some fun along the way.


Let’s talk sitemaps

My name is John, and I'm joined today


Let’s talk sitemaps

by Lizzie and Gary from the Search Relations


Let’s talk sitemaps

team, of which I'm also a part of.


Let’s talk sitemaps

Say hi, Lizzie.


Let’s talk sitemaps

LIZZIE


Let’s talk sitemaps

Maybe half a year ago or so, we had an episode about robots.txt,


Let’s talk sitemaps

and it was a fun episode.


Let’s talk sitemaps

And then I was thinking about what


Let’s talk sitemaps

else is similar to robots.txt, and we ended up with sitemaps.


Let’s talk sitemaps

And I happen to know that you were involved in sitemaps


Let’s talk sitemaps

in some ways back in the days, like 1902 AD.


Let’s talk sitemaps

And maybe you could talk about your experience with sitemaps.


Let’s talk sitemaps

And then we probably just deep dive


Let’s talk sitemaps

into certain parts of sitemaps.


Let’s talk sitemaps

Cool.


Let’s talk sitemaps

OK.


Let’s talk sitemaps

So I guess you could say sitemaps


Let’s talk sitemaps

is how I snuck into Google, which may be a good thing.


Let’s talk sitemaps

Who knows?


Let’s talk sitemaps

We'll see what happens.


Let’s talk sitemaps

I don't really know much about the story


Let’s talk sitemaps

internally with regards to sitemaps way


Let’s talk sitemaps

in the beginning, but I kind of saw that externally.


Let’s talk sitemaps

So I was active at my own software company


Let’s talk sitemaps

and kind of interested in the web


Let’s talk sitemaps

and then somehow got interested into SEO.


Let’s talk sitemaps

And then as it happens, Google happened


Let’s talk sitemaps

to launch sitemaps right around that time.


Let’s talk sitemaps

And I thought, well, this is a cool way to sneak into Google.


Let’s talk sitemaps

So I started trying to look into that a little bit.


Let’s talk sitemaps

And I noticed that there were no sitemaps generators


Let’s talk sitemaps

in the early days or no kind of easily usable sitemaps


Let’s talk sitemaps

generators.


Let’s talk sitemaps

So I made one of the first sitemap generators


Let’s talk sitemaps

for Windows at the time.


Let’s talk sitemaps

GARY ILLYES-SCHREIBER.


Let’s talk sitemaps

I think sitemaps was launched in 2005, maybe.


Let’s talk sitemaps

Like in the very, very beginning,


Let’s talk sitemaps

it was a Google initiative.


Let’s talk sitemaps

And then later, it became supported by other search


Let’s talk sitemaps

engines as well.


Let’s talk sitemaps

Right around that time, Google also published script


Let’s talk sitemaps

to create sitemaps.


Let’s talk sitemaps

How your generator was different from that script?


Let’s talk sitemaps

Well, I made a generator for Windows.


Let’s talk sitemaps

So instead of having to run some obscure script


Let’s talk sitemaps

in this weird programming language


Let’s talk sitemaps

that nobody knew about at the time called Python,


Let’s talk sitemaps

so I made something that was more usable by, I don't know,


Let’s talk sitemaps

like average site owners, or at least I thought it would be.


Let’s talk sitemaps

So basically, you enter your website,


Let’s talk sitemaps

and then it goes off and crawls your website


Let’s talk sitemaps

based on some settings that you provide.


Let’s talk sitemaps

And then in the end, when it knows about everything,


Let’s talk sitemaps

it generates a sitemap file for you.


Let’s talk sitemaps

I thought that was pretty neat at the time.


Let’s talk sitemaps

And the sitemaps project was, I think,


Let’s talk sitemaps

launched between different teams in Mountain View,


Let’s talk sitemaps

in Kirkland, and in Zurich.


Let’s talk sitemaps

So there was definitely a big team in Zurich at the time,


Let’s talk sitemaps

which I didn't really realize.


Let’s talk sitemaps

I was like, I don't know, as someone externally from Google,


Let’s talk sitemaps

you just see, oh, Google, and you have no idea


Let’s talk sitemaps

what is actually behind it.


Let’s talk sitemaps

But it was interesting because at some point,


Let’s talk sitemaps

I got invited to chat with the team in Zurich.


Let’s talk sitemaps

It was interesting to meet some of the people


Let’s talk sitemaps

that were active there.


Let’s talk sitemaps

And I think one of the people that I met there still


Let’s talk sitemaps

works in Zurich, so that was pretty cool.


Let’s talk sitemaps

I think the initial initiative from the sitemap side


Let’s talk sitemaps

or from Google was really about understanding the web


Let’s talk sitemaps

a little bit better, making it easier to crawl and find


Let’s talk sitemaps

all of the content on the web.


Let’s talk sitemaps

Did you feel like that helped at the time, when you started


Let’s talk sitemaps

using a sitemap?


Let’s talk sitemaps

Did that help with your early SEO experiences,


Let’s talk sitemaps

trying to get your software company site found?


Let’s talk sitemaps

To some extent, I thought, especially the process leading


Let’s talk sitemaps

up to that was really helpful.


Let’s talk sitemaps

I don't know, SEOs did it at the time,


Let’s talk sitemaps

but it's kind of that process of crawling your website


Let’s talk sitemaps

was really eye-opening.


Let’s talk sitemaps

Because in the early days, you're like, well,


Let’s talk sitemaps

Google is this big, magic, black box,


Let’s talk sitemaps

and nobody really knows what it's actually doing.


Let’s talk sitemaps

And then when you crawl your website yourself,


Let’s talk sitemaps

you realize, oh, there are actually


Let’s talk sitemaps

a lot of technical details that are involved with crawling,


Let’s talk sitemaps

and there are a lot of things that you can do right


Let’s talk sitemaps

or that you can do wrong on your website.


Let’s talk sitemaps

And that, I thought, was really interesting.


Let’s talk sitemaps

So regardless of whether or not sitemap files actually


Let’s talk sitemaps

helped with the visibility of your website,


Let’s talk sitemaps

that step of creating a sitemap file


Let’s talk sitemaps

forced you to look at your website


Let’s talk sitemaps

and think about, well, what are all of the URLs


Let’s talk sitemaps

that Google could find, and why is it not finding this part?


Let’s talk sitemaps

And what's up with all of these parameters,


Let’s talk sitemaps

and upper and lower case, and all of these things


Let’s talk sitemaps

where when you crawl your website,


Let’s talk sitemaps

it's suddenly this infinite mess.


Let’s talk sitemaps

And when you see that for the first time, you realize, well,


Let’s talk sitemaps

actually, this is something that I can control.


Let’s talk sitemaps

And this is something that a site owner


Let’s talk sitemaps

can kind of work on to make it easier for search engines


Let’s talk sitemaps

to crawl.


Let’s talk sitemaps

So then did you make some changes on your website


Let’s talk sitemaps

based off of the learning exercises?


Let’s talk sitemaps

And if so, what kind of changes did you do?


Let’s talk sitemaps

I don't know the details of what I changed on the website.


Let’s talk sitemaps

But things like URL parameters were super common.


Let’s talk sitemaps

And kind of understanding that using random URL parameters,


Let’s talk sitemaps

like session IDs and URL parameters at the time was,


Let’s talk sitemaps

I don't know, it was super common to have,


Let’s talk sitemaps

that you just have this really long number


Let’s talk sitemaps

as a parameter attached.


Let’s talk sitemaps

And every user gets a different number.


Let’s talk sitemaps

And that's something that in the early days,


Let’s talk sitemaps

you would look at a website and say, well, it is how it is.


Let’s talk sitemaps

And like, I'm not supposed to understand


Let’s talk sitemaps

all of these things.


Let’s talk sitemaps

But when you crawl it, you realize, well, actually, this


Let’s talk sitemaps

makes it pretty much impossible to crawl the website properly


Let’s talk sitemaps

unless the search engine can figure that out.


Let’s talk sitemaps

And if you can figure it out for the search engine,


Let’s talk sitemaps

it makes it a little bit easier.


Let’s talk sitemaps

I noticed this on our website.


Let’s talk sitemaps

But also, when I made the generator,


Let’s talk sitemaps

other people were using it.


Let’s talk sitemaps

And they would contact me and say, well,


Let’s talk sitemaps

I ran your tool on my website.


Let’s talk sitemaps

And it's not stopping.


Let’s talk sitemaps

And then you kind of are forced to look at other people's


Let’s talk sitemaps

websites and try them out as well.


Let’s talk sitemaps

And then you notice that these kind of crawling issues,


Let’s talk sitemaps

they're just everywhere.


Let’s talk sitemaps

I think a lot of that has gotten significantly better


Let’s talk sitemaps

because people use more common CMS systems.


Let’s talk sitemaps

And they don't generate this kind of messy website anymore.


Let’s talk sitemaps

But at least back in the early days,


Let’s talk sitemaps

it was super common to have a website that was pretty


Let’s talk sitemaps

much impossible to crawl.


Let’s talk sitemaps

GARY ILLYES-SELLSBERG I think the session ID, that was one


Let’s talk sitemaps

of those things that it was pretty much transparent


Let’s talk sitemaps

to a human eye.


Let’s talk sitemaps

Basically, it was so prevalent on the internet


Let’s talk sitemaps

that you didn't even notice as a human in the URL.


Let’s talk sitemaps

And you are just assuming that it's not there.


Let’s talk sitemaps

But for a search engine or for any crawler,


Let’s talk sitemaps

basically that meant that there's


Let’s talk sitemaps

an infinite number of URLs, well, pseudo-infinite number


Let’s talk sitemaps

of URLs on the site.


Let’s talk sitemaps

And as with any crawler, crawlers


Let’s talk sitemaps

are happy to crawl their URLs.


Let’s talk sitemaps

And sitemaps probably were a very eye-opening thing


Let’s talk sitemaps

for webmasters and site owners and developers in general.


Let’s talk sitemaps

What about the different tags that can show up in a sitemap?


Let’s talk sitemaps

Because I'm fairly certain that most people who


Let’s talk sitemaps

dealt with sitemap know that you have the loc lock tag, where


Let’s talk sitemaps

you put the URL, and then you have a bunch of other tags,


Let’s talk sitemaps

like priority and change frequency, that are basically


Let’s talk sitemaps

covered in myth.


Let’s talk sitemaps

And some people think that search engines use them.


Let’s talk sitemaps

Some other people think that search engines don't use them.


Let’s talk sitemaps

How were those with your generator?


Let’s talk sitemaps

Taking a step back, the sitemap files themselves


Let’s talk sitemaps

are basically text files.


Let’s talk sitemaps

And you can look at them in a text editor, which at the time,


Let’s talk sitemaps

that was kind of interesting for me to see.


Let’s talk sitemaps

I expected to see some, I don't know, machine language file.


Let’s talk sitemaps

But XML is essentially like HTML pages.


Let’s talk sitemaps

And you have different tags and different content in there.


Let’s talk sitemaps

And the main tags there for sitemap files


Let’s talk sitemaps

are really like the URL.


Let’s talk sitemaps

You specify the URL.


Let’s talk sitemaps

I don't even know what they're all called nowadays


Let’s talk sitemaps

or what they're still called.


Let’s talk sitemaps

But there are also extra fields that you can add,


Let’s talk sitemaps

which I think are optional, like the last modification


Let’s talk sitemaps

date, the change frequency, and the priority.


Let’s talk sitemaps

And I'm sure I'm forgetting something, but something


Let’s talk sitemaps

along those lines.


Let’s talk sitemaps

And the interesting thing, I think,


Let’s talk sitemaps

is the assumption I have from the sitemap side


Let’s talk sitemaps

is that Google wanted to understand a little bit better


Let’s talk sitemaps

which pages are changing how frequently


Let’s talk sitemaps

and which pages you think are important.


Let’s talk sitemaps

And that's kind of with the change frequency


Let’s talk sitemaps

and the priority data in the sitemap file.


Let’s talk sitemaps

But it feels like that was something


Let’s talk sitemaps

that was more like wishful thinking.


Let’s talk sitemaps

Maybe we can learn more about the web like this.


Let’s talk sitemaps

Because in practice, of course, if you give people


Let’s talk sitemaps

a field that says priority, they're


Let’s talk sitemaps

going to say, my website is the most important.


Let’s talk sitemaps

And all of my pages are the most important.


Let’s talk sitemaps

And using that as a way to understand more


Let’s talk sitemaps

about the website is then really hard


Let’s talk sitemaps

because people are just biased.


Let’s talk sitemaps

And they think their stuff is the most important.


Let’s talk sitemaps

Well, but is the priority supposed to be on the web


Let’s talk sitemaps

or within the context of your own site?


Let’s talk sitemaps

Because I guess that would be a good exercise


Let’s talk sitemaps

to prioritize within your own site which ones are the things


Let’s talk sitemaps

that change more often.


Let’s talk sitemaps

So why wouldn't you actually go through that exercise


Let’s talk sitemaps

unless you're thinking like, oh, this is


Let’s talk sitemaps

like me, my website compared to your John Mueller website.


Let’s talk sitemaps

I think mine is priority number one.


Let’s talk sitemaps

I think you are being way too rational.


Let’s talk sitemaps

Oh, OK.


Let’s talk sitemaps

I'm not accounting for other things on the internet.


Let’s talk sitemaps

I mean, the internet itself is not a rational place.


Let’s talk sitemaps

And if you are a small business owner, for example,


Let’s talk sitemaps

then why would you want to do that exercise, basically?


Let’s talk sitemaps

You just want to say that, well, I published these pages.


Let’s talk sitemaps

And all these pages are important to me.


Let’s talk sitemaps

So here.


Let’s talk sitemaps

I guess, is the crawl budget a thing here?


Let’s talk sitemaps

Most people don't even know about crawl budget.


Let’s talk sitemaps

There's a few bigger entities on the internet who


Let’s talk sitemaps

made crawl budget a thing.


Let’s talk sitemaps

But before, I don't want to say a stupid date,


Let’s talk sitemaps

but I will say 2013, I rarely ever heard


Let’s talk sitemaps

of crawl budget.


Let’s talk sitemaps

And then suddenly, it came to be.


Let’s talk sitemaps

And then we started talking about it because reasons.


Let’s talk sitemaps

I think the idea behind priority is kind of understandable.


Let’s talk sitemaps

But at the same time, if you're making


Let’s talk sitemaps

these files for any larger website,


Let’s talk sitemaps

you have to automatically fill out these values.


Let’s talk sitemaps

And you don't necessarily know what


Let’s talk sitemaps

is the relative priority of this random blog post that I have.


Let’s talk sitemaps

And at some point, you just say, well, everything is important.


Let’s talk sitemaps

Or you create this kind of artificial structure


Let’s talk sitemaps

of priority for your website.


Let’s talk sitemaps

But you can't really determine it yourself.


Let’s talk sitemaps

And at that point, the data is not really that useful.


Let’s talk sitemaps

And I think even in our documentation,


Let’s talk sitemaps

we now say we don't use priority from a sitemap file.


Let’s talk sitemaps

This is true.


Let’s talk sitemaps

Also goes for change frequency, I think,


Let’s talk sitemaps

where you can't actually expect to know when your page will


Let’s talk sitemaps

actually change.


Let’s talk sitemaps

How often should it change?


Let’s talk sitemaps

Because if you have the terms of service, for example,


Let’s talk sitemaps

or if we go to our documentation to developers.google.com


Let’s talk sitemaps

slash search, there are pages that we


Let’s talk sitemaps

haven't touched for two years now because we just


Let’s talk sitemaps

had no reason to touch them.


Let’s talk sitemaps

But when we published those pages,


Let’s talk sitemaps

we wouldn't have known that we are not going


Let’s talk sitemaps

to touch them for two years.


Let’s talk sitemaps

OK.


Let’s talk sitemaps

So there's the change frequency thing.


Let’s talk sitemaps

But then there's also the last mod thing.


Let’s talk sitemaps

I mean, the last modification date


Let’s talk sitemaps

is something that I would say there is an absolute value


Let’s talk sitemaps

that you can supply there.


Let’s talk sitemaps

And that's something that the script can look at.


Let’s talk sitemaps

And if it looks at your pages and says,


Let’s talk sitemaps

well, I updated this page one year ago or last week,


Let’s talk sitemaps

it's a real date that you can supply.


Let’s talk sitemaps

Whereas with the change frequency,


Let’s talk sitemaps

you don't really know in advance how often it'll change.


Let’s talk sitemaps

And it's more that search engines could, over time,


Let’s talk sitemaps

track how often this page changes on average.


Let’s talk sitemaps

And they could use that to determine how often


Let’s talk sitemaps

to re-crawl it.


Let’s talk sitemaps

So at that point, why would a site owner


Let’s talk sitemaps

specify that directly?


Let’s talk sitemaps

Because it's much more tempting to say,


Let’s talk sitemaps

well, this page could change every day, even if it doesn't.


Let’s talk sitemaps

GARY ILLYESISIOSAITIS-SABATO-MARTIN I think we are not


Let’s talk sitemaps

doing a great job explaining when you should update that tag


Let’s talk sitemaps

because it should be something like last significant update,


Let’s talk sitemaps

like when you are updating the content itself,


Let’s talk sitemaps

not some head tag or element in the HTML.


Let’s talk sitemaps

I disagree.


Let’s talk sitemaps

OK.


Let’s talk sitemaps

I know that some search engines use it.


Let’s talk sitemaps

For example, Bing, I know that Bing is using it.


Let’s talk sitemaps

And Google doesn't use it because reasons.


Let’s talk sitemaps

And one of the reasons is that it's highly unreliable


Let’s talk sitemaps

because people want search engines to believe


Let’s talk sitemaps

that their page changed so it should be crawled.


Let’s talk sitemaps

But in reality, the page didn't even change, for example,


Let’s talk sitemaps

or it changed just a little.


Let’s talk sitemaps

I think it's trickier in that regard.


Let’s talk sitemaps

I mean, it's still like you can pull out the primary content


Let’s talk sitemaps

and say this content changed.


Let’s talk sitemaps

But at the same time, you could change something


Let’s talk sitemaps

in your heading or something in your footer


Let’s talk sitemaps

or in the sidebar that has links to other pieces of content.


Let’s talk sitemaps

And technically, that's a change on your page.


Let’s talk sitemaps

And technically, that's something


Let’s talk sitemaps

that search engines could find value in.


Let’s talk sitemaps

So maybe the issue is more that there's


Let’s talk sitemaps

difference of opinion on what the date should be


Let’s talk sitemaps

and then at that point, it's like, well,


Let’s talk sitemaps

if people mean different things with the same value,


Let’s talk sitemaps

what can search engines do with it?


Let’s talk sitemaps

Well, that's fair.


Let’s talk sitemaps

Well, and you bring up a good point, significant changes


Let’s talk sitemaps

to search engines or to users.


Let’s talk sitemaps

Because maybe is that different?


Let’s talk sitemaps

What would be considered significant or an interesting


Let’s talk sitemaps

change, like just changing a link?


Let’s talk sitemaps

Or, oh, we added this reference or something.


Let’s talk sitemaps

This could be a new page for a search engine to identify.


Let’s talk sitemaps

But for a user, it's just like, well, that's another link.


Let’s talk sitemaps

OK.


Let’s talk sitemaps

I mean, it could be something like adding structured data,


Let’s talk sitemaps

where the user doesn't see any change at all.


Let’s talk sitemaps

But for the site owner, it's really important


Let’s talk sitemaps

because suddenly you're providing information


Let’s talk sitemaps

for search engines that they could show in a different


Let’s talk sitemaps

snippet, for example.


Let’s talk sitemaps

All right, fair point.


Let’s talk sitemaps

I'll buy in.


Let’s talk sitemaps

But I mean, kind of this discussion of what


Let’s talk sitemaps

is actually a change that should be flagged as a date,


Let’s talk sitemaps

doing that in an automated way across a larger website,


Let’s talk sitemaps

I imagine that's pretty tricky.


Let’s talk sitemaps

Well, the change frequency or the last mod?


Let’s talk sitemaps

Because the last mod seems like that could be OK


Let’s talk sitemaps

because it's like in the past.


Let’s talk sitemaps

I think the change frequency, you can't really


Let’s talk sitemaps

know ahead of time.


Let’s talk sitemaps

But that last modification, even that


Let’s talk sitemaps

feels like something where people might say,


Let’s talk sitemaps

oh, well, the last time I edited this article


Let’s talk sitemaps

or the last time the HTML changed.


Let’s talk sitemaps

Right.


Let’s talk sitemaps

I imagine that you are updating something


Let’s talk sitemaps

in the head for the whole site.


Let’s talk sitemaps

Like you are injecting a verification tag, for example.


Let’s talk sitemaps

And then it propagates across all your pages.


Let’s talk sitemaps

And you have 2 million pages.


Let’s talk sitemaps

And suddenly, all the last mod tags


Let’s talk sitemaps

are updated to basically now.


Let’s talk sitemaps

Is that useful?


Let’s talk sitemaps

I doubt it.


Let’s talk sitemaps

I don't know.


Let’s talk sitemaps

Or you change your copyright date at the end of the year.


Let’s talk sitemaps

It's like copyright 2022.


Let’s talk sitemaps

We've actually seen that.


Let’s talk sitemaps

I remember someone from the sitemap team back in the days


Let’s talk sitemaps

was complaining.


Let’s talk sitemaps

That was a real issue, that when New Year's


Let’s talk sitemaps

hit, the large portion of the change frequencies


Let’s talk sitemaps

updated to January 1.


Let’s talk sitemaps

So if it was an issue, that means it was used.


Let’s talk sitemaps

I can confirm or deny anything without the explicit approval


Let’s talk sitemaps

of the secretary.


Let’s talk sitemaps

Another thing that kind of came out with sitemaps.


Let’s talk sitemaps

So I thought two kind of semi-related things


Let’s talk sitemaps

were pretty cool at the time.


Let’s talk sitemaps

So the standard was kind of announced, or the beta.


Let’s talk sitemaps

I don't know how they framed it in the early days.


Let’s talk sitemaps

But they also created this kind of console thing


Let’s talk sitemaps

where site owners could go and verify their site


Let’s talk sitemaps

and add sitemap files.


Let’s talk sitemaps

Webmaster Tools.


Let’s talk sitemaps

Yeah.


Let’s talk sitemaps

The early Webmaster Tools.


Let’s talk sitemaps

Sitemaps.


Let’s talk sitemaps

Google Sitemaps.


Let’s talk sitemaps

Tool?