That is fair.


Cheese, Web Workers, universal search, and more!

So in the previous episodes, we were talking about serving.


And I think we are finished with ranking things more or less.


There is one interesting thing in most search engines, which


is universal search.


It's not Google specific.


Other search engines do it as well.


And because most search engines rank results roughly


the same way with different flavors,


this is applicable probably to every search engine


that I can think of.


So during the ranking process, we


assigned a score to every result that we


retrieved from our index.


We talked about the web index.


But the reality is that when we created our rewritten query,


the parsed query, then we sent that to every index


that we have.


So for example, if we have an index for images,


if we have an index for videos, if we have an index for news,


and so on, they would also get that parsed query.


And then they would do the same thing


that we explained in case of the web index, which is basically


console posting list, for example.


And then based on that, retrieve some documents that is then


ranked or assigned a score.


Now, once they've found stuff, then they


will return that to something called,


let's make up a name, Blender or Mixer.


I don't know, something, Universal Mixer.


And that will take all the results that came up


from all the different indexes and try to mix them together.


So the reality is that the result page is actually


very similar to an auction hall, where results are actually


bidding for their desired positions.


So for example, the featured snippet


might want the first position.


So it will submit a score that I can bid


this much for this position.


It can also say that I don't want


second position or third position or fourth position


or so on.


I only want the first position.


And then we look at the web results, for instance.


And I'm just gliding with my hand again,


even though I'm on a podcast.


I don't know why I do that.


I always do that.


So we have the feature snippet, which bid, let's say,


1,000 for the first position.


Then we have the first or the next web result, which


has a score of 1,500, let's say.


Then that means that that web result outbid the featured


snippet.


But because the feature snippet said


that I can bid 1,000 and 1,000 only,


and I only want the first position,


it will just fall out the result set,


because it didn't want the second place.


It bid for the first place and first place only.


The next web results that have scores assigned to them,


they will naturally follow.


Another thing that happens in Universal Mixer


is that, as I said, we take the results from the other indexes,


and then we will try to mix them into the result set.


Now, we have preferred positions for some things.


Like, for example, the video results,


they are just highlighted by assigning a thumbnail nowadays


to the web result where the video is found in.


Let's say johnmuller.com will get a video,


because John will put up a video of how


to make fondue, for example.


And then the way that's highlighted


is by assigning the screenshot or thumbnail from the video


to the web result. But in case of images,


we actually get a universal result, which is, nowadays,


is more like a carousel, I think, especially on mobile,


where you can scroll horizontally


through the images, I think.


That might not be completely true.


I use my own search engine.


I developed one yesterday.


It's called Steve.


Anyway, so we have preferred positions for those,


and we will try to match them into or among the web results.


What else do we have there?


Then we have certain search features


that always appear roughly at the bottom of search's


old page, like, for example, the related searches, which


is a feature that also gets the parse query or the interpreted


query.


And then based on that, it will return


a set of queries that are related to the original query,


in some sense.


But that feature doesn't want to take up space in the browsing


area, or where you would expect the searches also.


It's usually positioned at the bottom.


What else do we have?


News.


So the related searches on the bottom, I guess,


would need to bid something really low,


or would they also say, my preferred position is 10,


or whatever?


I think it's more like that.


My preferred position is this, and I


can bid this much for that.


Perhaps related searches is not a great example,


because you are not competing for a spot.


It's not like you are taking up the space of a result,


but it's more like you are extending the search's old page


with a new visual element.


Perhaps it's more similar to knowledge panel, I guess.


OK.


So it's almost like all of these different indexes


or kind of content have their own search engine.


And basically, they're saying, my result


is super relevant or kind of relevant.


And then there's a super search engine


on top of all of these search engines


that mixes them all together?


GARY ILLYESISIOSAKE LASCARIS-MARTINI


Technically, yes.


I mean, many of our systems are shared.


For example, Googlebot is not used only for web trolling.


It's used for fetching ads landing pages as well.


It's using a different user agent


for when it's doing some other job.


But it's still a shared system.


It's used by many teams.


And the same way the retrieval systems


might be used for different things as well.


So for example, if I need a featured snippet,


for example, then I can technically


use the same retrieval system that we


use for retrieving web results.


Because essentially, it's doing the same thing,


but perhaps differently.


Something has to be different about it.


And the systems, or many of these systems,


but especially the retrieval system,


is modular enough that you can just plug in your own module


that you wrote for ranking stuff differently.


Like for example, with images, you


have to rank the landing pages slightly differently


because you are focusing on the image, not the landing


page itself necessarily.


JOHN MUELLER-


So cool.


And you mentioned they can bid for individual positions.


How do those positions map to things like Search Console?


Webmaster Tools?


So every position has a number assigned to it.


For example, we have position 1, 0, whatever.


And it doesn't matter all that much what


we put in that position.


We can just say that there was something in that position.


Usually when we log, I imagine that you are talking


about impressions and clicks.


I haven't been to Search Console for a long time.


So I imagine.


No.


Yeah, I only use Steve Console.


Anyway, so since you are talking about impressions and clicks,


we can record that there was, like this


was the query that is obvious.


And then there was an impression of this position.


Basically, you showed up at this position.


We don't necessarily have to say what you showed up with.


Like for example, you could have been in a feature snippet,


which is just an expanded result essentially with smarter


snippetting, I guess.


But it was just a result. And then we


record that impression happened at this web result position.


And then when someone taps on it,


then we can just say that at that web result,


there was a tap on or a click for this page for this query.


And then we aggregate all that and present it


in Search Console.


OK, so here's a controversial question.


Oh, I'm glad that you found me for this.


OK, so featured snippets, you mentioned them a few times.


Externally, people say this is ranking at position zero.


OK.


Is it zero?


I don't know.


Never looked it up.


I mean, if you ask a developer, then


whoever worked with arrays, for example, then it will be zero.


It will be zero.


But if you have a normal web result as the first one,


is that also zero?


No, that's one, unless you're a developer,


because then it's zero, because it's an array.


Am I confusing enough?


I, um.


OK, so the non-funny answer is that I actually


have no idea what position that would be.


I could probably look it up, but I really don't want to.


OK, but it's like the general notion that is sometimes


seen externally of, like, if your web result ranks first,


that's position one.


But if you have a featured snippet, that's position zero.


That doesn't make that much sense, or does it?


Might make sense.


It's like.


Well, if you want my interpretation,


then I think it doesn't make sense.


OK.


I, like, externally, I did say position zero a few times just


to use the same language as SEOs.


Oh, it comes from you.


What?


No, no, no.


Why would you say that?


This is how rumors started.


Well, it's a good thing that we are off the record.


But I would argue that it is position one, not


the position zero.


Yeah, I also noticed, like, depending


on the programming languages, they also


have this either zero or one for the first item in the array.


So it's, like, super confusing.


Someone should figure that out.


It's like all of these off by one errors.


I bet it's, like, lots of things break.


Do they, though?


Do they?


I don't know.


I mean, everything that has software in it breaks.


I think that's part of the universal law.


It's not a bug.


It's a random feature.


It's like a fun surprise waiting for you to be discovered.


OK.


That's one way to kill a discussion.


Just bring up arrays.


No.


Fine.


Would you prefer linked lists, Gary?


Oh, god.


How about some graphs?


Graphs I actually don't mind.


I actually like working with graphs.


Cool.


Unless they're represented as an adjacency matrix.


I don't like that representation too much, to be honest.


This is getting, like, way too complicated technical for me.


But if we keep going on for one more minute and 45 seconds,


then we pass the one hour recording.


Oh my gosh.


OK.


You should see our producer face palming very, very hard


in the background right now.


Well, luckily, this is a podcast.


Nobody can see anything.


So we're all safe.


Woo-hoo.


That's true.


That's why I'm saying it.


We're off the record, after all.


We're all off the record.


Cool.


All right.


I'm sure there will be zero cuts.


We'll just, like, upload this uncut straight


to the internet.


Yeah.


OK.


Well, in that case, then there's nothing left for me


to say other than that's it for this episode of the Search


Off the Record podcast.


We've been having fun with these podcast episodes.


And I hope you, the listener, have


been finding them both entertaining and insightful as


well.


And at any rate, let us know if there's anything that you'd


like to have covered differently,


if you'd like to have some web workers featured as guests


along the way, or if there are other topics that we


should be covering as well.


And of course, don't forget to like and subscribe.


And see you all next time.


Bye.


English Google Webmaster Central office-hours from October 29, 2019

All right.


Welcome, everyone, to today's Webmaster Central Office Hours


Hangouts.


My name is John Mueller.


I'm a Webmaster Trends Analyst here at Google in Switzerland.


And part of what we do are these Webmaster Office Hours,


where people can join in and ask any web search, Webmaster


related questions.


And it looks like lots of people are joining in, which is cool.


We'll see.


Bunch of questions were submitted already on YouTube.


But as always, if any of you have a burning question


on your mind that you really need


to get answered as quickly as possible,


feel free to jump on in now.


If not, OK.


Well, that's fine.


OK.


I'll get started with the questions that were submitted.


If anything pops up in between, or if you have any comments,


or want further clarifications along the way,


just feel free to jump in.


Maybe I'll just ask one question.


All right.


So I figure the burnt stuff obviously


has significant impact, according to Google, on the


search results.


But Danny Sullivan tweeted that there's really nothing


to optimize for with burnt, I guess similar to when Google


said there's nothing to optimize for with blank brain.


So this is kind of similar to burnt


and blank brain in that sense.


Is that true?


If burnt is really looking at natural language content?


That I think.


I mean, if you're looking, when you think about it,


burnt's about language, how things are written.


If SEOs are writing for machines,


and then they start writing for burnt, is that better?


I mean, you understand why SEOs are confused by that.


I see articles left to right.


This is how to get your website ready for burnt,


this and that.


I kind of want to say this is really,


you're just wasting your time reading these articles,


but is that true?


Are you wasting your time reading articles about burnt,


or these SEO tips around burnt?


Just wanted to get your opinion on that, more long form


than just a tweet from Danny Sullivan.


I really don't have any insight into burnt,


so I haven't looked into that at all.


My understanding is it's really just


about understanding the natural language better.


So that's something.


If that's really the primary focus,


then essentially the way to optimize for it


is to write naturally.


So that kind of falls into the camp of, well,


there's nothing kind of specific that you


need to do to optimize for this.


But I haven't really looked into this.


Danny's been on top of this, and if he


tweets that there is nothing to do,


then I would pretty much trust that.


OK, we should trust Danny.


Got it.


Yeah.


It's like, believe what he says.


If you don't believe me, at least believe him.


He's closer to the outside world by number of years, I guess.


I guess.


All right, thank you.


OK.


All right, so let me jump in with some of the questions that


were submitted, and we'll see how far we get.


And as I mentioned, if there's anything along the way


that you really need to jump in and ask more clarification on,


feel free to do that.


The first one, is there any YouTube signal


used by Google to rank a specific content on Google


search results?


I don't think there's anything specific to YouTube.


So in all of the dealings I've had with the search


infrastructure teams, we try to treat YouTube


like any other website.


And there is a lot of content there,


but it's not that there is anything specific to YouTube.


If someone were to mention a website on YouTube,


then suddenly that would be more important.


It's a website like any other.


There can be links on there to external content.


It can rank in the search results.


It's essentially normal pages.


New content on a few of our sites,


and it's not indexing in Google.


Based on when this started, this seems


to be related to the August 8th bug in the system.


But while it was reported as fixed,


it's still an ongoing issue in some of our sites.


I reported this in the forum, but had no reply.


So I have not had a chance to look at that forum thread.


Looks like the question was just posted 13 minutes ago,


so not that fast.


But in general, if you're still seeing issues


from starting around August 8th, then that


would not be related to that.


So those kind of indexing issues that we had back then,


those will resolve fairly quickly.


And it would not be the case that one site got forgotten


in a batch of indexing issues.


Instead, what I would suspect is just


that your site is seeing some ranking changes there, perhaps,


maybe some technical issues on the site itself that


have persisted since then.


And that wouldn't be related to the indexing bug.


That would be essentially something that would


happen independently anyway.


But I haven't had a chance to look at your threads.


I don't know exactly which site it is


and what exactly might be happening there.


Search Console shows a number of impressions


a site gets at a keyword level.


However, keywords that rank number 50


seem to get thousands of impressions,


which is confusing, as few visitors


scroll to page 6 of the search results.


Could you explain how Search Console impressions


are calculated and if the listing needs


to be visible in order for an impression


to be registered for a keyword?


So Search Console is really based


on what was shown to users in the search results.


And it's done by search results page in the sense


that if something is on this page,


then we will count that as an impression, regardless


of if it's within the viewport of that search results page.


So that's something where if you're


seeing impressions at that ranking,


then that means we're showing those search results to someone.


There is a Help Center article about this


that goes into much more detail on which items


are counted how and how the position is calculated.


In particular, if there are multiple kind


of fancy elements on the search results page,


it might be that something like position number 15


is actually still on the first page.


But that's something you need to look at the individual search


results page for to see what could all


be playing a role there.


What also sometimes happens is that the sites


get an average ranking fairly low, or I guess,


fairly high position number, which


means it's ranking fairly low.


And it's just random people sometimes going there,


but not a lot of people.


So that's something that you would particularly


see for queries where you think the competition is very high,


where you might be surprised to see your site at all.


But if every now and then people go that far down in the search


results pages to see your site, then that would still count.


Regarding hreflang tags, we have set up sites


in a similar fashion to amazon.com and amazon.co.uk,


where we have mostly identical English language


content for two different geographic audiences


through different domain names.


We have not developed hreflangs between the URLs currently.


Would you advise we spend development time


to build this backend functionality,


or rather, put that time into improving the user


experience overall?


So I guess there are a few aspects here


that I'd like to touch upon.


First, just because a big site is doing something specific


doesn't mean that it's the correct thing to do


or that it's the right thing to do


for your site in your situation.


Most other websites out there won't be Amazon.


So if Amazon is ranking really well with one setup,


that doesn't mean that you can copy that exact setup


and rank just as high as Amazon, because you're probably not


Amazon.


So that's kind of the first thing to keep in mind.


And that applies to all kinds of big sites.


We often see that with people looking at google.com as well,


saying, well, google.com doesn't use this type of structured


data or doesn't have an h1 tag or something like that.


Therefore, nobody needs this kind of structured data,


or nobody needs a specific tag.


And that's something I would advise not following.


Just because a big site is doing something


doesn't mean that you should be doing the same


or that you should not be doing the same as they're doing.


You really need to adjust your strategy


for your specific case.


So that kind of out of the way.


With regards to hreflang link annotations,


essentially these would link two different pages


for either different languages or for different markets


together so that when someone is searching


and we would show one page, we would be able to swap out


the URL against the other one.


So that's something that, from my point of view,


sounds fairly simple but can be really complicated to set up,


especially if you have a lot of languages


or a lot of countries.


So I think it's good to ask if that makes sense to develop.


In general, when talking with the engineers about hreflang


in the beginning, they framed it as something,


if you're seeing a problem with local search results being


wrong, then hreflang might be one way to fix that.


If you're not seeing a problem with the search results


in the way that they're presented to users,


then probably you don't need to implement hreflang because it's


already working correctly.


So that's kind of one thing to put out there as well.


So if you already have your site set up


and it's working well in search and users


are finding the right version or making it


to the right country version as appropriate,


then maybe you don't need to do anything fancy for hreflang.


Also, hreflang is on a per-page basis.


So if there are individual pages where you're seeing a problem


or individual pages where you really


want to make sure that the right version is shown,


then you can just implement hreflang on those.


So that could be maybe something like a category page


or could be your home page where you say,


if people are searching for my brand,


I really want to make sure the appropriate local version is


shown.


Then maybe just do it on the home page


or on kind of higher level category or article pages


or whatever you have.


So that's kind of the direction I would head there.


On the one hand, if you don't see a problem,


then don't see this as something critical.


If you want to try it out, try it out


on individual pages where you think


this could have an effect.


And then from there, you can make a decision.


Does it make sense to expand this to more of the site


or not?


Another question about BERT where I really


don't have any useful answer.


Can you shed some light on whether a website could incur


a penalty by adding product schema to the website


when that product that we provide to the public


is a service?


Schema.org includes haircuts as types of products,


but it feels like this is a risk area.


We have thousands of reviews that are worth


showcasing and marking up.


And I'm noticing that services schema doesn't


tend to show review stars.


So that's, I think, an interesting question.


There are two things here.


On the one hand, if these are reviews


about your business in general and not about a specific


product that you're selling that anyone else could be selling,


then we would probably not show those rich results


for your website in the search results,


especially if you're putting review markup on your website


for your website or for your business,


then that would be, by policy, something


we would not want to show.


The other thing is if there is a specific schema or markup


type that matches the primary item of this page


and you're explicitly using something else


to try to get a specific type of search result,


then that would also not be OK.


So products and services is one thing here.


We often see it also with recipes, where you say,


well, my blog post is a recipe for success.


Therefore, I will mark it up as a recipe,


and I want to get all of that fancy rich results


for recipes.


And that doesn't make any sense.


So it should really, the type of structured data


should match what you're actually providing on that page


as a primary element.


Search Console shows I have 194,000 valid index pages,


while on site colon, it just shows 50,000.


Which one is correct?


I am tempted to say neither of these


are correct, because counting the number of index pages


is really hard.


But in general, Search Console would be the number


to base it off of.


The site query is something that we optimize for speed.


It's not optimized for accuracy.


It's not optimized for completeness.


And you may see widely different results


than from what you see from Search Console


or what we actually have indexed for a site.


So I would not use the site colon result


or the approximate count that is shown there


for any diagnostics purposes at all.


I would only focus on the Search Console number.


And even with the Search Console number,


I would tend to be critical when looking at that,


because that is essentially the total number of pages


that we have indexed, which is total number of URLs.


And with any larger website, you'll


know that there might be several URLs that lead


to the same piece of content.


So just because you have 194,000 URLs indexed,


that might only be, I'm just making up


a number, 120,000 pieces of content


that you actually have indexed.


So that 194,000 number is something


that you need to take with a grain of salt,


because it might not be an actionable number


to actually focus on.


One way you can make this a little bit more actionable


is to use sitemap files, because in the sitemap files,


you can explicitly list the URLs that you care about.


And in Search Console, you can filter for those sitemap files


and then see of those, I don't know,


40,000 that you have in this group of URLs,


this many are actually indexed.


And that's a little bit more actionable,


because then you know these are URLs that I explicitly


wanted to have indexed.


And of those, most of them got indexed,


or a small part of them got indexed.


And based on that, you can make decisions and say,


well, this is a problem I need to focus on,


or that seems reasonable, and there's nothing


I need to worry about here.


So that's kind of the direction I would head there.


Avoid using site colon.


Use Search Console index counts.


Or if you want to be more exact and have


more actionable information, use sitemap files.


I have one website that is not mobile-first indexed ready.


It has a really bad mobile pre-render


that's not working well.


After September 9, the number of crawl pages


per day in the old Search Console for the desktop site


dropped by 99%, from 6 million a day to 100,000.


The m.property usually had about 90k per day


and spiked to 1.7 million on September 8,


and then crashed back down.


The site is really big.


Site colon shows over 10 million indexed pages.


Server log showed the same thing.


Traffic-wise, nothing has changed so far,


but this information is really hard to understand.


I contacted Gary by email twice, and he didn't answer.


So Gary is a pretty busy person.


And in general, when people have our email addresses,


we can't guarantee that we're able to respond.


Sometimes lots of things are going on,


and sometimes we don't have anything specific


that we'd be able to answer back.


And it can just lead to a lot of back and forth.


That is not very useful for either side.


So just because Gary doesn't respond


doesn't mean Gary is ignoring it.


But things get busy sometimes.


With regards to the specific situation,


it sounds like our algorithms were trying to see


if mobile-first indexing would work there,


and it sounds like it didn't work out.


So I wouldn't necessarily worry about that.


However, at some point, we will be switching all sites


to mobile-first indexing.


And that's something that we will announce at some point


when that is ready or when we have the dates lined up.


But that will happen at some point.


So this is something I would personally


recommend to make sure that it's actually working a lot better,


that the mobile version contains the full content


that your desktop version would contain,


that the images line up, that the text is there,


the internal links are there, the URLs, the structure,


all of that works well.


On the one hand, we need that for indexing.


On the other hand, most users are using mobile devices too.


So if your mobile site doesn't work,


then that seems like something you should figure out fairly


quickly, because you're probably losing out


on a lot of traffic there.


With regards to the mobile pre-render,


I don't know, maybe if it's a JavaScript website


and you're using pre-rendering, then that


might be something that users wouldn't see so much.


But still, like I mentioned, at some point,


we will be switching all websites over.


And we have no plans to have any options so that you


can opt in or opt out of this.


This will essentially happen at some point.


So I would kind of get ready for that.


When we do announce that, we will give sufficient heads


up time so that people can prepare.


But it's not something that you can ignore forever.


Does non-responsive properties, such as HTML tables,


affect the ranking of a page on Google


after the shift to mobile-first indexing?


No.


So that's fairly straightforward in the sense


that if a page is not mobile-friendly,


that doesn't mean that we can't use it


for mobile-first indexing.


So if you need to zoom in on different parts


to see the content or swipe sideways and things like that


to get the content, the content is still there.


And we can still index those pages.


So in particular, desktop-only pages,


kind of the old-school websites that you


would make in a text editor before all


of the fancy CSS features and responsive design got popular,


those work completely fine with mobile-first indexing


because there is one version of the HTML,


and it works for all devices.


It's not easy to read on all devices, but it's available.


So that works completely fine with mobile-first indexing.


What will happen, though, is that we will not


see this page as being mobile-friendly.


And that can mean that we show it


a little bit lower in the search results in general on mobile.


Because if we know that a page is not friendly to mobile users


and people are searching on mobile,


then it kind of makes sense that we wouldn't rank it


above similarly relevant pages.


Can you clarify if there is any potential downside


to truly comprehensive, every-indexable-page,


XML sitemaps on crawl budget?


Are there cases where we should simply


let page rank internal links drive


the crawling versus asking Google


to process a massive XML sitemap with many extra or low-value


pages?


So I could imagine some situations


where a website is very limited in crawling


and you'd want Google to focus on some primary pages.


But in general, we would do that with sitemap files anyway.


So sitemap file doesn't replace our existing crawling,


but rather it helps us to crawl a little bit more efficiently.


So even for a large website, if we have clear information


about last-notification dates, for example,


then we can focus on those pages and crawl those a little bit


more.


And for really large websites, last-modification dates


can be really important because we


don't have to crawl all of the paths that


lead to that one or two links that go to this page.


We can just recognize, oh, this page we've known about for,


I don't know, five years has changed now.


We should go look at it the next chance we have,


and we don't need to kind of recrawl everything else.


So my recommendation, especially for large sites,


is to use a comprehensive sitemap file


and make sure that you have all of that in the sitemap file


properly, and in particular, that you


use a good last-modification date on those pages


or for the URLs in the sitemap file.


John, on that topic, Big has been doing stuff


around webmasters submitting content


they want them to crawl versus and then even going beyond that


and saying, send us all your HTML.


Have they reached out to Google about potentially doing that


with them that you can't talk about,


that you want to talk about live on this broadcast


so everybody can hear about it?


I'm not aware of anything specific happening


in coordination there.


So we do have the indexing API, but that's, I think,


significantly different from what


Bing is doing at the moment.


And the indexing API is specific to jobs and livestream content.


I believe.


And for that, you would also submit the URLs.


I think every now and then, there


are discussions around what can we


do to make it easier for sites to submit content, especially


large sites that change quickly so that we


don't have to crawl them and go through the HTML.


But in the past, all of these discussions


have ended up with, well, the HTML is actually


pretty useful, and it's useful for us


to make sure that this content is actually on the page itself.


And it requires a lot less unique infrastructure


on the site side to actually index the content.


So that's something where I don't know


if this will never happen.


But at least so far, it's been the case that we've said,


well, we prefer to be able to crawl pages


and to pick up the content from there.


SPEAKER 2


More crawl budget questions.


I'd love to know how you calculate and plan


crawl budgets.


Is Google planning and calculating or just


counting crawl demand and crawl budget on a folder level?


Or I think it's on a domain level or web server level


or any other option.


So I think Gary's blog post from 2017


goes into this fairly detailed.


So I would definitely check that out.


He has written that up fairly comprehensively.


And I believe all of that is still all relevant.


In particular, we try to calculate the load


that we can put on a server on a per-host level,


because that's usually the level where we see


the reactions of a server.


So not on a folder level, but more trying to understand


what the server is.


And sometimes that can include subdomains.


Sometimes subdomains are different.


It kind of depends on what we've discovered over time.


And this is something that adjusts itself automatically


over time.


So it's not that there's any specific setting


that you need to make in Search Console to see this happening.


Essentially, our systems try to understand


that when we crawl like this, your server responds fine.


So probably crawling is OK.


And when we crawl slightly differently,


your server seems to slow down or it returns


a lot of server errors.


So perhaps we should crawl a little bit less like that.


And that's something that happens totally automated.


Sometimes you can see that happening


if you have significant changes in your server infrastructure.


So for example, if you move to a CDN,


then that might be something where suddenly we


can crawl a lot faster.


And you'll see kind of this ramp up happening.


Or similarly, if there's something on your back end


that's slowing things down significantly,


then you'll see kind of that step down


and the crawl rate happening.


And you see all of that in Search Console.


So in the crawl stats reports, you can see that someone,


I forgot the name, just recently made a bookmarklet


to export that into CSV files, which


I thought was really cool.


I don't know if there are plans from the Search Console side


around something similar, but that's


I thought that was really neat.


That's a neat way to kind of track


how things are happening with regards to crawling.


You can, of course, also track this in your server logs


because you see the Googlebot requests.


And you can kind of compile those together


and see how many URLs are being crawled, what kind of URLs,


all of that.


But in general, this is done on a per server level


because that's the level that your server would react to.


Do external links pointing to pages with URL parameters


set to crawl none in Search Console URL parameters


tool hold any value?


So if someone is linking to a page on your site


that you've basically told Search Console not to crawl,


would that work out?


I think it depends as much as nobody likes to hear that.


But the crawl settings tool in Search Console


is a way to give us your preferences with regards


to crawling.


It doesn't block crawling.


So if you want something explicitly blocked,


make sure to use robots.txt because that would really


block crawling.


If you do use robots.txt, we don't


know what is on these pages.


So we don't know if there are any matching canonicals


that we can reuse.


So what can happen if there are links


to a page on your website that is blocked by robots.txt?


We might index that URL without having crawled the content.


And maybe we'll take something like the anchor text


and show that as a title in the search results.


So it's possible that we would show that URL


without any of the content in the search results


if it's blocked by robots.txt.


With the crawl settings tool, with the URL parameter tool,


where you can also specify that you don't


want things to be crawled, usually what would happen here


is we would try to determine what the canonical of that page


would be.


And we would try to forward those links


to the canonical URLs.


So again, we might crawl.


And we might see there's a rel canonical there.


And we might be able to use that directly.


If we end up not crawling a URL and we


think we have a good canonical for that URL,


we'll try to use that.


Sometimes that makes sense if, for example,


there is a session ID parameter.


And we know we can just drop that session ID


and find a valid page.


Then that's something we could do.


If we don't crawl that URL and we


don't know of any canonical on your site that matches that URL,


then that link would go nowhere.


And essentially be discarded.


So on the one hand, you're probably overthinking it


if you're looking at this on a per-link level.


And your site generally collects lots of valid links


across all kinds of pages.


So probably it's not something you need to over, I don't know,


micromanage, essentially.


But one way you could double check


to see what is happening there is to use the Inspect URL tool


and just to see if there's a known canonical for that URL.


And if there is a known canonical,


then that would be where the link goes.


If that URL is not known at all, then that link


would essentially be dropped.


And maybe it makes sense to not use that tool,


the URL parameter is handling tool,


if you're seeing a lot of links going to pages like that.


And instead of using the real canonical directly


so that we can crawl, find a canonical,


forward the link appropriately.


But my suspicion is for most sites,


doing this on a per-link level like this


is just way too much micromanagement.


And you could be spending all of that time focusing on something


much bigger on your website.


OK, complicated question about click-through rate fluctuation.


This is about process timeline and algorithmic penalty


recovery.


Wow, all of the keywords that bloggers


have been waiting for.


So in the performance report of Search Console,


impressions and clicks are directly relatable,


so fluctuates with the average position.


However, those don't alter title tag and description


of a single result. Why does click-through rate fluctuation


correlate and average position fluctuation


drawing the same graphic?


I think this is very site-specific in the sense


that on some sites, you will see clear correlations


between any one of these metrics, which could be like,


I don't know, the click-through rate is pretty stable.


And the clicks and impressions are pretty comparable


with regards to trends.


And on other sites, you might see


completely different results, or it might


be by query or type of page.


So I wouldn't assume that anything here


is generalizable to all websites, all URLs,


all types of queries.


So that's something where I try to look at it


on an individual level.


If 80% of a backlink pool is toxic,


and assuming all toxic links are marked in disavowed at once,


how long does it take Google to get this change into account?


So I don't know if you can say that links are toxic.


Essentially, toxic in this sense would


mean that someone has been building these links over time.


Usually, that's by story from old SEOs,


where someone previously went out and bought a lot of links


or set up this crazy link network,


set up some kind of a link network,


and things kind of you compiled a lot.


You collected a lot of bad links to the site.


So it's not that the links themselves are toxic,


but rather that things that people


have been doing to try to promote the site


are not really in line with the Webmaster guidelines.


And if you recognize a bigger issue like this,


and you don't have a manual action for your website,


then that's something you can use a disavow tool for.


And with the disavow tool, you upload


a list of links or domains.


And essentially, those are taken into account right away.


That means the next time we recrawl any of those links,


we will drop those from our link graph.


And they will not be used with regards to passing signals


for your website.


They'll still be shown in the Search Console reports,


because we show nofollow links there too.


But essentially, they're treated like normal nofollow links.


And that happens immediately when


you upload the disavow file.


And the effect from that takes place over time.


So it's not like you have to stage the disavow file


or do anything special there.


It's really that's taken into account right away.


But the effect you would see over time.


And that sometimes just takes a while to be effective.


If a site has an algorithmic penalty,


and assuming all necessary fixes are done at once,


submitting, how long does it take to recover?


So I think there are multiple aspects here for that one.


On the one hand, algorithmic penalty, people


include all kinds of things in there.


So that's one thing where I'd say it's not


an algorithmic penalty, but rather our algorithms


have determined that your site is different in relevance.


So that usually doesn't mean that there's something


that you need to fix.


But rather that we think that the current status


of your site, this is how we would


find that it's relevant to show in the search results.


And to improve that, you don't fix the issue,


but rather you improve your website overall.


So that's something that is kind of unrelated there.


And the other thing is if you see that algorithms suddenly


start seeing your site differently and thinking


that maybe it's not as relevant as we thought in the past,


that's not necessarily something that you can fix and get


that old status back, but rather we're taking into account


already how we think that it's relevant.


So for example, if your site, I don't know,


let's take something like keyword stuffing,


which is kind of basic.


If we look at your pages and we discover


that there's so much keyword stuffing here


that we can't properly take it into account,


then maybe it would be ranking because of all of this keyword


stuffing.


And when our algorithms notice that keyword stuffing,


we would drop it in ranking.


But if you fix that keyword stuffing,


then that doesn't mean it would go back up again,


because it was previously ranking


in an unnatural state, essentially.


So it's not that you would go back to that previous state,


because that previous state was probably


based on an unnatural situation.


So that's something also to keep in mind there.


In general, when you make bigger changes on your website


to improve the website overall, then we


have to recrawl and reprocess the pages of your website.


Some pages we process a lot faster.


Every couple of days, some pages take a lot longer.


If we need to reprocess the overall site


to get a better understanding of how this whole site stands,


then we need to have a significant part of the site


at least reprocessed.


And sometimes that can take several months.


So that's something where if you see


that your site has dropped because of algorithmic changes


in our search results, and you significantly


improved that website, let's say overnight,


you improve everything significantly,


then I would still expect to see several months'


time before our algorithms have recognized


that this website has significantly improved


and is suddenly significantly more relevant than we


had initially assumed.


I've seen a massive traffic loss on Google Discover


since the 24th.


Have you been rolling out something,


or has something changed?


I don't know.


So I don't know specifically about Google Discover,


but in general, we do make changes


to various of our algorithms over time.


So that's something that might be just a normal change


over time.


I haven't heard much from other people around Discover


with regards to the 24th.


So it doesn't feel like something


that is generally kind of an issue


that people are seeing.


So it's really hard to say there.


Also, with Discover, it's not based


on the queries themselves.


So it's something where sometimes it


can be tricky to figure out what you can do to kind of appear


well in Discover to match kind of the interests


that people generally have when they see sites in Discover.


Since a couple of weeks, we're noticing more and more.


Often, this kind of behavior, new and fresh pages


appear to be correctly indexed in Search Console,


but we can't find them with a site operator.


After a while, usually some hours


they appear in the search results,


but usually too late to get traffic as fresh pages


and news trend is over.


What could be causing it?


I don't know.


So I believe you also asked this on Twitter


and had some sample URLs.


So I've passed those on to the team to double check.


But from what I've seen, this seems


to be something specific maybe to your website.


I also haven't seen other people talk about this much


externally.


So I don't think there's something very generalizable


that is there.


In general, the things I'd recommend


doing with kind of a news website or a website that


is focusing on really fresh content


are things that we've been telling people for a long time.


So it's nothing really unique there.


Working with site maps, if you're in Google News


using Google News site maps, making it as easy as possible


for us to crawl as much as we need from your pages


and to be able to extract the information as quickly


as possible from your pages.


So those are essentially the normal things to do.


So news websites, once something happens,


all the news sites will report the same,


but in their own style.


Most of the content in news websites


looks like a spun of each other.


How does Google or ranking fact work here?


Does the content in the long run,


will it be treated as spun or low quality content?


So I hear this every now and then.


But I think in short, when you're creating news,


when you're writing about events,


everyone writes things in their own way.


And it would be really kind of weird for us


if we see exactly the same content on the same event


in the sense that that's usually not the way that it works.


On the other hand, we do see a lot


of websites that are essentially rewriting news content


from other websites.


And usually that's pretty obvious


when you manually look at these pages


and you kind of search across the web.


Either on the one hand, you kind of read through the content


and you think this is like weird phrasing.


How did this happen?


And if you start searching for it,


then you notice that actually sentence by sentence,


it's the same as something from some other media.


So from my point of view, that's not something


where it's like you're reporting on the same thing,


or you write the same thing.


It's really you're taking one article and rewriting it.


And from my point of view, that is essentially spun content.


So that's something where our algorithms, ideally,


should recognize this and say, well, this content is not


really as valuable.


We need to focus our resources, our crawling time,


more on this other website that actually


produced the original content.


So my recommendation there would be really


to focus on creating your own unique, compelling,


and valuable content and using that primarily.


If you need to supplement this with content from feeds,


something like, I don't know, from AP


or some of the other bigger kind of press agencies,


then that's fine.


But you don't need to rewrite that content.


That's something you can kind of reuse.


But you really need to make sure that the primary, the bulk


of your website's content is something


that is unique and compelling and something


that you've created, not something


that you've kind of either manually or automatically


written off of someone else's website.


And sometimes that means you don't cover exactly


the same thing as everyone else.


Sometimes that means you need to put in a lot more work


to get similar coverage.


But that's essentially what users expect.


They expect to find unique and compelling content


on your site if Google were to send people over to your site.


The rich results data for our website


is not showing in Google Search Console


when our developers removed it accidentally in July.


We have added back structured data


to all pages in the month of August.


The AMP, WebLite, FAQ, and other data


are showing, however rich results data is still not showing.


So not really sure what might be happening there.


In general, I believe we would continue to show that report,


even if most of your pages don't have that structured data


anymore, because we would kind of see the drop in indexed


elements over time.


And that would take place over several months.


My guess is maybe you just don't use the type of structured data


that we would show in that particular rich results report.


So I would double check the type of structured data


that you use and make sure that it matches what


that report would be showing.


If we're seeing the other types of rich results, which


includes FAQ, then I think you're on track.


And there is nothing specific that you


need to do differently unless there is really


one type of rich result that doesn't seem


to be working for your site.


Let's see.


We have a bunch of questions left.


But I assume there were some more things from your side


as well somewhere here in the Hangout.


There's some things also in the chat.


Let me just run through that briefly.


Or if any of you want to get started


with a question of your own, feel free to jump on in.


Wow, no question.


Hey, John, I have a question.


OK.


OK, my traffic has dropped since October 8.


And even across tech with the search console,


the crawl stats, the pages crawl per day, even that has dropped.


So what are the factors you'd have to consider


to improve my crawling?


To improve the crawling.


So for the crawling, there are two main things


that are in play.


On the one hand, there is the crawl demand,


where we try to figure out how much we need to crawl


or we want to crawl from a website.


And that's primarily based on how important


we think your website is, what kind of content


you're providing, how it's doing in general.


And on the other hand, there are things


that limit the amount of crawling


that we would do, which kind of goes


into the general category of crawl budget.


And there is a blog post from 2017 about crawl budget.


I would double check that for all of the details.


The short version is that we try to understand


when a server is overloaded, which


means that the server might be slower


or that the server is returning server errors when


we try to crawl.


And when we see that happening, we


will step back with the crawling.


However, that shouldn't directly affect


how we show the site in search.


So it wouldn't mean that we would rank the site lower


in the search results.


It's just we crawl less.


So we have to be more optimized in our crawling.


And that might mean we focus on more of the new pages


rather than updating the old pages.


And even in those cases, we would


be able to rank pages properly even


if we can't crawl that much.


So more crawling doesn't mean better ranking.


Similarly, less crawling doesn't mean worse ranking.


But if we can't crawl enough of your website


to actually keep up, then that's something


that would be problematic.


So instead of focusing on the crawling,


my recommendation there would be to try


to understand better what is happening with the ranking


side, with the traffic in general.


The usual tips that I have there is


to look at this on a query level.


So to go into Search Console, look


to see which types of queries have less traffic.


And then you can also compare that to which types of pages


have less traffic.


So in search analytics, you can compare, for example,


this month or the last month.


And then you can see the drop in impressions.


You can sort by the impressions that changed the most.


And you can look at the types of pages or the types of queries


where these changes happened.


And based on that, sometimes you can recognize technical issues


where you see, oh, this part of my website


used to get a lot of traffic but now doesn't


get a lot of traffic.


Maybe there is a technical issue.


On the other hand, maybe you can recognize general quality


issues as well, where you see that, well, my website used


to get a lot of traffic for these types of queries,


but actually maybe my website is not


the best result for those types of queries, which could be OK,


where you say, well, I get less overall traffic,


but I get more targeted traffic.


And that could be fine too.


You could also be getting less overall traffic


and less targeted traffic in the sense


that, well, I try to target those queries,


but I'm getting less traffic for those queries.


And that could be a sign that you need to just, I don't know,


just as easy to say, improve the quality of your website


overall.


So that would be kind of the different directions


I would take there.


And it's hard to have one trick that handles all of this.


But I would initially really separate the crawling side


from the ranking side and really try


to understand what kind of pages, what kind of queries


are affected, and then dig in from there


to figure out what exactly you could be doing to improve.


OK.


Hey, John.


Hi.


Yeah, so I have a question regarding manual penalty.


So we got a manual penalty for structured data


a couple of weeks back for the content mismatch.


And we resolve that issue and ask for reconsideration.


But it got failed.


So what other thing I should look


after to resolve this issue?


It's hard to say without knowing your site.


So usually what I would recommend doing there


is posting in the Webmaster Help column,


because the folks there have a lot of experience


with structured data questions.


And they can usually help you to figure out


what you should focus on next.


And the important part there is really


to be as upfront as possible about what


the initial issue was, what kind of pages


perhaps were mentioned in the manual action


and in the response to reconsideration request,


so that someone can really take an honest look at your web


pages and say, well, this is something


that you need to improve.


Or maybe you accidentally forgot something here


to kind of get that honest feedback


that you would need there.


So just a query.


So is there a chance that we have other structured data


errors?


So could it be because of those reason this can be an issue?


I don't know.


It's hard to say without looking at your site.


Theoretically, it could be that there


is something technically broken in the structured data


on your site that makes it so that what we pick up


would be not compliant with our guidelines.


Theoretically, that would be possible.


I think that's extremely rare, though.


So I would really, for the most part,


I would assume that you're marking something up


that according to our guidelines should not


be marked up like that.


And sometimes, that's a matter of interpretation.


And people who have looked into a lot of these cases


can say, oh, in this particular kind of case,


Google would see it like this.


Therefore, you should make a change like this.


OK, and I have one more question.


If you are on here.


So I'm trying to get my website for in the search bar.


Basically, in the search results,


basically, there is something called search bar, which


comes for some branded website, ideally.


So I'm trying to get that search bar for my website.


So I got to know that we have to add


the schema in my home page related to the search bar.


And at the same time, I'm trying to block all the URL parameter


which is coming for the search bar for internal searches.


So will it somehow impact the possibility


that I'll come in the search bar,


search was coming in the Google results?


Yes.


I don't think that would play a role there.


But in general, the site link search box


that you're referring to, that's something


that is generated algorithmically


for the website based on individual queries.


And we would only show that when our algorithms think


that it's a relevant part of the search results page.


So whether or not you have markup there,


or whether or not you have an internal site search page


as well, that wouldn't play a role there.


It's really our algorithms think that we should


go a site link search box.


And in that case, we would either show the one


that we generate, or we would show one from your site.


And if we have one from your site,


then we would be able to show that.


But it's not that because you have the markup,


we will show the site link search box.


So it's also something you could say, well,


I will wait until I see this in person.


And when I see it in person, then I


think about how I want to have it presented.


But it's not something that you can force.


Got it.


Thanks.


All right.


We're kind of running low on time.


But if there's any last question,


feel free to jump on in.


Hey, John, I have a question.


So I posted a comment.


It's regarding a real estate site


where we changed URLs about two years ago, resubmitted them.


And they've been stuck in discovered but not indexed


for over two years now.


I was wondering if you see any reason why that would be.


That's hard to say without being able to take


a look at the site.


Offhand, my guess would be that maybe these


are submitted in a sitemap file but not linked


like that within the website.


Something that I sometimes see.


And in that case, we would know about these URLs.


But if they're not used within the website,


we might think, well, we have other ones that we


can use within the website.


And sometimes with sites like real estate sites,


that can be tricky because sometimes these sites just


have a lookup bar, essentially, where


you enter an address or a city.


And then it goes and tries to pull that information.


And for crawling, we need to be able to find


links that go there.


And we generally wouldn't go to a search field and say,


oh, let me try all the cities in the world


to see what comes out.


We actually have an extensive sitemap submitted and also


on-site links for all the pages.


Before we changed URLs, everything


was working beautifully.


After we introduced these URLs, which were probably


about 800,000 new URLs, we thought


it would take Google a few months to recrawl them


and detect that.


But after two years, it still hasn't picked up those changes.


OK, so what I would do is find the higher level pages that


are still being indexed and use the Inspect URL tool


to check the HTML that comes out from those pages


so that you can confirm that Google has indexed links


to those pages.


Because if we know that the links are there,


we can kind of flow the signals through those URLs.


And we can start crawling and indexing


those a little bit more.


That's kind of, I'd say, the first step there.


The second step would also be to double-check


that none of these pages are indexed or all of them


are kind of blocked like this.


Probably about 10% of the site.


So some of them are being indexed in that case?


Correct.


OK.


That makes it more complicated.


But that's why it's been very confusing,


because it's a real estate site.


So we publish new pages every day.


And 10% get indexed.


The rest don't.


And there's nothing different on them.


Some example URLs of ones that you can use.


SPEAKER 2


And see if I can find something specific there


that I can point you at.


SPEAKER 3


All right, since we're kind of out of time,


let's take a break here.


There are still more questions submitted,


which is a good sign.


I have another English Hangout lined up for Friday.


Next week, I'll be at the Webmaster Conference


in Mountain View.


If any of you are there, come and say hi.


And I also have a German Hangout lined up on Thursday.


And we'll have more English Hangouts, I guess,


when I'm back from there as well.


All right.


So wish you all a great week.


And thank you all for joining in.


Thank you, John.


Bye.


Google Analytics for site ownership verification - Google Search Console Training

In this video, I'll go through the steps you need to follow


to verify your website in Search Console


using the Google Analytics tracking code.


Google Analytics tracking code


In order to verify your website


using the Google Analytics tracking code,


you'll need an account with Google Analytics


that use the same Google account.


You must have at least added permission


on the Google Analytics account.


If you're not sure which permission you have,


check the Google Analytics Help Center.


Before verifying, ensure that your homepage


has the Google Analytics tracking code


in the head section of the page.


Although your Analytics tracking code


might work for Analytics in the body section as well,


for Search Console verification,


it must be in the head section.


Open your homepage in a browser,


look at the page source code,


and confirm that the tag is present


in the head tag of the page.


After you click Add Property,


add your website URL under the URL prefix option


and click Continue.


You must be logged into Search Console


with the same Google account that you use for Google Analytics.


If you perform the steps above correctly,


verification should work immediately.


If it doesn't, click the Full Details link


to learn about potential errors.


You can find out more about those errors


and how to handle them in the Help Center.


The Google Analytics tracking code


will be used only to verify site ownership.


No Google Analytics data will be accessed following this process.


If you're looking to configure Search Console data in Analytics,


that's an entirely separate process.


Check the link below to learn more.


Thanks for watching,


and check the Google Webmasters YouTube channel


for more videos on Search Console.


How to change my business address & information on Google? | Search for Beginners Ep 11

Let's talk about how you can change your Google Maps information in search.


If you own a store or business with a physical address where you see customers, you can claim


the listing on Google My Business.


Google My Business is a free tool that helps you manage how your local business appears


on Google Maps or search.


If your business serves customers at a particular location or within a designated service area,


Google My Business helps people find you.


Google My Business is useful for maintaining accurate information about your business in


Google search, for interacting with your customers, and for attracting new customers.


Verified businesses on Google My Business are twice as likely to be considered reputable


by users.


It's very important that you keep your business information up to date so that customers can


find you.


Sign in to your Google My Business account to edit your business's details.


You will only have editing access if you have first verified your listing in Google My Business.


The details you can edit on your Google Business are your business name, your business category,


your hours of operation, your phone numbers, your website URL, photos for your business,


your opening date, and the address.


And that's it!


This video concludes the Search for Beginners series.


However, there are many more videos coming up to the Google Webmasters YouTube channel.


So if you're not subscribed already, consider clicking the subscribe button and tapping


the little bell to receive notifications if you are watching on mobile.


DNS record for site ownership verification - Google Search Console Training

In this video, I'll go through the steps you need to follow


to verify your website in Search Console using the DNS record method.


DNS Verification


DNS verification requires you to add a DNS record in your domain name provider.


This is the only method supported for domain properties.


So I'll demonstrate DNS record verification by creating a domain property.


But know that you can also use DNS record verification on URL prefix properties.


Add your domain to the domain option as shown in the screen and click Continue.


In the instructions dialog, you'll see two options.


If your domain register is listed in the instructions dropdown,


you'll be walked through an easier automated process to verify your site.


For example, if your DNS provider is GoDaddy,


choose it from the list and click Start Verification.


This will lead you to their login page where you'll need the credentials used in GoDaddy.


After logging in, click Authorize and you'll be redirected to Search Console.


But don't worry if the verification doesn't work immediately.


Just wait a few hours or a day and try verifying again by choosing it from the properties list.


Sometimes DNS record changes can be slow.


If your domain name provider is not in the list,


you'll need to follow written instructions in the Help Center


to copy your DNS text record into your domain provider.


Start by copying the text record from the dialog after entering your domain.


Then click Full Details to open up the Help Center.


You'll see the full process documented there.


We have instructions for a lot of domain providers.


Find yours in the list and click the link for more information.


Now follow the instructions for your specific provider.


If you're asked to provide a Google Security Token,


that's the token you copied from Search Console's verification dialog a minute ago.


After you change your DNS record, go back to Search Console


and click Verify in the Verification pane.


If the verification doesn't work immediately,


wait a few hours or a day and try again


as the change may take some time to take effect.


Just choose the unverified property from the properties dropdown


and it will try to verify you automatically.


To stay verified, don't remove the DNS record


even after verification succeeds.


If you perform the steps above and your verification does not succeed,


check the Full Details link mentioned previously


to learn about potential errors.


You can find out more about those errors and how to handle them


in the Help Center.


Thank you for watching and see you on YouTube.


Traffic fluctuation (and ensuring healthy navigation) | Sustainable Monetized Websites

Welcome back to the series Sustainable Monetized Websites.


I'm Aurora and I work in policy education for publishers at Google.


Oh, and these two are my co-hosts, Organica and Monetized.


In this episode, we will cover the reasons behind traffic changes


and we will share best practices to help ensure good traffic on your site.


So I'm here to learn ways in which my monetization strategy with Google AdSense


can help my organic performance on search.


I'm glad you're bringing that up. That's actually a common misconception.


Monetization does not affect how a site ranks in organic Google search results.


You can check more on how search works in the resources linked in the description.


Let me start by saying, traffic should result from genuine user interest.


Any other method to manipulate it could lead to demonetization of your content


and manual actions on organic search.


Google AdSense calls this manipulation invalid traffic.


In a nutshell, it's any interaction that doesn't come from real people


with real interest in an ad or your site.


Some invalid traffic is accidental, like when a badly placed ad


gets in the way of something the user is trying to click on.


Some is deliberate, like botnets or people who maliciously click on ads.


Invalid traffic includes, but is not limited to,


publishers generating clicks or impressions on their own live ads,


publishers encouraging clicks on their ads by explicitly asking users to click on ads


or by implementing ads in a way that may cause a high volume of accidental clicks.


Automated clicking tools or traffic sources, robots or other deceptive software.


Clicks on ads must come from genuine user interest.


High levels of invalid traffic on your account can lead to suspension


of your monetization through Google AdSense.


Advertiser budgets are refunded for invalid clicks,


and this will impact your finalized earnings.


And how does Google identify invalid traffic?


Through a combination of automated systems as well as human reviews.


Our ad traffic quality team is dedicated to stopping all types of invalid traffic


so that advertisers don't have to pay for it


and the people who cause it don't profit from it.


What if I am a victim of sabotage?


We understand that a third party may generate invalid traffic on your ads


without your knowledge or permission.


Ultimately, it is your responsibility as the publisher


to ensure that the traffic on your ads is valid.


If you think you have invalid traffic from a third party,


use this form linked also in the description to report it to our traffic quality team.


On the organic content side, your site's traffic might fluctuate


due to a series of factors that could affect the ranking, impressions, and clicks on your site.


Before you get much further,


make sure that you have your site's verified in Search Console.


Do this as early as possible so that you have access to the tools and the data when you need it.


Don't focus too much on your site's absolute position or even on small fluctuations.


However, if you see a dramatic or persistent drop in position,


you can start by checking the Search Console performance reports.


It will help you understand what numbers have changed for your site


and whether the drop is related to a specific category, such as query, country, or device.


Some of the causes for a website's traffic fluctuation could be due to


users seeing your site but not clicking through.


Maybe other search results are more compelling, reliable, updated, or authoritative.


Perform a search for some queries to see which sites are doing better than yours and why.


You can read about how to improve your titles and snippets


on the resources linked in the description.


Google is having problems finding or reading your site.


If you recently made bigger changes to your site, like moving to another domain name,


you moved existing pages to new URLs on the same site, or you moved from HTTP to HTTPS,


give us a few weeks to update our index.


You can follow the progress in the index coverage report in Search Console.


You used to qualify for search appearance features, but do no longer do so.


Examine your rich results and AMP reports in Search Console for new errors.


Your site's mobile usability has decreased.


Check the Search Console mobile-friendly test tool for this.


There was a change to our ranking or reporting algorithms.


Visit the Search Central blog and our Data Anomalies page.


You have a manual action or security issue.


Open the manual actions and the security issues reports in Search Console to fix them.


A periodic drop.


Is there a weekly, monthly, or yearly pattern in your traffic drop?


Use Google Trends to understand how this pattern relates to your site's content.


For example, if your site is about swimsuits,


you might see a drop in search traffic around winter.


These periodic fluctuations are hard to avoid,


but you could try to create content that's relevant all year round.


Now that you've heard about what can cause these issues,


here are some best practices you can follow to prevent or minimize


invalid traffic penalties and organic traffic fluctuations in your monetized site.


On the organic side, make pages primarily for your users, not for search engines.


Which brings me to, do not trick search engines.


Ask yourself, am I doing this for my users or to improve my search engine rankings?


Make your website stand out from others in your field by making it unique, valuable, and engaging.


Try to keep it accurate, useful, and up to date.


Check regularly the Search Console tools and reports available to you,


such as the index coverage and performance reports.


These will help you understand how your site is doing in search.


Review the SEO starter guide and keep up with changes on the Search Central blog.


And on the monetized side, understand your ad traffic and site visitors.


Monitor proactively how your traffic sources are performing


so that you can make educated decisions.


Avoid partnering with untrusted or low-quality parties that send bad traffic to your site.


Don't click on your own ads, even if you think it's okay to do so.


We may disable an account if it looks like a publisher


has been clicking on their own ads to inflate their earnings.


Double check your ad implementations.


Make sure they follow our ad placement policies and have no programming errors.


Also, check your ads in different browsers and platforms to ensure they work as intended.


Ask your questions on the AdSense help forum.


Most likely, another publisher encountered the same issue as you.


We hope this video helped you understand the reasons behind traffic fluctuation,


both for your organic website and your monetized content.


We wish you a healthy navigation.


In the next episode, we'll talk about duplicate content and what to do with it.


Subscribe to this channel so that you don't miss anything.


Do not forget to like and share this video,


and leave a comment if you have any good topic suggestions for future episodes.


Are URLS case sensitive? #AskGooglebot

Today, we have a question from Hamza on Twitter, who asks,


Are URLS case sensitive? #AskGooglebot

does upper or lowercase in URLs matter?


Are URLS case sensitive? #AskGooglebot

Well, thank you for asking.


Are URLS case sensitive? #AskGooglebot

URLs are the basis of all websites,


Are URLS case sensitive? #AskGooglebot

so it's good to double check your assumptions.


Are URLS case sensitive? #AskGooglebot

By definition, URLs are case sensitive,


Are URLS case sensitive? #AskGooglebot

and also things like slashes at the end do matter.


Are URLS case sensitive? #AskGooglebot

So technically, yes, these things matter.


Are URLS case sensitive? #AskGooglebot

They make URLs different.


Are URLS case sensitive? #AskGooglebot

If a website still shows the same content in these cases,


Are URLS case sensitive? #AskGooglebot

search engines will try to figure it out on their own.


Are URLS case sensitive? #AskGooglebot

And usually, that works out well, but it's not always ideal.


Are URLS case sensitive? #AskGooglebot

For example, search engines will try


Are URLS case sensitive? #AskGooglebot

to crawl all variations of a URL that they find.


Are URLS case sensitive? #AskGooglebot

This can make it a bit slower for them


Are URLS case sensitive? #AskGooglebot

to find other useful content on your website.


Are URLS case sensitive? #AskGooglebot

Also, when search engines find multiple distinct URLs showing


Are URLS case sensitive? #AskGooglebot

the same content, they have to decide which of these URLs


Are URLS case sensitive? #AskGooglebot

to keep.


Are URLS case sensitive? #AskGooglebot

We call this canonicalization.


Are URLS case sensitive? #AskGooglebot

It doesn't change ranking, but our systems


Are URLS case sensitive? #AskGooglebot

might choose a URL that you wouldn't have chosen.


Are URLS case sensitive? #AskGooglebot

Another place where the exact URL plays a role


Are URLS case sensitive? #AskGooglebot

is robots.txt.


Are URLS case sensitive? #AskGooglebot

In the robots.txt file, you can signal


Are URLS case sensitive? #AskGooglebot

which parts of a website shouldn't be crawled.


Are URLS case sensitive? #AskGooglebot

The robots.txt file also uses exact URLs.


Are URLS case sensitive? #AskGooglebot

So if you have entries there which


Are URLS case sensitive? #AskGooglebot

refer to one version of a URL, they


Are URLS case sensitive? #AskGooglebot

would not apply to other versions of that URL.


Are URLS case sensitive? #AskGooglebot

It's rare that we see this cause problems, though.


Are URLS case sensitive? #AskGooglebot

Using internal linking to link to a consistent version


Are URLS case sensitive? #AskGooglebot

makes your preference clear.


Are URLS case sensitive? #AskGooglebot

Adding a link rel canonical element


Are URLS case sensitive? #AskGooglebot

also helps to confirm that and encourages search engines


Are URLS case sensitive? #AskGooglebot

to focus on that version.


Are URLS case sensitive? #AskGooglebot

So in short, upper or lowercase does matter for URLs.


Are URLS case sensitive? #AskGooglebot

It's a good practice to be consistent in how you use them,


Are URLS case sensitive? #AskGooglebot

but it's usually not that critical for a website.


Are URLS case sensitive? #AskGooglebot

I hope this helps to answer this question.


Are URLS case sensitive? #AskGooglebot

See you in the next episode of Ask Googlebot


Are URLS case sensitive? #AskGooglebot

for more questions and more answers.


Are URLS case sensitive? #AskGooglebot


English Google SEO office-hours from September 2022

Hello, hello, and welcome to a very special edition of SEO Office Hours. My name is Lizzie,


and I'm here to answer some questions with you today.


Hi everyone, John Mueller here. We're all on the Google Search Relations team


based here in Switzerland. Like Lizzie mentioned, we're doing this Office Hours session a bit


differently. The goal of this format is to make it a bit easier for everyone to participate,


including on our side. We'd like to experiment with the formats a little bit,


so we may try variations over time. Also, that means that the processing here is kept to a


minimum, so at least in the beginning it won't be as polished as the other content on this channel.


We'll work on that once things have settled down a little bit more.


And now, let's continue with your questions. Where is it? Where is it? Where is it? Where is it?


It's here. Hold on a second, Marcin. You're not on mute. First up is Lizzie.


So the first question that we've got here is from Demo, a site that connects seekers and


providers of household-related services. Matching of listings is based on zip code,


but our users come from all over Germany. The best value for users is to get local matches.


Is there some kind of schema markup code to tell Google algorithm,


show my site also in the local pack? Please note, we do not have local businesses and cannot utilize


the local business markup code. The site is, and I'm going to redact that. Yes, so, Demo.


As you noted, local business markup is for businesses with physical locations,


and that means that there's typically one physical location in the world for that place.


So it should only show up for that city. And your other question, currently there's no


rich result feature for online services only that you could use structured data for, but you can


claim your business profile with Google Business Profile Manager and specify a service area there.


So I think that that could help. And now's your chance, Martin. Go for it.


Bindra asks, Google Search Console shows excellent Core Web Vitals scores for our site.


But I understand that it only shows details of Chrome users. The majority of our users


browse using Safari. Is there a way to measure package experience or the Core Web Vitals


on Safari browsers? And is there a way to improve the experience? Well, you can't really use Google


Search Console for this, but you can definitely measure these things yourself with the browser


developer tools in a Safari browser, and maybe ask around if you have any data from Safari users


through analytics, for instance. There's nothing here that we can do this for the page experience


or Search Console's page experience resource because the data is just not available.


For the next question, I'll paraphrase. How can I best switch from one domain to a new one?


Should I clone all the content or just use 80% of the content? What is the fastest way to tell


Google that they're both my sites? We call this process a site migration. It's fairly


well documented, so I would look up the details in our documentation. To simplify and leave out


a lot of details, ideally you'd move the whole website one-to-one to the new domain name and use


permanent 301 redirects from the old domain to the new one. This is the easiest for our systems to


process. We can transfer everything directly. If you do other things like removing content,


changing the page URLs, restructuring, or using a different design on the new domain, that all


adds complexity and generally makes the process a little bit slower. That said, with redirects,


users will reach your new site regardless of whether they use the old domain or the new one.


And our next question is from IndieQuest1. Do you support the use and the full range of schema.org


entities when trying to understand the content of a page outside of use cases such as rich snippets?


Can you talk about any limitations that might exist that might be relevant for developers


looking to make a deeper use of the standard? So to answer your question, no. Google does not


support all of the schema.org entities that are available on schema.org. We have the search


gallery, which provides a full list of what we do support for rich snippets like you mentioned in


Google search results, but not all of those things are visual. We do talk about certain properties


that might be more metadata-like that aren't necessarily visible as a rich result, and that


still helps Google to understand things like authors or other metadata information about a page.


So we are leveraging that kind of thing. And next up we have Gary. Hey Gary, which questions did you


pick out? Anton Littow is asking, in Search Console, I get the message sitemap could not be read


in the sitemap report. No other information is provided. What could be the reason that the


sitemap cannot be read by the Google bot? Good question. The sitemap could not be read message


in Search Console may be caused by a number of issues, some of them technical, some of them


related to content quality of the site itself. Rarely it may also be related to the hosting


service, specifically if you are hosting on a free domain or subdomain of your hoster,


and the hoster is overrun by spam sites that may also cause issues with fetching sitemaps.


Our next question is from Nicholas. Hi John. Well, I'm not John, but anyway. Hi Nicholas.


We would like to know how algorithms treat cartoon illustrations. We've got guides and


tips that are illustrated on our website and they're not performing well in the SERP. We tried


to be unique using some types of illustrations and persona to make our readers happy. Do you think


we did not do it right? I don't know because I don't think I've ever seen your cartoons,


but I can speak to the how to improve your cartoon illustrations in SERP. So our recommendation would


be to add text to the page to introduce the cartoons, plus alt text for each of the images


Think about what people will be searching for in Google Images to find your content


and use those kinds of descriptive words versus just saying the title of your cartoon.


Hope that helps. Kibus or Lawrence is asking, does posting one content daily increase rankings?


No, posting daily or at any specific frequency for that matter doesn't help with ranking better


in Google search results. However, the more pages you have in the Google index, the more your


content may show up in search results. Okay, and the next question is from Suresh. About the helpful


content update that only 10% write quality content and the rest 90% don't write quality content,


lengthy content. But how should they write quality content? Does Google agree with the word count or


not? Well, nope. Content can still be helpful whether it's short or long. It just depends on


the context and what that person is looking for. It doesn't matter how many words if it's 500,


1000. If it's answering the user intent, then it's fine. It can be helpful. These are not


synonymous things. I'll paraphrase the next question, hopefully correctly. In short,


when using words from a page title in the URL, should I include stopper words too? For example,


should I call a page, why is the sky blue dot HTML or why sky blue dot HTML? Well, thanks for asking.


Words in URLs only play a tiny role for Google search. I would recommend not overthinking it.


Use the URLs that can last over time, avoid changing them too often, and try to make them


useful for users. Whether you include stop words in them or not, or decide to use numeric IDs,


that's totally up to you. Sanjay Sanwal is asking, does different bots type, image and desktop share


crawl budget? And what about different hosts? Fantastic question. The short answer is yes,


Google bots and its friends share a single crawl budget. What this means to your site is that if


you have lots of images, for example, Google bot images may use up some of the crawl budget that


otherwise could have been used by Google bot. In reality, this is not a concern for the vast


majority of the sites. So unless you have millions of pages and images or video, I wouldn't worry


about it. It's worth noting that crawl budget is per host. So for example, if you have subdomain


.example.com and you have other subdomain.example.com, they have different crawl budgets.


Christopher asks, we've sold the German subdirectory of our website to another company.


They request us to 301 redirect the subdirectory to their new German site. Would you advise against


it? Would it hurt us? Well, on the one hand, it all feels kind of weird to sell just one


language version of a website to someone else. On the other hand, why not? I don't see any problems


from redirecting from there to a different website. The only thing I would watch out for,


for security reasons, is that you avoid creating so-called open redirects, where any URL from there


is redirected to an unknown third party. Otherwise, that sounds fine.


Sam Gooch is asking, I'm experimenting with the new learning video Rich Result and can see it's


being picked up in Google Search Console. Can I expect to see clicks and impressions from this in


the search appearance filter as we can see with some other rich results? Well, to answer this


question specifically, there's no guaranteed time that you'll be able to see a specific rich result


in Google Search after adding structured data. But I think what you're asking about here is for


a specific thing to be added to Search Console, and we'll have to check with the team on the


timeline for that, and we don't pre-announce when certain things will be added to Search Console.


But you can check the Rich Result status report for learning video and make sure that you're


adding all of the right properties and that it's valid and ready to go for Google to understand


what it needs in order to generate a rich result. Hope that helps. Roberto asks, we're planning to


share the same backend and frontend for our two brands. We're ranking quite well with both of


them in Google. How big is the risk of penalization if we use the same HTML structure, same components,


layout, and same look and feel between the different brands? What would be different


are the logos, fonts, and colors? Or would you suggest migrating to the same frontend


but keeping the different experience between the two brands? Well, this is a great question.


Thanks for submitting it. First off, there's no penalty or web spam manual action for having two


almost identical websites. That said, if the URLs and the page content is the same across


these two websites, then what can happen for identical pages is that our systems may pick


one of the pages as a canonical page. This means we would focus our crawling, indexing, and ranking


on that canonical page. For pages that aren't identical, we'd generally index both of them.


For example, if you have the same document on both websites, we'd pick one and only show that


one in search. In practice, that's often fine. If you need both pages to be shown in search,


just make sure they're significantly different, not just with a modified logo or color scheme.


Anna Giacinto asks, JavaScript SEO, what to avoid along with JavaScript links? Well, the thing with


links is that you want to have a proper link. So avoid anything that isn't a proper link. What is


a proper link? Most importantly, it's an A HTML tag that has an href that lists the URL that is


resolvable, so not like a JavaScript colon URL. And that's pretty much it. If you want to learn


more on JavaScript specific things for search, you can go to the JavaScript beginner's guide


on developers.google.com slash search and see all the things that you might want to look out for.


Our next question is from Sakshi Singh. Let's say I research on a keyword which has no volume


or keyword density, but we are appearing for those keywords on the first page. Should we target that


keyword? Well, Sakshi, you can optimize for whatever keywords you want. And it's not always


about the keywords that have the most volume. I would think about how people should find your page


and target those keywords. Kim Ona-Sile is asking, hello, you previously advised that there are no


SEO benefits to audio versions of text content, and that audio specific content doesn't rank


separately like video content. However, given you also said it might be that there are indirect


effects like if users find this page more useful and they recommend it more, they might be more


useful and they recommend it more, that's something that could have an effect. Will audio content be


given more priority and independent ranking following the helpful content algorithm update?


This is an interesting question and ignoring the helpful content algorithm update part,


no audio content on its own doesn't play a role in ranking of text results. Our next question is from


John Mu on Twitter. He's asking, what is the answer to life really? Well, and that one's easy,


John. It's the number 42. I thought you already knew that. Someone asked, is it okay to fetch


meta-contents through JavaScript? I think that means is it okay to update meta tag data


with JavaScript. While that is possible to do, it is best to not do that. It may give Google search


mixed signals and some features may not pick up the changes, like some specific search result type


might not work the way you expect it, or it might have incorrect information, or it might miss


something. So I would suggest not to do that. Anonymous is asking, both of my websites have


been hit by different updates, around 90% drops, and are suffering from some type of flag that is


suppressing our sites until the soft penalty is lifted? Or is there even a soft penalty?


Good question. No, the named updates that we publish on the rankings updates page on


Search Central are not penalties in any shape or form. They are adjustments to our ranking


algorithms so they surface even higher quality and more relevant results to search users.


If your site has dropped in rankings after an update, follow our general guidelines for content,


take a look at how you could improve your site as a whole, both from content and user experience


perspective, and you may be able to increase your rankings again. Aion asks, when would be the next


possible update for the search results? Well, on our How Search Works site we mentioned that


we did over 4,000 updates in 2021. That's a lot of updates. Personally, I think it's critical to


keep working on things that a lot of people use. Our users and your users expect to find things


that they consider to be useful and relevant, and what that means can change over time. Many of


these changes tend to be smaller and are not announced. The bigger ones, and especially the


ones which you as a site owner can work on, are announced and listed in our documentation. So,


in short, expect us to keep working on our systems just like you, hopefully, keep working on yours.


And our next question is from Darius. Hi, Darius. So Darius is asking, does having


star-aggregated ranking on recipes improve its position? I think what Darius is asking about


is the stars that show up for recipes and with structured data and whether or not that has an


effect on ranking. So, while the stars are more visual and eye-catching, structured data in and


of itself is not a ranking signal, and it isn't guaranteed that these rich results will show up


all the time. The Google algorithm looks at many things when it's creating what it thinks is the


best search experience for someone, and that can depend on a lot of things like the location,


language, and device type. Christian asks, I have set the rel canonical together with a no-index


meta tag. When Google does not accept the canonical at all, all internal links are dropped.


When I don't set a rel canonical, then I can see the internal links in Search Console in the links


report. Is this normal? Well, this is a complex question since it mixes somewhat


unrelated things. A no-index says to drop everything, and the rel canonical hints that


everything should be forwarded. So, what does using both mean? Well, it's essentially undefined.


Our systems will try to make do the best they can in a conflicting case like this,


but a specific outcome is not guaranteed. If that's fine with you, for example, if you need


to use this setup for other search engines, then that's fine with us too. If you want something


specific to happen, then be as clear as possible for all search engines. And our next question is


from Thijs. Not sure if I'm saying that right. Thijs. If a video is indexed in the video indexing


report, is it still worth adding the video structure data on that page, and why? Well,


yes. Just because something's indexed doesn't mean that there's not opportunity to improve how it


appears. Structured data helps Google understand more about your video, like what it's about,


the title, interaction statistics, that kind of stuff. And adding structured data can make


your videos eligible for other video features like key moments. So, it's not just, oh, get your


video indexed, and that's it. There are other things that you can do to improve how your content


appears on Google. Tamash asks, can I cloak a list with lots of products to Googlebot and show users


a load more button? I think this is not cloaking as what users see when they click on the search


result roughly matches what Googlebot sees. And if you have a load more button, users will click


that if they don't see the product they're expecting there. So, I don't think this is cloaking.


And that's a solution that I think works from a crawling point of view.


And that was it for this episode. I hope you found the questions and answers useful.


If there's anything you submitted which didn't get covered here, I'd recommend posting in the


Search Central help community. There are lots of passionate experts active there who can often


help you to narrow things down. Let us know on Twitter how you find this. We hope to line up


more of these for the future. In the meantime, may your site's traffic go up and your crawl


Does a CMS matter for ranking in Google Search? #AskGooglebot

And now for a question from Vishal, who asks whether the choice of CMS matters for good


ranking.


So, first of all, as far as I know, our search systems don't look for any particular content


management system to treat it differently.


For us, a CMS is just one way of creating web pages.


Google systems don't focus on how the page was created.


Instead, they focus on the final result.


And created web pages can be just as good as those generated by WordPress, Wix, Squarespace,


or similar.


As far as I can tell, all mainstream CMS systems can create pages that work well in search.


For the most part, SEO is not magic.


It's well-documented, and there are many testing tools so all providers can include SEO elements


if they choose to do so.


There are slight differences across systems, just like all web pages are slightly different.


But for the average site owner, they either all work well for search out of the box or


can easily be tuned to do so with a few settings or plugins.


So there you go, Vishal.


I hope that helped to clarify things a bit.


Catch you next time here on Ask Googlebot.


And if you're after more search, check out our Search Off the Record podcast, where we


go behind the scenes at Google, talk to some incredible SEOs, and have some fun along the


way.


And join me, John, and my fellow teammates from Google Search Relations.


Improving Search Over the Years (WMConf MTV '19)

So I'm going to actually talk a bit about how


we improve things in search.


I'm going to go through a bunch of case studies.


And I actually put together a few.


And I only realized after I put them together


that all are in one way or another about language.


And I think that really gets to how


we think in search about things, which


is that language is really important to understanding


what we do.


So how do we think about ranking problems?


I could try to put together a step-by-step methodology


of what we do.


But it would be a lie because there isn't one.


We have lots and lots of different ways


we approach the problems we think about.


And there's a lot of debugging and experimentation,


guesswork about what might work, evaluation to see if it does,


research both into what we know and what


we can learn from academics and other industry publications,


and then just a lot of luck and maybe stumbling


upon something that works.


And hopefully, you'll see a bunch of examples of that


throughout this.