And there's kind of the theoretical approach, where


English Google Webmaster Central office-hours from March 17, 2020

you just run a series of tests, and then


there's the field data where we look at what people are


actually seeing.


And all of these things are kind of mixed together.


And they always bring kind of a blurry number.


So it's usually, if you're just tweaking milliseconds,


then you're kind of still within this range of blurriness,


where you're not really seeing any big changes.


Gotcha.


Thank you very much.


Sure.


Let's see.


I think there is some stuff in the chat.


Is there an alternative to crawl this URL and its direct links


in the new search console?


No.


Not that I'm aware of.


It's really essentially inspect URL,


and then submit that URL for indexing.


There's no mechanism to say kind of like submit


all URLs that are linked from here to indexing.


What usually I'd recommend doing here


is just using sitemap files to kind of do


all of this in a more programmatic way,


rather than to manually go through and submit things.


Google is planning to eliminate third party cookies entirely


in Chrome browser by 2022.


For the sake of more private web,


what actions should be taken about it in general?


Well, I think if they're removed in Chrome,


then probably you want to make sure


that your sites don't break because of that.


So I'd say the primary effect thing that I would watch out


for is that your site still works well for users.


When it comes to SEO, this is usually less of an issue.


However, we do use Chrome to render pages for indexing.


So if your site is significantly different with regards


to cookies, then that's something


where from an indexing point of view, if after rendering,


we can't get all of your content,


then we'll index the content that we do have.


So that's something where if you're somehow


reliant on these third party cookies


to create a page that contains all


of the content for your site, then that's


something where you need to watch out for.


The other thing, I guess, with regards to cookies


is Googlebot doesn't keep cookies.


So if you give cookies to Googlebot,


then those cookies might be used for that one time rendering.


But they won't be used for the next crawl or the next run


when Google reprocesses your pages again.


So sometimes people trip over that


and they expect that Googlebot keeps reusing the cookies


that it has.


And we essentially try to crawl and render pages


with a completely fresh slate.


Let's see.


Any German e-commerce?


My domains are declining in coronavirus season.


OK.


I don't know what I can say there.


I imagine that depends a lot on the type of your site.


I don't know how it is in Germany at the moment


here in Switzerland.


We essentially closed all stores yesterday,


except for grocery stores and maybe pharmacies, I think.


So if people want products, they end up going online.


So I would imagine that a lot of these things


will be picking up and more people going online


to buy stuff because they can't go and buy things in person.


That said, this is kind of a unique situation.


And who knows how it will develop?


So I'm not going to make any kind of prognosis


on how that happens.


What are all the places discovered content can appear?


We're aware of Chrome article suggestions and Discover


feed and the Google app.


Are there other sources discovered content


can be seen by users?


I don't know of all of these sources.


I think there are three main sources.


But apart from the two that you mentioned,


I can't think of the third one.


I think it's kind of the Chrome new tab, the Discover feed,


and then something else.


But I can't think of it at the moment.


I believe we have a Help Center article that has this a little


bit more in detail.


We're a new site, and we lost all of our traffic


over years because we appear from top stories in Discover.


What can we do before we close?


I don't know.


That's, I think, a tough situation


to be in all the time.


With regards to both top stories and Discover,


these are algorithmic features.


So we use a number of factors to determine when we would show


sites or when we wouldn't.


And these kind of algorithmic changes can change over time.


So it's really hard to say what exactly you


should be doing there.


It's definitely not the case that there


is one meta tag that will automatically


make your site appear or disappear in these sections.


The thing I would generally try to focus on is quality.


I think I've seen your site before,


and it seems like a pretty reasonable site.


But it's still something where our algorithms focus


a lot on quality.


And especially for elements like the top stories,


where we try to show sites that have recent content


for specific topics, or Discover,


where we're trying to highlight content


that matches what we think users might be interested


in at the moment, that's something where quality


is very important for us.


I think Discover is a bit unique because it's


something where there's no query associated with it.


So it's really kind of tricky for us


to find the right results that we can show there.


All right.


Any more questions for many of you?


Nothing at the moment.


OK.


That's good, too.


Hi, John.


Hi.


Yeah, I have this very strange situation


with a client of mine.


They've had a website for, I believe, more or less five


years.


And then the last, I would say, within the last 12 months,


we've lost approximately 80% to 85%


of the organic traffic, which is very strange.


We repealed the site a year ago.


And then the first three months, everything was going great,


getting actually even more traffic.


And the rankings were going up.


And then at some point, everything


started to decline to the point, like I said,


we've lost approximately 80% of the traffic.


The technical side of the site, everything


seems to be working fine.


We don't find any errors.


We've maintained the same backlinks.


The speed of the website seems to be OK.


We don't see any slow pages.


It's mobile friendly.


But there's nothing that we can see that's not working.


So we don't really know.


There's only one thing.


There was another website, I believe


another version of the website, who someone accidentally


uploaded.


And it got indexed on a different domain


with the exact same content.


And that seems to be at the same time when


we saw the ranking drops.


And so what they did was they removed everything.


They took everything down from that domain.


But we never recovered the ranking.


So I'm not really sure what to do.


OK, so usually what would happen if we see the same content


being hosted again on a website is


we would try to canonicalize that content, which essentially


means our systems would say, well, we have multiple URLs


for this piece of content.


Which one is the right one to show?


And usually, we'd be able to keep the existing URLs


and say, well, this is where it's always been.


And this is what we're going to show in search.


It can happen that we switch to another domain.


That can, for example, happen if you do a domain move.


And for whatever reason, you can't set up redirects.


And we'll try to figure that out and switch those URLs over.


However, the switching of the URLs for canonicalization


is purely a technical matter on our side.


It's something where we're essentially


just changing the URL that we would


show in the search results.


So if it's the same content, we switch the URLs,


it would rank exactly the same way as before,


just with the different URLs.


If you switch the URLs back, if we


see that things should go back to the previous URL,


then that's just like a technical change


of those URLs.


We'll switch back over to the other version.


And that's not something that would have lingering effect.


It's really just, well, this URL is the right one.


Oh, wait, no, the other one is the right one.


And from that point of view, that kind of copy of the site


that you accidentally had live, I


don't see that as having any lingering effects there.


If you're worried about that, if that's something that just


recently happened, the one thing I would try to do


is make it so that the canonicalization is


as obvious as possible.


So set up redirects from the old domain to the new one


so that we can really tell if we were to crawl those old URLs


where you had a copy, then you clearly


tell us, actually, this is the one that


should be used by Google.


If you just delete the old version,


then that's a bit trickier.


But I


That's the thing.


The thing is, sorry to interrupt.


The thing is the other website is live,


but it's a completely different domain.


It's actually a subsidiary company.


And by mistake, they just copied the exact same content.


It went live.


It got indexed.


And then when they noticed that it was problematic,


then they took it down.


And so I'm not sure if that might be the reason.


And so if that's still being canonicalized to something


that doesn't exist, I don't know if that's


the root cause of the issue.


Yeah.


I think one way you can double check


that is to take your existing website, kind


of the original version, and using


that URL tool in Search Console to see what we pick as canonical


And my hunch is that this is less of a technical issue


and really more of a quality issue overall,


where maybe our algorithm determined


that for whatever reason, this website is not


as relevant for some of those queries as it was before.


And that's something that you probably


need to take a look at from a bigger picture point of view.


The one thing I always do when I see sites like that


is to try to understand the situation before and after


a little bit better.


So to see which pages received impressions and clicks


from search before and after, and which types of queries


were there before and after.


And sometimes what I'll see is that there's either


a change across the board, which to me is more a sign of,


overall, we're not really that happy with the quality


of your website, or we don't think it's as relevant as it


was before.


Or I see that just some very high traffic queries


have changed significantly.


And sometimes those high traffic queries


are things that are not really that relevant for your website.


So one thing I've seen, for example, with some news sites


is suddenly you had a page on how to log into WhatsApp.


And that was really popular because it happened to rank


really well for that.


But your news website, so it's not


that you're the authoritative source to logging into WhatsApp.


But because that page received a lot of traffic,


a lot of impressions, if you just


look at the overall traffic in total,


then it'll look like a significant drop.


But it's actually you're getting more targeted traffic.


And it's traffic that matches better what your website is


actually about.


So that's something where looking into those details,


trying to understand the picture before and after,


that can help you to better figure out


what kinds of things you should be focusing on.


SPEAKER 2 OK, thank you, John.


Actually, it seems to be, I mean, some of that situation


seems to be happening because those high volume queries


are the ones that dropped the most.


Those are the ones that I'm not ranking at all.


And I think that might be the case.


OK, sounds sound?


Cool.


OK, wow, I just noticed we're a bit over time.


So let me pause here.


I have the next Hangout session set up


for Friday in the morning.


So maybe less useful for those of you in the US,


but more useful for those of you maybe in Europe


or in India or Asia.


I hope you all stay safe.


Until then, thank you all for joining in.


And look forward to seeing you again


in one of the future Hangouts.


Thank you, John.


Bye, everyone.


English Google SEO office-hours from January 14, 2022

All right.


Welcome, everyone, to today's Google Search Central SEO


Office Hour Hangouts.


My name is John Mueller.


I'm a search advocate here at Google in Switzerland.


And part of what we do are these Office Hour sessions,


where people can jump in and ask their questions around web


search and their websites.


And we can try to find an answer.


Looks like, I don't know, a bunch of people


already submitted questions.


So we have some of that to go through.


But we can also go through some of the live questions first.


Looks like a handful of you have your hands raised.


So let me see.


Let's start off with David.


Hi, John.


I'm an SEO manager for a news publisher in Germany.


And we have a huge complex robots txt


with more than 1,500 lines with a multitude


of disallows that has grown steadily over the years.


And the disallows often refer to HTML fragments and URLs


where HX calls are used.


And where it is not possible to set a no index tag.


So my question is, or I have several questions about this,


are there any negative SEO effects


that can result from a very huge robots txt?


I mean, not direct negative SEO issues with that.


But it makes it a lot harder to maintain.


And it makes it a lot easier to accidentally push something


that does cause issues.


So just because it's a large file


doesn't mean it's a problem.


But it makes it easier for you to create problems.


OK.


OK, I understand that.


And are there any negative SEO effects


if we don't include the sitemap into the robots txt?


No.


Those different ways of submitting a sitemap


are all equivalent for us.


Third question, what would be the SEO effect


if we radically shortened the robots txt?


For example, set everything to allow.


Does Google recognize non-user relevant HTML fragments?


Or do they end up in the index?


And how does Google deal with pages


where HX calls are used?


So it's hard to say what you mean with regards


to those fragments.


Like, are they?


Like a header or a footer element,


they are disallowed because they're not


interesting for the user.


Like, it's a big custom-made content management system


with a lot of these HTML fragments.


And are these fragments that you use in the CMS


to construct the page?


Or are they kind of referenced within the page itself,


like with JavaScript files and things like that?


I think both of these.


Yeah, like they used to build a page and also for JavaScript.


So my thought there would be to try to figure out exactly how


these fragment URLs are used.


And if you're unsure, maybe take one of these fragment URLs


and allow its crawling.


And look at the content of that fragment URL.


And then check to see what happens in search.


Does it affect anything with regards to the index content


on your site?


Is some of that content findable within your site suddenly?


Is that a problem or not?


And try to work based on that.


Because it's very easy to block things


by robots.txt, which actually are not used for indexing.


And then you spend a lot of time maintaining this big robots.txt


file.


But it actually doesn't change that much for your website.


OK, and apart from radically shortening it,


is there any kind of a guideline for the robots.txt?


How do you build it?


No, no.


It's essentially up to you.


Like some sites have big files.


Some sites have small files.


They should all just work.


We have an open source code of the robots.txt parser


that we use.


So what you can also do is get your developers


to run that parser for you, or kind of set it up


so that you can test it.


And then check the URLs on your website with that parser


to see which URLs would actually get blocked


and what that would change.


And that way, you can test things before you make them.


OK.


And where can I find this parser?


It's on GitHub.


I think it's linked from our robots.txt documentation


as well.


OK, thank you.


So I have one additional question about Web Stories.


So I want to know, is there any timeline


when Web Stories for Google Search and Google Discover


are available in Germany?


I don't know.


So I mean, these things usually start off


in some countries or languages, and then they go everywhere.


But I don't know what the time is.


Thank you.


Thanks a lot.


Sure.


Ahmed.


Hi, John.


So my question for this week is actually about heading text.


Some SEOs say that they do not affect your rankings.


And some of them say the only thing


that matters for your ranking is just their size,


like h1 should be bigger than h2, for example.


But when I analyzed your own blog and Google Search Central


documentation, I saw that you and Google


use heading text properly.


Like, you use only one h1 tag per page,


and there is a hierarchy between these heading tags.


And also, with Chrome Lighthouse,


there's a warning when you do not


use heading text hierarchical, it says


you should use them hierarchical.


So should we use heading text like you and Google use,


or can we just use them randomly?


Can I just add, for example, five h1 tags to a page?


So I think, first of all, you should not


assume that just because Google does something,


that it's the best way to do it.


Across all of our sites, there are


people with various level of search SEO-related knowledge.


And not everyone focuses on all of the details


and does it all perfectly.


So I would kind of say, well, Google does this,


but it doesn't mean that it's perfect.


And the same for any other big website.


So many people are always involved


with these big websites.


You can't assume that everything they do is perfect,


even if they rank number one for some of these queries.


So that's kind of my first thing.


With regards to headings, we do use them


to understand a little bit better what the context is


of individual pieces of content.


And for that, it can help if you have a kind


of a hierarchy on the page.


But I think almost the bigger effect that you would see


is with regards to usability, that especially people


with screen readers or other kind of assistive devices,


they rely on these headings to better understand the page


and kind of know where within the page they need to go.


And because of that, it's something


where I would say, well, if you're


going to work on headings on your pages,


you might as well do it, right?


And often it doesn't take a lot of extra work


to kind of figure out how to do the hierarchy right


and set up the templates for that.


So my recommendation would be to do the headings right,


but it's not something where I'd say


you're required to do it the best.


OK, thank you for answering.


Let's see, Saket.


Yeah, hi, John.


Hope you are doing well.


I'm Saket from CED Commerce from India.


We provide affordable solutions for e-commerce,


digital marketing, and global marketplace requirements.


So we are planning to move our one product


category of multi-vendor on a new domain or subdomain.


So we want to know how we move forward in this planning


and how do we implement it and what obstacles we


are going to face and what are the solutions, how


to rank the new domain, because the current domain ranks well


on Google and other search engines


and provide us good organic search traffic,


and how much time it will take to make the new domain


to receive the amount of traffic we are receiving now.


I don't think there is any fixed time for that kind of change,


because it sounds like you're not just moving from one domain


to another.


You're actually moving to one infrastructure


to a different infrastructure.


And often that means the content will be different,


the structure of the pages, maybe even


the URLs will be different.


All of that can change.


And all of these things are elements


which take time to be processed.


And it really depends on the website with regards


to how long that would take.


And it's not that you can have a specific timeline for that.


And I think the other part to also keep in mind


is these kind of changes can have overall positive or


negative effects for a website.


So it's very possible to take this kind of a migration


and say, we will also work on SEO


and improve the interlinking of our pages and the URL structure


and the HTML format of our pages.


And all of these can have very positive effects


for your website.


But at the same time, if you don't watch out for these things


and suddenly you have a big mess of URLs


and the HTML is not easily understandable by search


engines, then that can have a negative effect.


So it's something where you shouldn't assume


that if you're migrating like an e-commerce shop from one


platform to another, that it will


be the same on the other platform


after a certain period of time.


It can be similar.


It can be much better.


It can also be much worse.


So you really need to watch out for all of those details


and think about what is the final structure that you want


and what kind of SEO elements do you want


to include with that migration.


SPEAKER 2


OK, and what kind of negative aspects


we will face when we are going to moving that infrastructure


to a new domain?


It's really hard to say because it


depends on all of the details of what you actually change.


Usually what happens in a situation when everything


is very, very well lined up, you will


see some fluctuations over the time


when we learn about the new website


to when we shift everything over.


And that's something where I would


assume you will see some less visibility in search.


But it really depends on all of the changes


that you make along there where it can take a lot longer


or it can also be something where


the final result is much worse or even much better


than it was before.


SPEAKER 3


If you allow me, I'll share some data with you regarding


the product we are going to move to a new domain.


So basically, the product is six to seven years old.


And there is multiple add-ons and extensions


related to that product.


We have 200 plus blogs, 10k plus backlinks,


and we have multiple partnerships


with leading brands in this industry related


to that product.


And the competition is also too strong for that product


on search engines.


Now, I think it's really hard to say offhand, so what


I would recommend doing if you feel


like you have no idea which direction even to check,


I would take these details and post them


in the Webmaster Help forums and try to get


some advice from other people.


And usually, they can quickly tell you,


oh, it looks like you're moving from this well-known platform


to another well-known platform, and that's no problem.


Or you can just do this extra step,


and everything will be the same.


But it's really something where someone


needs to look at those details.


And usually, that's best done in a help forum,


where you can post all of the URLs and some of the queries


that you're looking at.


SPEAKER 2


Let's see.


Gal.


SPEAKER 3


Hi.


Can you hear me?


Yes.


Great.


So I'm an SEO, and I was given this question,


and I want to provide the best possible answer.


I've got a client that operates in only a handful of states.


And right now, when you go to the home page,


the state that they're headquartered in


is loud and clear.


You know that they're in that state.


But then they feel like it's leaving the other states out


when they come to the page.


So the conversion team has an idea


to create state landing pages, but they


want to automatically redirect the user when


they come to the home page to that state page.


So a dynamic geo IP redirection.


And then since those state pages are essentially the same home


page, just with that state showcased better,


their idea is to no index these state pages.


And I now have a tight ball in my stomach


about this whole idea, and they want to know the SEO


implications of doing so.


OK, so I think there are a few things to keep in mind there.


On the one hand, Google for Search,


we generally just crawl from one location.


And probably for most systems, that


would map back to California.


And essentially what that would mean


is that the content that we can look at


would be the content for California,


and we would not have access to the content


for the other states.


Which depending on what kind of content


you have there for the other states, that might be OK,


but it might be problematic.


So that's kind of the first thing to keep in mind,


is probably it will look like when


you search for your company, it will look like,


oh, this is purely in California,


or maybe even in San Francisco.


I don't know how the IP addresses would map there.


So I think that's something that often throws people off,


especially with geo IP redirects or dynamically swapping


the content.


The other thing is if you noindex the individual state


landing pages, then of course the state landing page


that someone from California would go to


would also be noindexed, which would basically


mean that your home page would drop out of the search results.


So that would be a pretty bad thing.


My general recommendation for these kind of situations


is instead of redirecting automatically


to a specific location, to make it so that the user can


find that content much easier.


So something like a dynamic banner on a page,


when the user goes to the home page,


there's like a banner on top that says,


oh, it looks like you're in Texas,


and we have an office in Texas, and here's the information,


and click this link to find out more.


And that way, the user has the ability


to go to these individual pages.


And ideally, those individual pages


would also be indexable because that way, if someone looks


for your company name plus the state name,


they would be able to find that landing page, which


would be essentially ideal.


The other approach that you could take


is to swap out some of the content dynamically


on the home page.


So instead of having separate state landing pages,


you have kind of your general home page,


and you have that state-specific information dynamically


swapped out.


The important part here is to make sure


that overall, that home page still


has enough generic content so that it doesn't come across


as everything is for California, but rather, it's


like this is lots of information about your business.


And since it looks like you're in California,


here's specific information for California or whatever state


that you're in.


So those are generally the two directions


that we recommend there.


With regards to the individual state landing pages


for a handful of versions, we wouldn't really


see that as being problematic.


If you had landing pages for every city in every state,


then that would start looking a little bit iffy for our web spam


algorithms.


But if you're talking about a handful of states,


or I don't know, maybe even all states,


it's something where you have 50 different versions


of the home page with your local address,


with phone numbers opening hours,


and if that additional local information on them,


from our point of view, that's generally fine.


Thank you so much.


I really appreciate it.


Yeah, I think you can get also a lot of inspirations


from a lot of affiliates in the space.


You can take a look at also Expedia,


how they do it really well.


But yeah, inspiration from that angle would really help you.


Yeah.


Thank you.


With some of the bigger sites that


do this kind of automatically swapping out


the content for location, you can also


see that in the search results.


And it used to be, I think if you search for Craigslist,


the top pages from Craigslist were all California pages.


And then it looked like, well, this is a California company,


and it's not even relevant for me.


And those are the kind of things that you


want to watch out for, to make sure


that the content that's indexed and findable in search kind


of represents your business overall.


And you additionally have that local information,


but it doesn't drown out the general information


for your business.


Perfect.


Thank you.


Cool.


All right, Hazel.


Hey, John.


How are you doing?


Hi.


So I've got three questions.


And the first one is about the redirecting.


And do you know what are the risks of redirecting users


from site page to app?


Because does it have a negative impact on traffic


from an SEO perspective?


I don't think so.


So you mean if you register the URLs for the app?


I think you do that in the operating system,


so that if you go to that URL, it goes directly to the app?


Not exactly.


It's more about because as we were doing the data analysis,


we found that the bounce rate of our mobile is very high.


And it also has low conversion rates.


But our app has a more higher conversion rate.


So we are thinking that maybe we can redirect some users when


they got on some product pages or category pages.


We redirect some users to app or app store.


Maybe we can conversion these users to maybe contribute


to a higher conversion.


Now, I think overall you can do that.


The aspect I would generally watch out for there


is that you do it in a way that lets users


go to the app if they want to.


So I don't know all of the details at the moment


with regards to the connection between the apps


and the web pages.


But I believe there is a way to do a smart banner, where


if you can recognize that the user has the app installed,


it is very easy for them to move to the app experience


from there.


But I don't know the specific details


for Android and iPhone, how that currently is handled.


In general, from a search point of view,


if we can index the individual mobile pages, the desktop


pages as well, or whatever you have available there,


that's perfectly fine.


If people from your pages end up going to the app,


that's also perfectly fine from our point of view.


XIAOCHEN ZHANG HUANG YUEN YANG YUEN So the way you are talking


is about maybe a top banner on the site page.


And maybe if we redirecting them,


like force them to redirect, will that


be maybe bad for SEO or site?


JOHN MUELLER I think probably that would be OK.


I mean, there are two things I have in the back of my head,


which might be something to watch out for.


One, since Googlebot also uses an Android user agent,


you need to make sure that you don't redirect Googlebot


to the App Store or the app, because we


won't install the app.


So that's one thing.


The other thing is with regards to specifically the metrics


around Core Web Vitals, if you always redirect mobile users


directly to the app, then you won't have a lot of data


for the Core Web Vitals.


And depending on your site, maybe that doesn't bother you.


Maybe that's fine.


But it's also kind of something to keep in mind there.


But I think as far as I have it in my head, at the moment,


there is nothing kind of like negative from an SEO


point of view if you redirect users to an app.


I think from a usability point of view,


making it optional is a lot nicer.


But ultimately, that's between you and your users.


Yeah, so good to know maybe we can start by maybe doing


some tests, doing A-B tests to test.


Yeah, thanks.


I think we also still have some documentation about this,


because it used to be a part of search with regards


to app indexing.


And I'm pretty sure we have some documentation either on our side


or on the Firebase app indexing side


with regards to that specific kind of configuration.


Great.


I will check it out later.


Cool.


So the second question is about region-specific subdomains.


Like if we want to build region-specific subdomains


and use half-line to indicate our language,


does the page content need to be exactly the same


except currency?


No.


You can change these pages.


I've also seen situations where you swap out the product name,


because maybe you have a different brand


name in different countries.


That's all fine.


Essentially, it should be equivalent,


that if a user was searching for a specific kind of product,


that they find the equivalent kind of products on those pages.


Yeah.


Yeah, so we're just worried that we


build different orthograms for different subdomains


countrysides.


And we are afraid that maybe different orthogram makes


like some recommendation models and maybe some products,


product listing maybe are different.


So.


That's fine.


Yeah, that's fine.


It's like also the availability of individual products


varies across countries.


And if the category page is the category page


that a user would see in that country, that's fine,


even if it's not the same as in other countries.


OK.


So how does Google measure the similarity of pages?


I think we don't.


I think we basically use the hreflang


to understand which of these URLs are equivalent


from your point of view, and we will swap those out.


So not from the content point of view,


maybe some similar content?


No.


We would only do that for things like the rel canonical


to understand what the canonical URL is.


But for hreflang, I think it's impossible for us


to kind of understand that this specific content is


equivalent for another country or another language.


There's so many local differences


that are always possible.


Great to know.


The last question is about backlinks.


So we are a big e-commerce site, and there


are like millions of backlinks.


And we have a standard procedure to check some spam backlinks


every month or maybe several months.


And we just noticed that the upper limit of the Google


Disabilities is only 2 MB.


And I just wonder if our file has exceeded the limit, then


how to deal with those spam backlinks.


Yeah.


Usually, I would recommend, on the one hand,


trying to use a domain directive as much as possible.


That saves you kind of multiple entries from the same site.


And also, not to focus too much on trying


to clean up all links, because that's always impossible.


I would really focus on using the disavowal for links


where, when you look at them, you


think if someone from the website team


were to look at this, they would be 100% certain


that you bought them or that there was some kind of exchange


happening here.


But for all of these kind of random links


that a website gets, and even from spammy pages


or copy pages or random forum posts,


those are not things that you need to put in the disavowal file.


So currently, most spam links we found are Korean poor sites.


It's very clear that they are spam.


They targeted to our site in to our search pages,


which is super weird for me.


Yeah.


I mean, what they might be doing,


I don't know if this is the case in your situation,


but I've seen that before, is that they target search results


pages with a specific query that includes things


like a phone number or a URL in the hope


that that phone number shows up in the search results.


And if you just no index your search results pages or search


results pages that have maybe a longer query in them,


something like that, then they automatically don't get it.


Oh, that's good to hear.


We will definitely try this method.


Cool.


OK, great.


Thank you.


Great.


Let me run through some of the submitted questions,


and I'll get back to those of you with your raised hands


as well.


Let's see.


The first one I have here is, do I


need to mark up content on my page with HTML semantic


elements, for example, footer?


Does it help somehow to understand


the content for Google?


I think we talked about this very briefly with headings,


but it's essentially similar here.


For some types of elements, it does give us


a little bit more context about the content of your page,


but things like footer don't really give us


that much more information.


So some amount of semantic HTML definitely


makes sense for SEO.


It definitely makes sense for usability and for compatibility


across different browser types, all of that as well.


But it doesn't necessarily mean that it's important for SEO.


Should we expect a drop in traffic if we remove AMP?


So I assume this is a setup where


you have traditional HTML pages and AMP pages,


and you link between them.


I think there are three things that kind of come together when


you remove AMP pages like this.


On the one hand, there are some search features that


are limited to AMP-only pages.


I don't think there are any at the moment.


I would have to double check.


But I don't think there are any search features at the moment


that are only available to AMP pages.


So from that point of view, you wouldn't


be losing anything there.


The other thing is that AMP pages tend to be very fast,


or it's easier to make very fast AMP pages.


And since we do use speed and kind of the page experience


as a ranking factor, it is something


where if you have a lot of very fast pages in AMP


and you switch over to slower pages that are non-AMP,


then you might see an effect there.


You can, of course, make very fast pages


that are not AMP as well.


It's not limited to AMP.


So that's something where I would double


check to see how kind of things with regards to speed


apply there.


And I think the third one is there's occasionally


this assumption that AMP pages somehow rank better,


and that's not the case.


AMP is not a ranking factor, so it shouldn't be something


where you would see a change in ranking


just because you have AMP pages or you don't have AMP pages.


So all of those things kind of together,


if you can make sure that your normal pages are


fast and equivalent and you have all of the structured data


that you need in those normal pages,


then probably you can turn off AMP,


and it'll be essentially very similar.


What you'll probably see is kind of a transitional period


of kind of some AMP pages still being in the AMP cache


and kind of taking a while to bubble out.


But in general, it's possible to turn these off.


We have a Help Center article about this kind


of turning off of AMP pages, so I would double


check that as well.


Yeah, I think that's pretty much it with regards to AMP.


With regards to crawling, you probably


already see that AMP pages tend to be


crawled a little bit more often because we refresh the cache


through crawling as well.


So if you don't have those extra AMP pages,


then our crawling will be focused more


on the rest of your site.


So you might see a rise in kind of crawl activity


on the kind of traditional pages that you have there.


Let's see.


I think we talked about the different states.


I have a site that has a lot of weird spam links


pointing to it.


I added thousands of these to the disavow file.


But I'm concerned I might have taken out


some good links in the process.


There was never a manual action, so I'm


wondering whether it would be wise to test this


by removing the disavow file with the option of adding it


again later if things didn't improve.


I mean, you can always test this kind of thing.


So my assumption is that for most websites,


you don't need to use a disavow file,


and it's easy to get things wrong.


But that doesn't mean that all websites should never use it.


So from that point of view, it's hard for me


to say that you will see an improvement or sort


of a problem with regards to that.


But I would almost take this as something


where you go incrementally through your disavow file


and maybe split it up into parts and then incrementally take out


individual parts and see how that works for you.


And that way, you kind of remove some of the risk


by just kind of blindly switching things on and off


to kind of figuring out which parts of the file


actually don't cause any problems.


And maybe in the end, you'll figure out that, well,


you didn't need this at all.


But it kind of takes away that risk of,


like, if I turn this off completely,


will everything blow up?


Which probably it won't.


Probably it'll be fine.


In recent months, I noticed Google


surveying knowledge panels for certain name searches


vary consistently in mobile and not at all on desktop


for the same query.


The examples where I'm seeing this pattern


are generally business executives who are notable


but don't have a Wikipedia page.


Is it possible to understand why a knowledge panel is considered


appropriate to serve to mobile users


but not to desktop users in this situation?


And is Wikipedia the critical factor


for Google when deciding whether or not


to show a knowledge panel?


So I'm not aware of specific things


that we do differently on mobile and desktop


with regards to knowledge panel.


But it is very common across the different search features


that, depending on the device type,


on the real estate that we have available,


we'll turn some features on and some features


off to try to make sure that we're showing something


that is useful to the user based on the query


that they were using.


So from that point of view, I wouldn't


be surprised if you see different knowledge


panels on desktop and mobile.


But I also don't think there is any one particular factor


that we'd say, this is why you're


seeing this knowledge panel at this time and not


at another time.


Sometimes with regards to these kind of queries


where you are seeing this change,


it might be that it's just on the border of let's


show a knowledge panel or not.


And then maybe the device type just turns it over


and says yes or no in the end.


But that's something where I don't


think there is one specific factor that


is involved with showing these or not showing these.


We do use a variety of different sources for knowledge panels.


And some of that you'll see in the knowledge panel directly.


So that's one thing you can follow up a little bit.


Another tip that I would give with regards


to these kind of things is there are some people


externally from Google who've spent a lot of time kind


of looking into the knowledge panels


and how things are shown, when Google picks things up.


And some of these write about the things that they found


and kind of have their own tools to try that out.


I would try to look up some of those folks


and get in touch with them and kind of like look at the details


that they figured out.


Jason Bernard is one of the people


I know who does this really well.


He's posting on Twitter all the time around knowledge panels.


And maybe that gives you some ideas as well of what


you could be looking at there.


Then synonyms, how automated is it?


Is there a human component?


For example, automobile rentals brings up rental cars.


That makes a lot of sense.


I'm seeing situations where searches


for generic keyword phrase, moving truck or rental truck,


shows a brand budget truck rental instead of a synonym


or no alteration at all.


It's similar to a person searching for inkjet printer


and then getting HP's knowledge panel, in addition.


So as far as I know, the whole synonym system


is completely automated.


It's not something where someone is manually


working on a spreadsheet of synonyms


and kind of like saying, well, this is the same as this,


because that would never really work out.


We see, I think, somewhere along the lines of 10% to 15%


of queries completely new every day.


So it wouldn't be possible for anyone to kind of like keep up


with this all the time.


There is a video from one of the events that we did,


I think like two or three years back now,


where Paul Haar, one of the engineers that works


on this system, kind of presented some of the situations


that we ran across with regards to synonyms,


where we tried to figure out what are the synonyms here,


and some of the cases where we got things completely wrong.


And I thought that was really interesting to look at there.


And if you're interested around synonyms


and how Google figures that out, I


would definitely check that out.


With regards to that kind of generic switch


between moving truck and budget truck rental,


that's something where most likely we also


learned that automatically.


And some of these cases, we get well,


where things kind of work out, or essentially kind


of make sense.


And some of these we get wrong.


If you see cases where we get synonyms wrong,


then definitely send us feedback in the search results


so that we can try to figure out what


we can do to improve that.


My client has business in several locations in India.


In that case, what would be the best method


to include local business schema?


So this is similar to, I think, the question we had earlier


about different states.


If you have different landing pages


for those different locations, you


can put the individual local business schema


on those individual pages with the address, phone numbers,


opening hours, all of that additional information.


So that's kind of the approach I would take there.


I have 15 to 20 FAQs on my web page.


Should I include all the questions in the FAQ schema


or just the questions that I consider important?


When it comes to structured data,


we want to see the structured data visible on the page,


but not all visible content has to be marked up


with structured data.


So if you have individual pieces of content


on your page that you want to give structured data for,


then go ahead and do that.


You don't have to do that for every piece of content


on your page.


So if you have 20 FAQs and you mark up five of them,


that's totally up to you.


You can even use the data no snippet


to kind of completely block some of these other items


from appearing in a snippet, if that's something


that you'd like to do.


Can Google use content that is initially hidden


within accordions, tabs, and dropdown elements


to use for featured snippets?


And people also asked.


The design elements are great for user experience.


However, they have the same probability


of being used by Google for rich snippets compared


to standard paragraph within an article that


is always visible without requiring


the user to click anything.


So for structured data type things like FAQs,


we do require that the information


is visible on the page.


And specifically for FAQs, we require


that the question is visible.


The answer can be kind of within this kind of accordion setup.


For the elements that you mentioned,


the featured snippets and products


that you mentioned, you can use the feature snippets


to be able to access the information


and the featured snippets.


And people also asked.


I don't know if that really applies there.


So featured snippets are essentially


just normal snippets of a page that we would show.


And they can sometimes include things


that are behind an accordion.


My understanding is that we try not


to show much kind of like initially not visible content


in snippets in general.


But it can happen.


If we think that this is particularly relevant for a page,


that it makes sense to give the user additional context so


that they know why your page is a good match for what they're


looking for, then maybe it does make sense for us


to show some of that.


With regards to people also asked, I don't know.


My assumption is it's probably also similar.


But my assumption is also people also


ask is probably more based on what people actually


search for rather than content that is visible directly


on your pages.


OK.


Wow, we still have a bunch of questions left,


but also more and more hands going up.


So maybe I'll just go and switch over to you all here live.


Let's see, Clement, I think you're first.


CLIMANT GOYEN-JOHNSONIYAN LEEUWALIKE


Yes.


I'm working for a French e-commerce website.


We are selling goods for sexual wellness.


Like massage oils, gifts for couples, or sex toys.


That's why we are classified as an adult website by Google.


One year ago, we were ranking first


on the main queries of the market,


like the query sex toys.


And our competitors were second, third, fourth, et cetera.


Today, there are no more adult websites on these queries.


There are only general websites, like Amazon.


I wonder if it's something that Google


wants that e-commerce websites specialize in selling sex toys


are not ranking on sex toy queries, or if it's a mistake.


And I wonder if there is not a bug or something wrong


with a SafeSearch update.


Because for example, when I'm searching sex toy


with my Google account, with SafeSearch deactivated,


and my Google account is more than 18 years old,


I still don't see the adult websites ranking.


They are only general websites, like Amazon.


I don't know if it's normal or not.


I don't know.


If you can maybe copy some of the queries that you're using,


and some of the sites that you expect


to find there into the chat, I can take that out


and forward that to the SafeSearch team to look at.


I mean, it's always possible that there is something


that we could be doing better there.


But I don't know offhand if they would consider


this to be acceptable, or if this is problematic,


or borderline, or how they would see this particular situation.


But if you can maybe include some of the queries


that you're using and some of the sites


that you expect to find for those queries,


that would be very useful for me.


SPEAKER 2


Yes.


SPEAKER 1


So I have a few questions about product reviews,


which is my domain.


So I noticed recently you made some moves in the US


to promote product reviews where there were real testing,


instead of just comparison.


Do you have any time estimates of when


we could expect these upgrades to be in market stock


in French or German?


I don't know.


OK.


With most things, it's something where


we tend not to pre-announce them.


So it's hard to say ahead of time.


And for some of these updates, the team


moves very fast in rolling things out globally.


And for other types of updates, it's very slow.


And sometimes there are also policy and legal reasons


that make things a little bit harder.


So it's always very tricky to give an estimate.


OK, because those were very nice upgrades.


And we're looking forward to have them in Europe, I think.


OK, cool.


I'm curious, could you explain a little more in general?


How could Google train an algorithm


to understand where there were real testing or not?


Is it machine learning, or is it also human reviewers?


For search, for these kind of general ranking updates,


it's not something where we would have human reviewers


in the loop, because there's so much content out there.


We can't review it all.


And a lot of times, we also don't


have a clear yes or no understanding,


where someone can go through a website


and say, oh, this page is good.


This page is bad.


It's just not scalable.


So these are all essentially algorithmic approaches


that we take.


And we do use a lot of machine learning.


Whether we use that for this particular update,


I don't know.


My guess is probably also, at least to some extent.


But it's hard to say which exact technology


was used for which update.


OK.


And do you plan to integrate this kind of understanding


of a web page into the core Google algorithm?


I mean, it's always tricky to define


what is a part of the core update, the core algorithm.


So I don't think I have an answer for that.


No problem.


Because usually, what happens is when


we see that some algorithm is working really well,


then we'll just keep using that for a longer period of time.


And at some point, you could say,


well, it's a part of the core algorithm,


even if it's not in a file that is called core algorithm.


And so I have a small case study where basically,


for example, a website upgraded 20 comparison articles,


two comparison articles with testing.


And you can clearly see on the article


that there are the website bots, the stuff tested for a few time.


And so it was like one year ago.


And unfortunately, it's been one year,


and there hasn't been any noticeable positive evolution


for these specific articles.


So it's just that because of that's


not been integrated into the algorithm yet, I'm guessing?


I don't know.


Because it's weird.


Is it the case that they have some articles that are good


and some articles that are kind of like old style or?


Ah, yeah.


Yeah.


I could imagine that kind of scenario throwing things off.


Because for some kinds of understanding of a website,


we do look at the bigger picture.


And sometimes that bigger picture


goes so far as kind of like the website overall.


And if we have this mixed set of these are really good


and these are kind of OK or not that good,


then kind of what averages out is really hard to estimate.


OK.


I understand.


And so long term quality should win, of course.


But do you have any tips on what can websites


do like short term to do?


Because long term, you can work on upgrading


your business stuff in the short term.


Short term.


I don't know.


It's always tricky because there's


some things that move very quickly in search


and some things that take a longer period of time.


What I would always recommend for sites in general


is to make sure that they're not purely


kind of like only focused on search,


to make sure that they have a diverse set of traffic


to the website.


And usually what you'll see is also


that if you significantly improve


the quality of a website overall,


you'll also see better traffic from social media


and kind of like the different other channels


that you use to drive traffic.


And that usually kind of that mix of different channels


makes it a lot more stable on the one hand,


but also means that if you make bigger changes,


you'll see some of these channels reacting faster,


some of them taking a little bit longer


to kind of like update on that.


So that's kind of like almost like saying,


well, you shouldn't use search for these kind of fast changes.


But I think that's kind of the reality.


As a business, you try to cater to a wide variety of options.


That's all for me.


Thanks.


Cool.


All right.


Akash.


Hi, John.


Hi.


Hello.


Am I audible?


Yes.


John, my question is related to two coverage issues.


One is crawled currently and not indexed,


and one is discovered currently not indexed.


And in both the cases, the pages are not indexed.


This is my main problem.


However, we had recently submitted around 150 URLs


and out of which many of them have got already indexed.


But few pages are not getting indexed.


Google SRP or the Google search, I


know that Google does not index or like the whole content,


it takes time and all those things.


But the thing is that it doesn't give much more information.


It just shows their last crawl or something.


And A, like not applicable or something like that,


because the pages are created, because the rest of the pages


and these pages are almost the same,


they do bring a lot of values.


What should I do when there is very much less information?


Or what should I do to make these pages at least faster?


Linking from the home pages or linking


from the pages which are already ranking for some of the queries


of my particular website.


Will that help me getting those pages ranked a bit faster?


Something like that or could it be bringing more backlinks


and so on?


I think all of those things kind of help.


And it sounds like you're on the right track


and you know a little bit what to expect.


And from our point of view, it's really the case


that we don't index on content on all websites.


And that's kind of expected from our side.


So if you're seeing a big part of your content


already being indexed, I think you're on the right approach.


But it doesn't mean that everything is perfect.


And things like internal linking,


making sure that the overall quality of the website


is really good, those help a lot.


Sometimes it might also make sense


to look at the website overall and say,


well, I have submitted 500 pages in my second file or wherever,


and 200 of them are being indexed.


What is the value of those 300 pages


that are not getting indexed?


And is there something that maybe I


can do to go from having 500 random pages on a website,


maybe reducing down to maybe 300 really good pages on a website,


to kind of concentrate the value into fewer pages


so that at least as those fewer pages get indexed,


you get a lot of the value of those pages back, which


could be that they rank for different keywords


or they work for the users that you care about most,


kind of as a way of prioritizing on your side


before you hand everything over to Google to do.


So that would kind of be my approach there.


On the one hand, making sure that you have everything


lined up properly with internal linking and the overall website


quality.


And on the other hand, if you're seeing


that lots of your pages are not being indexed,


trying to find a way to make it clear to Google which pages


they should be prioritized, which


could be like removing some pages that you don't really


care about or that are not critical for your site.


SPEAKER 2


Is it like, let's say, there are a few keywords which


has a very high search volume, let's say, a shoes keyword.


A shoes keyword will probably have a very high search volume


rather than a shoe at my place or near the home.


A particular query has maybe 10 search volume or low.


Is it something like getting an internal link


from a page which has a high search volume, which


is, again, ranking, rather than a page which


has a low search volume.


Again, it's ranking good, but why


should I at least build internal links from those pages?


Is it something like the high keyword search volume pages


if you interlink the other pages of your website, that


is something would be more valuable or something


like that?


I don't think we would look at the search volume


at all for things like that.


So it's really a matter of understanding


the importance of pages within your website,


and that's not tied to how many people search for those pages.


One last question.


Let's say, if for a particular product,


the manufacturer has been publishing


that particular product to 10 to 20 different sites,


so the overall product description or the product


guidelines or the product value became same, right?


The images could be the same.


The product information could be the same on multiple pages


or the multiple vendors to which they are selling.


Is it still OK to write something extra,


kind of something more unique on my website,


just to make sure that I have something


extra about that particular product, which


rest other people does have, but they are just


copy-pasting the overall description


from the manufacturer, whereas the products are just


like shoes or t-shirts.


It's not something the medical products, which I'm just


bluffing the content, but making sure


that writing more unique or more addition to these


might be helpful to the users as well.


So would that help me to rank better just


to add more value in this app?


I think if you can add more value on top


of generic manufacturer information,


that's always a good thing.


What I would not do is just add kind of like fluff


to these pages and just like, I don't know,


copy a Wikipedia article about the product category


and put that on every page.


It should really be something that adds value to the user.


So if they go there, they see, well, the generic information


is on lots of different sites, but this website


here has some additional information, which


helps me as someone who might be interested in this product.


So really kind of unique and valuable information,


adding that is a great idea.


Just filling extra text on a page, I would not do that.


Thank you so much.


Thank you.


Sure.


Cool.


All right.


We still have some hands raised.


I can go over a little bit, but let me pause the recording


here.


Thank you all for sticking around.


If you're watching this on YouTube,


thanks for watching to the end.


If you'd like to join one of these yourself,


feel free to watch out for these links.


We post them in the community section of our channel.


Usually they're around weekly.


On Fridays, often, we alternate between morning and evening


times, my time, which kind of maps to different locations


in the world.


Anyway, thank you all for submitting so many questions,


and thank you for asking questions here live.


And hopefully we'll see some of you


again in one of the future episodes.


Preparing for page experience ranking | Session

Hi, my name is Jeffrey, and I'm a product manager on Search.


Thank you all for joining,


and today I'll be walking you through the page experience ranking


that's soon coming to Google Search.


And later, Naina will be joining to give us an update


on how the AMP team is setting developers up for success


with this upcoming launch.


Hi, folks. Glad to be here.


Google Search has always considered user experience as part of ranking.


You'd remember some of our past efforts from 2014,


where we announced mobile friendliness as a ranking signal.


The same year, we also launched an effort to rank based on HTTPS


to ensure web pages are delivered in a secure fashion to our users.


Let's take a closer look into page experience.


When we think about page experience,


we think in terms of page experience.


We think in terms of the four pillars of user experience.


They are loading, which signifies how fast or slow


the resources of the page is downloaded and displayed on users' browser.


User annoyance, an important pillar that quantifies some of the web page behavior


that might get in the way of a user accomplishing a task.


Security and privacy, a critical aspect of how safe,


secure, and privacy-friendly a web page is.


And accessibility, the World Health Organization's


Disability Health Fact Sheet finds that over a billion people,


about 15% of the world's population, have some form of disability.


We as web developers have a huge responsibility to build pages


that are inclusive to all our users.


These four pillars provide a structure for how to think about


page experience for your pages.


We believe when pages perform well on these,


they provide exceptional value to the users,


which can drive long-term relationship with your customers.


Let's take a look at how these four pillars translate


in practice for page experience.


For loading, we use Largest Contentful Paint and First Input Delay.


The Largest Contentful Paint, LCP for short,


is a metric that reports the render time of the largest image


or text block visible within the viewport,


relative to when the page first started loading.


First Input Delay, FID for short,


measures the time from when a user first interacts with the page.


That is when they click on a link, tap on a button,


or use a custom JavaScript-powered control,


to the time when the browser is actually able to begin processing


the event handlers in response to that interaction.


For the user annoyance pillar, we have Cumulative Layout Shift, CLS.


CLS measures the sum total of all the individual layout shift scores


for every unexpected layout shift that occurs on the page.


A layout shift occurs any time a visible element


changes its position from one rendered frame to the next.


No Abuse of Interstitials is an existing search ranking policy


and an associated signal that we've long used on search


that detects the presence and use of interstitials that are user-hostile.


We've noticed that such interstitials are often used to trick users


to doing something that they do not want to,


preventing them from reading or interacting with the page


that they landed from Google Search.


There are a lot of great uses for interstitials,


such as the ones required by law, GDPR consent is an example,


or an interstitial that shows updated business hours


during the coronavirus pandemic.


Those are not affected by the signal.


For security and privacy, we want to ensure that the sites are delivered


using the HTTPS protocol.


Users should be able to confidently browse the internet


without having to worry about man-in-the-middle attack


or improper impersonations.


Safe Browsing Systems at Google brings together decades worth of experience


in identifying and mitigating web-based threats,


such as malware, unwanted software downloads, or social engineering.


Similar to some of the other signals, we've long been using safe browsing


on Search to warn users when we detect


that they might be clicking on a potentially harmful site.


Every user deserves to have a safe and secure browsing experience,


and these two signals work together to ensure that sites


are good citizens on the internet.


The mobile friendliness signal covers the accessibility pillar,


which measures how effective the pages are on small screens,


often used by mobile phones.


The first three metrics, largest contentful paint,


first input delay, and cumulative layout shift,


are the Core Web Vitals,


a set of metrics that apply to all the web pages


and should be measured by all site owners


and would be surfaced across all the Google tools.


Each of the Core Web Vitals


represents a distinct facet of user experience


and is measurable in the field


and reflects the real-world experience of a critical user-centric outcome.


They can be measured using JavaScript,


using the standard web APIs,


and provides an excellent product-agnostic path for instrumentation.


To learn more about Core Web Vitals,


check out the talk from the Chrome team titled


What's New in Web Vitals at IO 2021.


Core Web Vitals are not just a set of metrics,


but also a robust set of threshold guidance


that map to user expectations.


The Chrome team has done extensive research


and come up with the guidance for what it means


to be performing good, poor, or somewhere in between.


For instance, LCP,


where we measure the loading performance of the page,


values less than 2.5 seconds


means the page is delivering good user experience.


Pages that take more than 4 seconds are deemed poor.


Similarly, for first input delay,


100 milliseconds is the maximum delay users have to encounter


during the initial input and its response.


Anything greater than 300 milliseconds


starts to feel like the page is frozen


and leads to bad user experience.


Cumulative layout shift is a unitless metric.


0.1 score or below is good,


while 0.25 or greater is deemed poor.


A quick aside.


When the Chrome team first introduced cumulative layout shift,


the metric accounted for the entire lifecycle of the page.


Recently, the team has updated the definition to be more accurate


by capping the maximum duration of the window at 5 seconds.


When Google Search launches page experience ranking,


we will be using this updated definition


of the cumulative layout shift.


As previously announced, when we launch page experience ranking,


we also plan to update the Top Stories feature on Google Search.


The changes include broadening the eligibility requirement


to all qualifying web pages,


irrespective of their Core Web Vitals score or page experience status,


provided they satisfy the Google News content policies.


We plan to use the page experience ranking


to ensure high-quality news articles are surfaced for our users on mobile.


Speaking of devices, today I am happy to announce


that we are bringing page experience ranking to desktop.


While we are launching page experience on mobile soon,


we believe page experience is critical


no matter the surface the user is browsing the web.


This is why we are working hard


to bring page experience ranking to desktop.


As always, we'll be providing updated guidance,


documentation, and tools along the way


to help you make your pages perform at its best.


Stay tuned for more details on this.


One other way you can get more page performance for your users


is taking advantage of prefetching on Google Search


via the use of signed exchanges.


Earlier this year, we announced


that Google Search will prefetch websites built using signed exchanges.


Pages are prefetched and stored onto the user's browser,


ready to be loaded when the user clicks on the result,


leading to near-instant loading.


This is possible through the use of Google's Fast Cache servers,


distributed around the world that you can now take advantage


without any additional cost.


Nikkei, a large Japanese publisher,


saw tremendous improvements in both performance numbers


and business metrics using signed exchanges prefetching.


In their tests, LCP improved by 300 milliseconds,


about 20%, and this was before the team did micro-optimizations.


In other words, 20% was the lowest hanging fruit.


As a result of fast loading,


they saw 12% more user engagement.


Congratulations to the team for such an amazing accomplishment.


If you're raring to get go,


Search Console's Page Experience Report is an excellent place


for all of you to get started.


The Page Experience Report offers valuable metrics,


such as the percentage of URLs with good page experience


and their search impressions over time,


allowing site owners to quickly evaluate their performance.


Site owners can dig into the components of Page Experience Signal


to gain additional insights on opportunities for improvement.


The Search Performance Report has also been updated


to allow for filtering pages with good page experience,


which helps site owners keep track of how these pages


compare to others on the same site.


I'll now turn it over to Naina to talk more about AMP,


our turnkey solution for Page Experience.


Thanks, Jeffrey. Hi, folks.


As Jeffrey mentioned, my name is Naina Raisinghani,


and I lead product for the AMP team.


Now, on the AMP project, we strongly believe that AMP


puts developers on the path to success to not just creating,


but also maintaining great page experience.


This means that the AMP runtime provides developers


with the constraints needed to create a great performing page,


and then AMP caches can actually help further improve


the performance of these pages by making them near instant.


However, like many other frameworks, AMP can't implement


all web development best practices into its runtime.


And this is why in the run-up to the Page Experience ranking change going live,


we actually encourage developers to ensure that AMP pages serve


from both the publisher's site or via an AMP cache


are both equally optimized.


And the first step to a great page experience here is the AMP core.


That is the AMP extensions and the runtime that enforce


the design constraints that make AMP great.


The next building block is how the page is being served.


Core Web Vital metrics for a page are determined by observing


real user interactions with web pages.


And in the case of AMP, this means that the pages could be served


from either the publisher's domain or via an AMP cache,


depending on how the user encounters the content.


Many sites actually see a significant proportion of their AMP visits


actually occurring on their own domain.


Now, to get the strongest possible user experience on your AMP pages,


we highly encourage all site owners to implement a number


of additional optimizations that they can perform themselves.


AMP optimizers are actually tools that bring the AMP cache optimizations


to your own site.


And by using the latest AMP optimizer,


you're getting the easiest way to automatically benefit


from any new server-side optimizations for AMP.


You can learn more about AMP optimizers at the link on the screen.


And that's how the AMP project actually thinks about performance.


We make sure that you have the tools to create


and then maintain really great page experience.


And in fact, we also launched the AMP page experience guide last year.


It's a one-stop tool to help developers debug


why their AMP page might not be meeting page experience criteria.


And since launch, we've actually added even more actionable feedback


to the AMP page experience guide, making it even more useful.


These improvements have actually been driven


by websites testing their AMP pages using the tool.


So we encourage you to try out the tool


at the link shown on the screen and provide feedback.


And that's that.


We on the AMP team genuinely believe that AMP sets you up for success


when you're looking at creating and then maintaining well-performance sites


in the long run.


We hope you give it a try and leave your feedback.


And with that, I'll hand it back to Jeffrey.


Thanks, Naina.


With page experience ranking, we are on a multi-year journey


to surface high-quality content


with exceptional user experience for our users.


We want to give developers actionable guidance


through the various tools we offer,


such as Search Console's page experience report


and AMP's page experience guide.


With that, I want to thank all of you for joining us today.


We hope you are staying safe


and have a good rest of IO 2021.


Thank you.


Removals in Search Console - Google Search Console Training

Hi, I'm Daniel Weisberg, Search Advocate at Google.


And today I'll show you how to request your content


to be temporarily hidden in Google Search results


using the Removals report in Search Console.


And I'll also talk about the ways to review requests made by other users


to remove outdated or inappropriate content on your site.


Google Search Console


How to Review Requested or Inappropriate Content


Before I go into Search Console capabilities,


I would like to stress that the tool should only be used


for temporary removal requests.


You must take additional steps in order for your content


to be removed permanently.


I'll talk about permanent removal of pages from Search


at the end of this video,


but check the links in the description for more detailed information.


In addition, you cannot use the tool to remove personal information,


report offensive information,


or doing anything but managing search results from your own website.


To remove information from Google, from the web in general,


for content you don't control, check the links in the description.


Now that we're aligned on what can be removed and for how long,


let's talk about the Removals report in Search Console.


That's the reason we got together today.


The Removals report is divided into three main sections.


Temporary removals,


outdated content,


and safe search filtering requests.


Let's go over each of them in more detail


to help you understand how to use them effectively.


A temporary removal request is a way to remove specific content


on your site from Google search results.


For example, if you have a URL that you need to take off


Google search quickly, you should use this tool.


A successful request lasts only about six months,


which should be enough for you to find a solution


to either allow the content to be seen or to remove it permanently.


When you log into Search Console,


you'll find the Removals report in the navigation bar.


Select the temporary removals tab, if it's not already selected,


to submit a new removal request.


Then click New Request.


You have two types of requests available.


Temporarily Remove URL will block the URL from Google search results


for about six months.


This will also clear the cached copy of the page.


Use this option as a quick first step


when permanently blocking a page from Google search results.


Clear Cached URL clears the cached page


and wipes out the page description snippet in search results


until the page is scrolled again,


when the snippet is generated from the new content.


Until the next scroll,


the page description will be empty in search results.


Use this option when you remove sensitive information from a page


and want to update your results snippet in Google search


without removing the page itself from search.


Be sure that you've removed the sensitive information from your page


before clicking this,


or else Google will just recrawl the page and find the same content.


For each of these two options,


you can submit a request for a single, specific URL


or for an entire URL prefix.


Make sure you think twice before submitting a request


for an entire prefix


so that you don't unintentionally block pages that are important to you.


In the temporary removal section,


you'll find a summary of the page removals


requested through this report in the past six months.


You'll see the URL, the request type, the date requested, and the status.


You can also use this report to cancel a request


if it's no longer necessary.


The outdated content section provides information on removal requests


made through the public remove outdated content tool,


which can be used by anyone to update search results


showing information that is no longer present on a page.


This is something that you probably won't need to look at much,


since Google users can't force your pages from search using this tool.


There are two types of requests.


Outdated cache removal is used when the page still exists,


but some content has been removed.


Google clears the page snippet in the search result until the next crawl,


and searches for the removed content will no longer show this page.


Outdated page removal is used when the page no longer exists,


and a user has requested its removal from the Google search results.


The report table shows the history of all submitted requests for this property


in the past six months using the public remove outdated content.


I won't go into the tool in this video,


but you can check the links in the description to learn more about it.


Whether you're using search at work, with your family, or for yourself,


SafeSearch can help filter most sexually explicit results.


In addition to setting your personal account preferences,


parents can turn on the filter for their kids' supervised devices,


and workplaces or schools can turn on the filter at the network level.


The SafeSearch filtering section in Search Console


shows a history of pages on your site


that were reported by Google users as adult content


using the SafeSearch suggestion tool.


URLs submitted using this tool are reviewed,


and if Google feels that this content should be filtered from SafeSearch results,


these URLs are tagged as adult content.


The report table shows the history of all submitted SafeSearch requests


for this property in the past six months.


Check the links in the description to learn more about it.


Permanent removals should not really be part of this video


since it's not requested or reported by Search Console.


But since we're talking so much about temporary removals,


I thought a short explanation might be in order.


To remove content from Google Search permanently,


you must take one or more of the following actions.


Remove or update the actual content from your site


and return either a 404 or 410 HTTP status code.


Non-HTML files, like PDFs, should be completely removed from your server.


Block access to the content, for example, by requiring a password.


And use a noindex meta tag to indicate that the page should not be indexed.


This is less secure than the other methods.


Please know that simply redirecting an old page with a 301 response


is not enough to remove it from Google Search results.


And neither will a robots.txt directive work as a blocking mechanism.


And that's it. We're done for today.


Hopefully now you understand how to request to remove


specific content on your site from Google Search results.


And also, how to track these requests along with other user removal requests


related to your site.


In the next episode, I'll talk about how to use Search Console


to check security issues on your website.


Don't forget to subscribe to the Google Webmasters YouTube channel,


where we'll be publishing lots of Search Console videos.


Stay tuned.


Remove...


SEO Starter Guide: What’s all the drama?

Welcome, everyone, to another episode of the Search Off


the Record podcast.


Our plan for this series is to talk a bit about what's


happening at Google Search, how things work behind the scenes,


and who knows, maybe have some fun along the way.


My name is John Mueller.


I'm a search advocate on the Search Relations team


here at Google in Switzerland.


Today, we have a special episode lined up


with Lizzie and Michael.


Hi, everyone.


Hi.


Hello.


Lizzie's been on our show a few times.


She's also on the Search Relations team,


and she's our tech writer.


But Michael, do you want to introduce yourself briefly?


Sure.


Yeah, I'm Michael, or a lot of people also know me as Wiz.


I worked on Search from 2006 to 2015


and got to do a bunch of things with people


who work on websites.


So you've been around for a while.


Like, maybe you've seen me at conferences or in the forums


or something like that.


I no longer work on Search, but I'm still at Google.


And I write a documentation for internal tools now.


So good to see everyone.


Fantastic to have you here.


Thank you.


Many of you are surely wondering why I have summoned you at this hour.


What started as a joke turned into reality.


Someone, a user, commented on our documentation


that it would be fantastic if we could have our documentation


in audio format.


Lizzie took the opportunity to notify us about this feedback.


And now we are going to do a dramatic reading


of our own documentation.


And the very best place to start with this journey


is our SEO starter guide.


How do I get my site on Google?


Inclusion in Google search result is free and easy.


You don't even need to submit your site to Google.


Google is a fully automated search engine


that uses web crawlers, crawlers, crawlers


to explore the web constantly, looking for sites


to add to its index.


In fact, the vast majority of sites listed in our search results


aren't manually submitted for inclusion,


but found and added automatically,


automatically even when we crawl the web.


Now the SEO starter guide actually predates most of us


in the search relations team.


I know of very few Google dinosaurs that have been around


back when it started, but I know for certain that Wizz and John


both have been around.


Wizz, would you like to say a few words about how


this thing started, and most importantly, why?


Sure, absolutely.


I actually reached out to the original author


to get the inside scoop on how this all happened.


So I had a little conversation with the author,


a little conversation with Brandon,


who used to work on the search quality team.


And the first thing that he said is that most good things


from our team, it was Matt's idea.


So Matt Cutts, who we all know, used to be in our videos


and speak at conferences and really did a lot


to get this search relations stuff going at Google.


What we wanted to have was advice straight from Google


about SEO, because we would get asked about SEO all the time,


both internally and externally.


Internally, we can't give any different advice


than we give externally.


So it was like, well, let's see if we can create something


that we can give internally and externally,


and it will be great to have that.


It was also important for us to show


that Google isn't against SEO.


Sometimes we get questions about that, and we weren't.


If it's done well, this is what we wanted


to show, the best practices.


So that's how it came to be.


Originally started as a PDF, but then we also


had a booklet version with the Google Bot Coon character,


who's now everywhere.


It's been really cool to see that it's still going and still


getting updated.


JOHN MUELLER, MD, PhD, MPH, CEO, Google


I totally remember the booklet.


I think we had a bunch of those printed as well,


and we brought them to conferences, right?


Yeah, that's right.


And actually, Brandon was saying that Matt wanted


to have something at conferences,


but in the initial versions, we just did it as online.


But then independently, the Tokyo team,


so we have a search relations team in Tokyo,


they wanted a booklet as well.


And this was shortly after Chrome had come out,


and they had had that comic book with the Chrome browser


release.


And also at conferences, I think a few of us


had mentioned that we'd sometimes be asked,


do you have a manual for Webmaster Tools,


or a booklet, or something?


And we would sometimes be the only booth that didn't


have anything to hand out.


Everyone else had all this swag and printed materials


to give out.


So anyway, eventually, we, thanks to the Tokyo team,


ended up doing the guide and a printed version,


and with that, got the illustrations


of the great Google Bot Coon character that we have now.


Oh, yeah.


That was, I think, the beginning of the Google Bot character.


We still have that.


I don't know if you noticed.


We also added a new character to the lineup.


Have you seen it?


I heard there's a spider.


Yeah, the spider.


What's that spider's name again, Lizzie?


Oh, don't hurt me like that.


We still don't have a name, so I guess


this is another open call.


If you have any ideas, you can tweet at us on Twitter


your suggestions for the name for the spider.


Spider names.


We can call it, I don't know, four eyes or something.


Actually, I don't know how many eyes spiders have,


so maybe that was insensitive.


Sorry, spiders.


For sensitive information, use more secure methods.


A robot's txt file is not appropriate or effective way


of blocking sensitive or confidential material.


It only instructs well-behaved crawlers


that the pages are not for them, but it does not


prevent your server from delivering these pages


to a browser that requests them.


Also, non-compliant or rogue search engines


that don't acknowledge the robot's exclusion standard


could disobey the instructions of your robot's txt.


Finally, a curious user could examine the directories


or subdirectories in your robot's txt file


and guess the URL of the content that you don't want seen.


Yeah, one of the early sections in the guide


is all about robots txt.


It feels like one of those things that just remains


the same for ever and ever.


How do you see that?


Do you think all of that is still relevant?


Where did that come from?


Yeah, I was rereading it, and I saw that it also mentions


that you shouldn't use robots txt for sensitive content.


I mean, it's kind of like a sign that says,


don't look here, and it tells you exactly where not to look.


And that actually also reminded me that,


well, this was a while ago at this point.


We were noticing that we didn't have a lot of documents,


especially, I think, from government sources from Korea.


And the team actually did a lot of work to figure out,


why is this happening?


And a lot of these sites were just blocked completely


with robots txt.


And they did some investigation and asking around,


and they found out that there was actually


a government document with recommendation.


I think it was in the security section that was like,


make sure that you block all robots with robots txt.


So that's why there wasn't this good coverage.


And so the team actually did a lot of work


to reach out to site owners.


And I think they even got that document changed.


Yeah, so it's still relevant though.


It's actually fascinating because the issue still persists


to some extent, and we still have to do some outreach


to government sites in Korea due to the robots txt rules


that they set.


And the more fascinating thing to me


is that they just blanket block every single robot.


It's not like they would allow certain robots that


is more typical in other regions of the world.


They are just blanket block everything.


So I find that fascinating, even though I


think it's a misunderstanding of robots txt


and what it's supposed to do and not do.


So what's the best approach with robots.txt?


Should they just allow everything or delete the file,


or is that all the same?


I mean, it's for controlling crawling.


Like, what do you want to have crawled or actually not


have crawled?


But since you are writing in plaintext format


in the robots txt file, you can't use it to hide stuff.


Like, for example, if you have a secret folder slash


where you store your secret cat pictures,


then you don't want to put that in the robots txt file


because a human can also open it and then discover


your secret folder.


So for that, it's not good.


But it's good to manage, for example, crawl budget.


Like, if you have a section of your site


that you don't think that it's that useful,


or if you have certain URL patterns that


are just a different version of the same contents


that you allow crawling, for that you can use the different


robots txt disallow rules.


So you mentioned that these government sites


had a misunderstanding of how, like,


what was their initial understanding?


Like, why were they, oh, we want to block everything?


Did they think it was a secure method?


Or like, why was it tied to security, I guess?


I don't know.


Like, we never found out why they were doing it.


We just asked them, like, what's up with that?


And can you please do it this other way?


Yeah, I'm not sure.


John, do you remember?


I remember this was mostly the Korean team


that was working on it.


Yeah, I think one of the issues there was on some of these sites


they had personal information where


you could look up something about your citizen ID


or something like that.


And the government sites were kind of worried


that the site owner might not know


where the confidential information is.


And kind of as a best practice, it's just like, oh,


just block everything.


Then you'll be, like, oh, I'm going to block everything.


On the safe side.


On the one hand, a little bit of understanding


of how the web works, a little bit like search engines


go off and find stuff on websites,


but not that level of understanding that actually


if you make something public on the web,


then it'll be accessible.


But you don't have to make things public


to publish them on the web.


You can put them behind a login.


You can kind of lock them down in more legitimate ways.


So it's kind of almost like a feeling that, well, these site


owners don't really know what they're doing.


Therefore, they should just block everything,


which then, of course, leads to a situation


that nobody can look up anything about that site.


And it's like, where can I look up


local government information?


And it's like, well, it's nowhere.


Or rather, we almost have to guess


because if it's blocked by robots.txt,


we wouldn't index the content on there.


But we might see links pointing there.


And we'd be like, well, maybe this site is relevant to you.


But we don't really have any idea.


So that's a really weird situation where that ends up.


I think also in Korea, there was kind


of a unique situation with the search engines in general


because the other internet sites there


were more directory-type sites where people would explicitly


list websites rather than them going off and crawling


the web to find stuff.


Yeah, I think I remember that same idea about,


I forget if it was like forums or portal kind of format, which


is maybe why the issue wasn't so obvious in the local markets


unless you used a search engine that


actually goes out and crawls.


In a world of mobile devices, most people


are searching on Google using a mobile device.


The desktop version of a site might be difficult to view


and use on a mobile device.


As a result, having a mobile-ready site


is critical to your online presence.


In fact, starting in late 2016, Google


has begun experiments to primarily use


the mobile version of a site's content


for ranking parsing structured data and generating snippets.


Oh, I actually have a funny story


about the mobile-friendly test.


It was when it first launched.


I was like, OK, I'll just try my personal website on it.


And so I opened it in the mobile-friendly test,


and it's a super, super simple hand-coded HTML page


with HTML from the 90s.


So it's like a link, and then a break, and then another link,


and another break, and then a link.


And I'm like, there's no way that I


have to do anything special for this to work on a phone.


It's the simplest possible thing.


And I put it in the test, and it's


like the tap targets are too close together or too small


or something like that.


And I was like, that's not right.


This is the simplest possible page.


There's no way that it's right.


And I was sharing an office with two other people on the team,


Eric and Mary.


And I said, the test isn't right.


Like, it's saying that this very simple page isn't working.


And it says that the tap targets are too close together.


And I'm like, look, let me show you that it's wrong.


It's the simplest page.


And so I opened up my phone, and I opened up my home page.


And I'm like, look, so if I just want to tap on this link


here to my blog, and I went to tap,


and I completely missed the link,


and I ended up on the wrong page.


And so that was a pretty humbling moment


where I was like, oh, I guess the test is right.


We're all good.


It's so cool, all of the mobile-friendly stuff that we had


there and kind of the guidelines in general for mobile.


It's funny, some of the first SEO conferences I went to,


I was like, well, we have all of this mobile stuff.


This is to access the internet.


It just felt so interesting to have all of this information


there so early.


And essentially, over time, we didn't end up changing too much.


Most of that just ended up working well


with mobile phones.


So that's pretty cool.


Well, I guess apart from the feature phone stuff, where


we still have some of that in our docs every now and then.


But I imagine nobody actually has these phones anymore.


So what's up with feature phones, by the way?


I don't know.


I mean, it's probably not that they broke


or that the batteries are empty now,


because they're pretty robust, right?


So it's like my old feature phone compared


to my smartphone, the display is still OK,


and the case is still OK.


Wait, are you still using it?


I have it in a drawer.


And every now and then, I'll look through my drawers


and it's like, I need to throw out some old stuff.


I'll be like, oh, but this feature phone,


it could probably still work if I charged it up.


Imagine if the apocalypse came and smartphones no longer


worked, I would be the only one with internet access.


Probably not.


No?


I mean, the edge network was turned down here


a few months ago.


Oh my gosh.


So you're saying I'm keeping it for nothing?


That's terrible.


Yeah.


So I shouldn't optimize my site for feature phones?


Ask yourself, does it spark joy?


Probably, yeah.


Then keep it.


Keep it, OK.


I think the harder part is finding


the right charger for it.


Seems like a critical part.


Critical part?


I don't know.


Every now and then, I get tempted


to take a photo of the Google homepage on a feature phone


just to show that it still works.


But maybe it doesn't work anymore.


It's just like keeping it for nothing.


So awkward.


I don't want to be dramatic, but you should really


be promoting your website.


Putting effort into the offline promotion of your company


or site can also be rewarding.


For example, if you have a business site,


make sure its URL is listed on your business cards, letter


heads, posters, et cetera.


You could also send out recurring newsletters


to clients through the mail letting


them know about any new content on the company's website.


Yeah, so there's this other section


about promoting your website.


And I was looking at what's in there.


And there's a section that just says, know about social media.


And I was wondering, what's up with that?


Like when this guide was written, what year was it?


And what were we meaning by that just


to know about social media?


I mean, the year was 2008.


And?


So what was social media back then?


Twitter, right?


Was Twitter?


Twitter was, I don't know what Twitter was.


Was Twitter there?


I think Twitter just started.


I think that's roughly when it started.


We had Google Plus.


Oh, no, wait.


Google Plus.


Oh, Google Reader was also a social network, right?


Where you share things with other people.


Oh, that's right.


Google Reader.


That's true.


It was.


Yeah, and Buzz.


Buzz.


Don't you remember Buzz?


Wasn't that after 2008?


I have no idea.


I think it was after.


It was like 2010 maybe.


But we definitely had Delicious, for example.


We definitely had Stumble Upon.


We definitely had Dig.


And based on personal experience and based on data


from hundreds of my sites, these were actually very, very good


for driving traffic to certain kinds of sites.


So back then it was less about ranking and more


about just getting traffic?


I mean, it depends what you wanted to do with your site,


same as today.


So there's this point in here about that you should avoid


attempting to promote each new small piece of content


and instead only do the big interesting things.


Do you think that we would be still recommending that today?


Or is that something that we should go back and adjust?


Should people post when they've, I don't know,


done a new blog post, for example?


Like we tweet when we do a blog post.


Would that be all right?


I think it depends on the site or the section of the site.


Because if you have a blog and you post once a week,


for example, then you do want to let people


know that you posted.


But if you are a big news site, say CNN or Wall Street Journal,


then you can't possibly tweet everything, for example.


If you tweet everything, it will be just like a constant flow,


like a writer's feed, for example,


the professional feed, where you literally


have a constant flow of headlines coming in.