have a ton of experience with these kind of problems.


English Google Webmaster Central office-hours from September 6, 2019

And they can quickly help you to narrow things down.


So it might be a case that there was just


like a temporary crawling problem that essentially you


don't need to worry about because it's


come and gone again.


It might also be the case that there's


something systematic on your website that


is preventing Google from being able to crawl these URLs.


And there are a number of ways that you can test these pages,


using maybe the Inspect URL tool in Search Console,


using other tools as well.


And by getting help from other people who


have done similar things, it's a lot easier


to kind of narrow things down and to figure out


what exactly is the problem here.


And at the same time, you also learn


how to kind of diagnose these types of issues, which is,


I think, always a good skill to have.


So I'd recommend taking this with some sample URLs,


maybe some queries or screenshots even,


and posting it in the Webmaster Help Forums.


What happens if we disallow the robots text


by robots text?


Then everything explodes.


No, not quite.


So essentially, we will still have


to fetch the robots text to be able to see the robots text.


So disallowing the robots text file in robots text


does not change much with regards


to the requests made to the robots text file.


It does prevent indexing of the content of the robots text


file, though.


So what you would see is if someone links to your robots


text file in the sense of, well, maybe this


is an interesting page to show, which probably it isn't,


then technically we might go off and try


to index the robots text file.


And we could theoretically show that in the search results


if someone is searching for a way that clearly looks


for this robots text file.


And if you block crawling of the robots text file in robots text,


then we would still show that URL in search.


We would just have the snippet saying,


well, we can't give you any more information


because we can't crawl this page.


So practically speaking, I don't think anything will change.


I don't think you make things any more complicated or harder


or anything, but it's kind of unnecessary.


You don't really need to do it.


It's similar, for example, if you


want to block all crawling of a website,


you would use disallow colon and then slash.


That would also block crawling of the robots text file.


But I think blocking it explicitly


doesn't really make much sense.


It adds a little bit more complexity


to your robots text file.


And a year later, you'll be like, what was I thinking?


There must be a reason for me to block the robots text


file when perhaps you're just doing it for a bit of fun.


My blog is so-and-so.


I'm writing quality content for my readers.


Currently, I don't have a lot of traffic.


Let's see.


Recently, my blog was approved for AdSense.


Still, I choose not to show ads because I'm worried


about the quality of my traffic.


I'm trying to look at my articles again and again.


What can I do to better present my site to users?


I don't know.


So I took a quick look at your site.


It seems like a reasonable site.


It's really hard to say what exactly you


should be doing differently.


What I'd recommend doing is maybe


going through the SEO starter guides that are out there.


There's one from us.


There are a bunch from other people as well.


And kind of going through to make sure


that you have everything covered.


On the one hand, there are things around the way


that you present the content on your site.


Maybe you're doing well there.


Maybe the content that you're writing is fairly well.


On the other hand, there are also ways


that you can kind of help promote that content


and get it out there so that other people talk about it


and so that other people have a chance


to find it and link to it if they find it useful,


all of these things.


It's sometimes tricky to get started.


I think it's also really tricky to get started and kind of


start with an SEO topic on a blog


because there are lots of SEO blogs out there.


It's really hard to kind of differentiate one blog


from another sometimes.


So that's something where my first thought when


looking at your site was maybe it would make sense


to find a topic where you're really passionate about.


That's not something that's already extremely well covered


in the industry.


And to kind of find your audience there


and to start off with a small audience that


really loves your content and grow that step by step


rather than starting off completely fresh and saying,


well, I want to create an SEO blog


like all of these hundreds of others that are out there.


Because if you're kind of starting off completely fresh


and you want to compete with this giant set of other sites,


then that's going to be really, really hard.


That's not something that you can just kind of tweak


by having some nice blog posts on your website.


You really need to provide something unique and compelling


across your site as well.


Googlebot.


Yeah.


Go for it.


SPEAKER 2.


I have a question.


So John, we know that Googlebot can't


be able to crawl the infinite scroll pages.


But my question is, how can Googlebot


be able to figure out all the URLs or all the links


in the infinite scroll pages?


Example sitemap is the one way.


So is there any other way that Googlebot


can crawl all the infinite scroll links?


We have a blog post on infinite scroll.


I will take a look at that.


There are some techniques that you


can do where you show paginated links


on the bottom of the page, for example,


and then we can follow those paginated links as well.


So I'd definitely check that out.


I think it's maybe three or four years old in the meantime.


But it's still valid.


So I have a question.


So yes.


Go for it.


Yeah, if you see there are a number of headers


and photo links there in each and every website.


So whenever Googlebot crawls, how


it will consider the header and photo links?


Because in your Googlebot will crawl from the top


to the end of the page.


So how it will consider the headers and photo links?


It will crawl each and every page headers and photo links,


or it will just keep the headers and photos,


and it will just crawl the remaining links?


Does it affect the crawl budget?


We do try to crawl all of the links on a page,


regardless of whether they're in the header or the footer.


It's something if you're linking your site in the header,


if you have a menu on top, then that's


a great way for us to crawl through the website.


So it's not a matter of where those links are.


It's essentially just that these links exist.


The thing to keep in mind with regards to headers and footers


and sidebars is that, generally speaking,


these are repeated across the website.


So from our point of view, we see them


as something that's not critical to that specific page.


So we understand these belong to your site.


We can still crawl them.


We think there's probably good content here.


But when we try to index pages and show them


in the search results, we try to focus on the primary element


on the page.


So if you have a blog post, for example,


we'll try to take that blog post and treat


that as the primary element.


And the headers and footers, we'll follow the links.


We know they kind of exist.


But it's not seen as critical as that primary blog post


that you have in the middle.


SPEAKER 3 OK.


So I have one final question.


So yeah, so in our site, we have a number


of old 301 redirection URLs are presented over there.


So let's say I have 1,000 URLs or 1,000 links of old 301


redirection URL presented in my website.


So does it affect my new category


of which is redirected?


Which does it affect the ranking of our new category?


No, it wouldn't affect the ranking.


That definitely not.


It could potentially make it harder to crawl your website.


So in particular, if we need more than five redirect hops


to find the final page, then we will try


to recrawl that in a second step.


If you just have a collection of redirects from one old page


to the new page, then that's perfectly fine.


If you have maybe two steps in between, that's perfectly fine.


If you have more than five steps,


then that's something that I would avoid.


In general, I recommend making sure


that your redirects go as direct as possible,


just to kind of avoid running into a situation where you go


from two to three to four, and then suddenly you


have seven or 10 redirect hops, which make things


a lot harder for us to crawl.


OK.


Thanks, John.


Sure.


Yeah, there's still a few questions left,


but maybe I can just switch over to you all.


What else is on your mind?


What can I help with?


Hey, John.


Hi.


I asked that question in the chat, but I can ask it here.


We have syndicated some of our content


on a new website with another website,


but they are outranking us.


We've added our canonical tag, but still they're winning.


Is there another way to avoid this apart


from adding a no index on the other website?


Because they really want more traffic apart


from organic traffic, since it's a new website,


but I'm afraid it's hurting us, since it's no longer what


do you call it?


The content will be duplicated, so it's no longer unique.


So generally, it wouldn't harm your website


in this kind of situation.


So I think that part is kind of fine.


But any time you syndicate content to other sites,


it can potentially happen that they outrank you.


So that's something that is always worth keeping in mind.


I think it's also something that's, I don't know,


both a risk and an opportunity in that sense,


in that it could be that your site is ranking number three.


And if they kind of syndicate the content for you


and they can rank number one, then suddenly your content


is a lot more visible than if it was just on your website.


But at the same time, it's also a little bit of a risk


in that maybe you want people to go to your website


instead of to their website.


So that's, I think, always something to keep in mind.


When you're syndicating content or when


you have affiliate relationships,


if you kind of have resellers that sell your products,


it can happen that any of these other people that are


essentially reusing your product or reusing your content,


that they rank above you.


And there is no kind of meta tag at Google


where we'd say, this is the original.


You have to show us a number one, not the other people.


The rel canonical helps us a little bit there,


but it doesn't guarantee that we will always follow that.


So I don't really have a solution there.


I think it's something where you kind of need to probably


discuss internally and think about,


do we want our content to be visible like this,


or do we need to make sure that people only go to our website?


But it's not hurting us.


Exactly.


It's not hurting you.


It's not that your website would be ranking lower because


of this kind of syndication setup.


OK, great.


Thank you.


Sure.


All right.


I mean, we still have time.


I can run through some of the submitted questions,


but sometimes it's more interesting with live questions


if there's anything from any of you all.


I can add something if that's fine.


It's regarding the how-to structure data markup.


I'm wondering for more complex content,


for more technical content perhaps,


where you kind of lay out the steps you


need to take to do something, but you also


have to include a lot of maybe more in-depth explanations.


But you don't really want to include that


in the how-to markup because it kind of makes it a lot bigger,


and you don't really have the right attributes


to kind of include that.


So I'm wondering if the content is not exactly the same,


so maybe it's more of a summary of steps within the markup.


Is that still fine, or does it need


to be a one-to-one exact match?


I think that would probably still be fine.


So on the one hand, it's something manually


if we look at that, that would probably be fine.


Algorithmically, I could imagine that our algorithms


might be a little bit confused if they say,


this isn't a markup, and this is visible on the page,


and it doesn't match.


If in the markup you're providing


a subset of what you're showing on the page,


then I don't see a problem with that.


So if you have something like how-to steps,


and you say, here's the first step,


and then click here to see the rest,


I generally wouldn't see a problem with that.


And one more thing.


I'm wondering, so for a more technical IT-related content,


like where you actually need to run,


you have an example, run this command, maybe install Python


on Ubuntu or something like that,


and you need to run this and this and this,


does that need to be included?


The actual lines of the actual commands,


do they need to be included within the markup,


or how would you go about this?


That's totally up to you.


I think that depends a lot on the website and on your goals.


So if, for example, you need to get that information out,


and you don't really need to have people visit your page,


put it in the markup.


If, on the other hand, it's really important for you


to have people visiting your page,


maybe that's how you track your metrics, your performance,


maybe you have ads on the page, then that's


something where you might say, well, I'll limit the amount


of content I have in the markup so that it encourages people


to visit my page instead.


I think there are totally valid use


cases for both of those directions.


For example, if someone is looking for your opening hours,


they don't need to visit your page.


They want to come and visit you in person.


They just need to have the opening hours.


Whereas if someone is interested in potentially hiring you


because you're an expert on this topic,


then probably you don't want to expose all of the secrets


that you know directly in the markup directly.


You want people to visit your page


and learn about the fancy things that you've been doing.


So for example, if I have a command like git


and to take the latest version of Python,


maybe it should be better to just including the markup,


use git or get the latest version from GitHub


and not actually include the actual line.


So that's fine as well.


That's totally up to you.


And that's something that I would certainly test.


And see, given the number of impressions that you get,


are you getting the clicks that you think are useful?


Kind of double check your other metrics


as well to figure out, those people who


are coming to my pages, are they actually relevant users?


Or are these people who are basically confused and just


need something to copy and paste and they will disappear again


and never come back?


I don't know.


It really depends on what you're trying to achieve.


OK, I just want to make sure that Google, as you mentioned,


doesn't get confused.


Well, you're saying this in the markup,


but the content is a bit different


if I look at the actual page.


I think if the markup is a subset of your actual content,


I don't see a problem with that.


If it's completely different, then that


might be something where algorithms say, well,


wait a minute, this doesn't match what's on the page.


We don't know if this markup is really


important to be shown directly in the search results.


OK, I'll test it out.


Sure.


Cool.


There's a question in the chat from someone


in a noisy environment.


I have experience with, I think, e-commerce apparel sites.


And this is the first time I'm encountering a site


from a manufacturing company.


The site structure is not set up in a way


that the product detail pages only contain one product,


like I usually see for a typical e-commerce site.


Is it common for manufacturing companies


to set up their site structures so that multiple,


so in this case, four specific products,


are shown on a single product detail page?


Is there a big effect for SEO friendliness and ease


of crawling if each product has their own respective product


detail pages and not bundled into one


page for multiple products?


In general, if the content is on your site,


then we can show it in the search results.


So it's not a deal breaker completely.


But it does make it a little bit harder


for us to recognize what these pages are exactly about.


So if the multiple products are all related,


it's more like variations of the same product,


maybe different colors, different sizes,


then I don't see a problem with that.


Because that's all about this one product.


And these are different variations.


And having one product page rather than one page


per variation makes it a little bit easier


for us to focus our signals on those pages.


On the other hand, if these are completely unrelated products,


then it's really hard for us to recognize


that this is actually a useful landing page for someone who's


looking for that product.


So let's say if you have, I don't know, shoes,


and you have one model shoes with different colors


and sizes, having that on one page is fine.


People will search for shoes, and they can go to that page.


On the other hand, if you have shoes, and then dresses,


and then hats, and you put a variation of these


all on one page, then if someone is searching


for a pair of shoes, we don't really


know if this is the best match for them.


Because it has so much information about hats


and dresses as well that it's like maybe someone


searching for a collection of clothing makes sense.


Maybe someone's searching for an individual hat.


Maybe it doesn't make so much sense.


So that's something to kind of keep in mind there.


If these are variations, I definitely


don't see a problem with that.


If these are completely unrelated,


then that makes it a lot harder for people to find those pages.


If this is a manufacturer site where end users are not really


meant to go to those pages and buy products there directly,


maybe it doesn't matter.


Maybe it's enough that distributors


are able to kind of look up, I need


to find a manufacturer who has this specification of products.


Then that's specific enough to still bring up these pages.


So I guess in a typical SEO way, I would say it depends.


But it's something where based on your knowledge


of that website, of the company, and of their goals,


that's something that you can probably make a decision on


whether or not this is OK or if this is perhaps suboptimal.


Maybe there's also a way to tweak things a little bit where


you say, well, currently these are unrelated products,


but I can tweak it so that the related products come together


on one page.


Then that would probably be a lot better.


Hello, John?


I still have a bit of time.


So if any of you want to ask questions, jump in.


Hi, John.


Hi.


John, I have a question regarding Google Search Console.


So I was checking this URL.


I mean, I can show you that URL in chat also.


So that URL particularly is, I mean,


when I was checking it into the Google indexing,


it was showing me that URL has some crawl indexing issues.


So this is the URL I have shared you in the chat.


So this is the URL.


It's showing me URL is not on Google indexing errors.


But the same URL, when I check into the live test,


it's showing me URL is available to Google.


And it's showing me the availability


URL can be indexed, mobile usability, facts, logos,


everything is done on the page.


But when I check this into the Google index,


it's showing me that the pages, the referring page


is not detected because maybe the URL is not interlinked,


maybe that could happen.


And the page fetch is also showing in the error crawl


anomaly.


So I'm just confused, like, which one I should follow?


I mean, Google index or live test?


Because right now, one is telling me


that my page is on Google.


But one is telling me I have indexing errors.


So what should I do in this situation?


I don't know exactly which errors you're seeing.


So it's kind of hard to say.


Just purely from looking at the URL,


my worry is a bit that on this website,


you're generating URLs for a ton of different variations.


So it looks like you have car service center,


and then you have a location, and then you


have a type of car brand.


All of these are URLs.


So my worry would be that you have


hundreds of thousands or even millions of URLs


that are essentially variations.


It's like all car service centers,


and then you have lawyers, and then you have teachers,


and you have all of these different locations.


And you're generating URLs for all of them.


Then that's something where I could definitely


see that our systems might say, well, we're seeing so many URLs


from this website.


We don't know if it's worthwhile to crawl and index all of them.


And that's something where probably the right approach


first would be to think about what kind of content


do you really need to have indexed,


and also to think about are there perhaps ways


to limit the amount of content that is on your website?


Are there ways to make sure that all of the content


that you're providing is of the highest quality possible?


So that our algorithms, when they run across this URL,


they try a little bit harder to actually get it indexed.


SPEAKER 2


So that's kind of the, I think, the first step


that I would take there is to avoid running into a situation


where you have millions of URLs that are essentially like,


I could index them, but they're not really fantastic.


SPEAKER 1


We are planning to rewrite the URL for the whole website,


because right now we have one extra directory in our URL


system.


So that's why I have already discussed with my developer


team.


But right now, I will just, this bit I was confused.


Because maybe this error is coming because of that pop-up


also.


Maybe you have seen that pop-up once the page is loading.


But the problem is this URL is in two different categories.


So right now, this is in two submitted URL


has a crawl issue into this error this is showing me.


So I mean, my confusion is just a little bit


this like I'm seeing it into the Google indexing,


as well as in the live test.


So yeah, of course, I'll be making the changes


to the particular all the, I mean, around the website.


I'm changing this, the city-based pages


into one particular like car service center and car service


center Hyderabad.


So I'm merging these two categories.


So I'm creating only one category, car service center


Hyderabad.


So I'll remove the previous category.


And then I'll be in this way.


But yeah, I mean, my question was like this.


How can I solve this issue?


Or maybe if this is having the crawl issue,


so reducing the crawl depth can help me, or I mean,


or should I take any?


So what I guess happened here is that we tried to crawl this URL.


It didn't work out.


And because we have so many URLs otherwise on this website,


I didn't check like what the rest of your website


looks like.


I'm just guessing.


But because we have so many other URLs on this website,


we don't bother spending a lot of time retrying it.


Maybe we'll try it again in a couple of months.


But it's not something where I would say, well,


we really need to get this URL.


We'll retry a few times until the crawl error goes away.


So my guess is it kind of ran into that.


The other thing to keep in mind is


that with mobile first indexing, we


would be using a mobile crawler for that.


So what we sometimes see is that testing tools,


when they check with a desktop crawler, it looks OK.


But when they check with a mobile crawler,


it doesn't work.


So that might be another thing to kind of double check


and make sure that that's OK.


Well, yeah, currently, actually, the source is smartphone.


So the Google master is showing this source is smartphone


for this particular URL.


So I mean, yeah, I have to look into the directory level.


But yeah, I mean, that was, yeah, cool.


I'll show.


Yeah.


I think it's tricky with really large websites like this.


If you look at individual URLs, then it


can often be the case that we say,


we just have so many URLs from this website.


We look at individual URLs maybe once a month,


maybe once every two or three months,


because we just have such a mass of things


that you want to have from.


It's also pretty good, but dynamically.


So yeah.


And then errors stick around for a little bit longer,


because we don't get around to double checking them


that quickly.


Cool.


Thank you, John.


OK.


No, I have a question too.


All right.


Let me just switch off the recording


so that the YouTube video doesn't get infinitely long.


I still have a bit of time, so you're welcome to stick around.


But I'll just stop the recording now.


Thank you all for joining in, and hope to see some of you


again in one of the future Hangouts.


And if you want to stick around for a little bit longer,


feel free to do that as well.


Intro to Google Search Console - Search Console Training

Hi, I'm Daniel Weisberg, Search Advocate at Google.


And in this video, I'll explain the basics of Search Console


and how it can help you succeed on Search and the web in general.


Search Console is a free tool from Google that helps website owners,


SEO professionals, and developers understand how they're performing on Google Search,


and what they can do to improve their appearance on Search


to bring more relevant traffic to their websites.


Search Console is not a requirement for your site to appear on organic search results,


but it can definitely help you monitor and optimize how Google crawls,


indexes, and serves your website to users.


If you're not acquainted with the terms crawling, indexing, and serving,


you should read through our documentation.


It will help you get the most out of this video series.


Here are some things you can do in Search Console.


Learn how Google crawls, indexes, and discovers your pages.


Fix errors Google finds when crawling your pages.


Submit updated content to the Google index.


And monitor search performance trends by queries, countries, pages, and more.


You might be asking yourself how you should be using Search Console,


or what is the most relevant information for you.


Here are some ideas, but don't worry if you don't understand all the terms.


We'll discuss each of them separately in this video series.


If you own a small website and you're responsible for maintaining it,


but not as your main job, you should start simple.


Check your search traffic in the performance report,


and try to understand which queries, pages, and countries are driving most of your traffic.


If you're responsible for building or maintaining a large website,


you should come to Search Console to check all your pages are indexed


and have no errors.


Check for indexing errors in the coverage report, the mobile usability report,


the AMP report, and other enhancement reports, if any are shown to your website.


If you're an SEO professional, well, you'll have lots of fun with Search Console.


Depending on whether you're more focused on technical SEO, content development,


strategy, or something else, you'll find important insights in the tool.


Just browse our channel and watch some more videos.


Of course, this is just a short list,


but you can check the Search Console Help Center for a more detailed explanation


on how to use the tool depending on your role, your site, and your personal needs.


I hope you enjoyed this video.


There are many more to come.


In the next video, I'll talk about how to activate Search Console for your site


so that you can get access to your Google search data.


Don't forget to subscribe to the Google Webmasters YouTube channel,


Make your Vue.js web apps discoverable - JavaScript SEO

If you are building single page apps, you might be using a JavaScript framework to do so.


One of the most popular frameworks is Vue. Vue is popular for being pretty lightweight,


using components like most other JavaScript frameworks, and having a vibrant active community and ecosystem of developers and tooling.


In this video, we will build a sample application and make sure it is discoverable via search,


so your users and people looking for your app are able to find it.


We will use Vue CLI in version 3.04.


We create a new application, then install Vue Router and set up two components,


PostsList, that will list 10 posts, and PostDetail, that shows a single post.


After setting up the routing and components, we have this application that roughly resembles a block, pulling the posts from an API.


We also enable the History API mode of the Vue Router to make sure our URLs are proper URLs instead of abusing the fragment identifier to show different views.


Let's look at what we can improve in this application for SEO.


First things first.


To make our posts more findable and more relevant in search results, we should give each page a descriptive, helpful title and description.


Those two things will show up in the search results and help users identify the best result for what they are looking for.


To give all our views meaningful metadata, we will use the Vue meta utility.


It allows us to set the title and meta description of our PostList view to something meaningful


and use an excerpt of the post and its title as the meta description and title for the post detail view as well.


Note that ideally we would have some summary or key takeaways for each post,


but in this case we are just going to put the first 100 characters from the post in the description instead.


100 is an arbitrarily chosen number of characters.


Meta descriptions aren't restricted in length.


When we put this into the mobile-friendly test, we see our meta description and title show up in the code.


That means we will see them in the search results as well.


But there's even more we can do to both bots and users.


We can make sure our content arrives within the initial HTML sent to the client without having to wait for JavaScript to execute.


As we explained in the previous videos, Googlebot does two waves of indexing.


In the first wave, we crawl your page and index the content we find, but we do not execute JavaScript due to resource constraints.


Eventually, and unless you tell us not to, we will do a second wave of indexing where we execute your JavaScript and index your content again.


But if you have a large site or lots of frequently changing content, this might not be optimal.


A way around this is pre-rendering or server-side rendering.


Vue has extensive documentation and tooling for this.


Alternatively, you may consider doing dynamic rendering, which we provide some documentation for as well.


I also recommend watching the other videos from this series for more SEO tips and tricks for single-page apps in JavaScript.


Also, in the next episode, we will be discussing dynamic rendering.


Security issues report in Search Console - Google Search Console Training

Hi, I'm Daniel Weisberg, Search Advocate at Google.


And I'm Aurora Morales from the Trust and Safety team.


What? How did you get here? It's a security breach.


Want to check my ID?


In a bit.


At the moment, I could do with some help explaining the security issues report in Search Console.


That alerts webmasters about malicious behavior on their websites.


And since your team protects Google users, including Google Search, would you be up for that?


Por supuesto.


The Search Console security issues report shows warnings when Google finds that your site might have been hacked


or used in ways that could potentially harm a visitor or their device.


For example, a hacker might inject malicious code in your pages to redirect your users to another site


or to automatically create pages with nonsensical sentences filled with keywords on your site.


These are examples of website hacking.


An attacker might also trick users into doing something dangerous, such as revealing confidential information


or downloading malicious software.


That is called social engineering.


When a site has been compromised to manipulate search rankings,


it can be labeled as this site may be hacked to warn users in search results.


When a site has been compromised to harm users,


browsers with Google Safe Browsing technology enabled may display interstitial warning pages


or alerts when files are downloaded.


For example, to warn users when they are about to access malicious websites.


Google Safe Browsing also labels results in search, such as this site may harm your computer.


In the next video, I'll go through some of the most common security issues that Google sees on websites,


so make sure you don't miss it.


To learn more about the subject, you can also check the links in the description.


Now, Daniel will show you how to find the affected pages on your website


and how to use Search Console to fix them.


When Google detects a security issue on a website,


Search Console verified site owners will receive an email alerting them about this issue


with a link to more information on how to solve it.


Read your emails carefully.


It's important that you see these alerts as soon as possible.


Even if you miss an email, you can always log into Search Console to check if you have any issues.


In the overview page, you'll already be notified if you have security issues on your site


through a banner at the top of the page.


Clicking the alert will lead you to the Security Issues report,


where you'll find a list of all security issues related to your website.


Google does its best to check sites for the most common security issues,


but it should still keep your eyes open.


You can also navigate to the Messages panel and choose the Security Issues category


to see all related messages that have been sent to your website in the past.


These can give you context on your website history,


especially if you're new to a company or took over an existing website.


To fix a security issue on your site and let Google know,


we recommend the following steps.


Expand the Security Issue Description panel to learn more about the issue


and follow the Learn More link for detailed information and steps to fix it.


After you read about it, decide whether you think this is something you can fix.


If not, find someone on your team or externally that has web development experience


or visit the Webmaster Help forums.


Use the sample of affected pages provided in the Details section


to troubleshoot and fix your issue.


You might have a security issue with no example URLs.


This doesn't mean that no pages are affected,


only that we couldn't generate samples for that specific case.


Fix the issue throughout your entire website.


Fixing it on just some pages will not solve the problem,


so make sure you leave no pages behind.


If the report lists multiple security issues, fix all of them.


When all issues listed in the report are fixed in all pages,


select Request Review in the Security Issues report


and describe your fixes in your review request.


A good request does three things.


It explains the exact issue on your website,


it describes the steps you've taken to fix the issue,


and it documents the outcome of your efforts.


Check out John's latest video on web spam manual actions


and reconsideration requests for more on that.


Most reviews take a few days, but some can take up to a week or two.


You'll get a review confirmation message


when you submit your request to inform you that the review is in progress.


You'll also get a message when the review is completed.


There is no need to submit another review request


before you get a final decision on your outstanding request.


If you can't see hacked content on the URLs provided in Search Console,


this might be an example of cloaking.


Cloaking makes cleaning a website more difficult


because it shows different content to users


than it does to search engines.


For example, a page on your site may not have visible hacked content to you.


This might lead you to believe that there is no hacked content


and that Search Console is sending you wrong messages.


However, a search engine like Google accessing the same page


might be served hidden spammy text and links.


Check the links in the description to learn more about cloaking.


In this video, we shared an overview of security issues


and the process you need to follow to fix them on Search Console.


In the next video, Aurora will go deeper into different types of security issues


Search Console reports on.


Subscribe to the Webmasters YouTube channel


for more on security and Search Console engine.


Stay tuned.


So where's your ID?


Service worker sorcery

of SEO fairy tales.


Service worker sorcery

With me today is Day Smart, technical SEO and dev


Service worker sorcery

at Tame the Bots.


Service worker sorcery

And I just found out you invented purple ceramics


Service worker sorcery

for use in jet engines?


Service worker sorcery

I did indeed, yeah.


Service worker sorcery

How did that happen?


Service worker sorcery

That was my first job out of school and out of uni.


Service worker sorcery

And I'm a specialist in technology.


Service worker sorcery

I'm a senior at Tame the Bots.


Service worker sorcery

And I go to my graduate school.


Service worker sorcery

And I do a lot of engineering.


Service worker sorcery

of school and out of uni.


Service worker sorcery

I did material science in an aerospace lab


Service worker sorcery

and got to work on that.


Service worker sorcery

It was great fun.


Service worker sorcery

What?


Service worker sorcery

You brought a technical SEO challenge to me


Service worker sorcery

that you experienced.


Service worker sorcery

What is that?


Service worker sorcery

It was to do with service workers.


Service worker sorcery

No, OK.


Service worker sorcery

Twice this actually happened.


Service worker sorcery

Two people have approached me now with sites


Service worker sorcery

that it just didn't quite work.


Service worker sorcery

One, it just didn't work at all.


Service worker sorcery

And they were relying very heavily on the service worker.


Service worker sorcery

So they were just ending up with blank pages and no indexing.


Service worker sorcery

Oh.


Service worker sorcery

The other was related, but maybe quite so serious in some ways.


Service worker sorcery

But they only ended up with partial indexing.


Service worker sorcery

OK, but that to me is confusing.


Service worker sorcery

Because for those of you who don't know service workers,


Service worker sorcery

the idea is it's kind of like an add-on thing.


Service worker sorcery

It adds to the functionality of a site,


Service worker sorcery

but does not necessarily, or not per se, have such an impact.


Service worker sorcery

Because what it should be used for,


Service worker sorcery

I got to be treading carefully here,


Service worker sorcery

is that it should basically just be


Service worker sorcery

a layer between your site and the network.


Service worker sorcery

And that allows offline capabilities,


Service worker sorcery

and push notifications, and background sync and stuff.


Service worker sorcery

So what did they do?


Service worker sorcery


Service worker sorcery

In the first case, when you have a service worker on a site,


Service worker sorcery

you need to register it.


Service worker sorcery

Yes, correct.


Service worker sorcery

So the browser will come along.


Service worker sorcery

You'll find the reference to the service worker, register it.


Service worker sorcery

Try to register it.


Service worker sorcery

Try to register it, even.


Service worker sorcery

However, what they were doing was waiting for that to happen


Service worker sorcery

before they were doing all the other client side rendering


Service worker sorcery

things.


Service worker sorcery

So before they went and fetched all the data


Service worker sorcery

and put everything to do.


Service worker sorcery

Which kind of worked for a user.


Service worker sorcery

It could be a little slow, because you


Service worker sorcery

have to wait for that service worker to register.


Service worker sorcery

However, something like Googlebot,


Service worker sorcery

it doesn't register something.


Service worker sorcery

It has to basically refresh.


Service worker sorcery

So.


Service worker sorcery

That's actually not even a bug or anything.


Service worker sorcery

Because the idea is, again, it's an optional feature


Service worker sorcery

that you might opt out of, or browsers might not support it.


Service worker sorcery

So you are requesting it to be registered,


Service worker sorcery

and it might fail or never register.


Service worker sorcery

And we actually document that behavior,


Service worker sorcery

because a service worker might change


Service worker sorcery

the behavior for a returning visitor versus a first time


Service worker sorcery

visitor.


Service worker sorcery

And we assume that people on search results


Service worker sorcery

are first time visitors.


Service worker sorcery

Sure.


Service worker sorcery

Yeah.


Service worker sorcery

How did you figure that one out?


Service worker sorcery

Because I think that debugging that is not super easy, is it?


Service worker sorcery

Yeah, it's not always very easy.


Service worker sorcery

Basically, a lot of what you're doing


Service worker sorcery

is looking for stuff that isn't there, that should be.


Service worker sorcery

So the first thing you can do with them,


Service worker sorcery

with the search console, we could look and see


Service worker sorcery

there's very little actually indexed.


Service worker sorcery

And lots of things like software offers.


Service worker sorcery

So I'm saying, this looks like an error page.


Service worker sorcery

The few pages that were indexed were basically just the shell.


Service worker sorcery

So when a site is client side rendered,


Service worker sorcery

they send just a minimal shell.


Service worker sorcery

It's a placeholder, basically, until everything


Service worker sorcery

gets fetched in.


Service worker sorcery

Yeah, that's right.


Service worker sorcery

And basically, that's as far as everything got.


Service worker sorcery

So then it's onto the live testing tools.


Service worker sorcery

So things like URL Inspector does a lot of tests.


Service worker sorcery

There's rich results tests.


Service worker sorcery

There's a mobile friendly test.


Service worker sorcery

If you run it through them, you can see,


Service worker sorcery

well, it's the same result. It's empty.


Service worker sorcery

However, what you do get is, under the More Info little tab


Service worker sorcery

you'll find on the side, you can see the resources that


Service worker sorcery

are loaded in or haven't loaded in.


Service worker sorcery

So you can see the request being made.


Service worker sorcery

Yeah, that's it.


Service worker sorcery

So you can see if they've failed,


Service worker sorcery

if they've been successful, or if they've just not


Service worker sorcery

been tried at all.


Service worker sorcery

I was about to say, let me guess,


Service worker sorcery

you probably saw them not even being tried.


Service worker sorcery

Exactly, yeah.


Service worker sorcery

Oh, and then the conclusion would


Service worker sorcery

be, oh, JavaScript in Google Search does not work.


Service worker sorcery

Nah.


Service worker sorcery

What there also is is there is a JavaScript console as well.


Service worker sorcery

Right, yes.


Service worker sorcery

So that can give some useful feedback.


Service worker sorcery

But it depends a little bit on how the site's made,


Service worker sorcery

because it depends on what they're logging out.


Service worker sorcery

Exactly, if you have no error logging,


Service worker sorcery

nothing ends up in the console.


Service worker sorcery

So at that point, it's time.


Service worker sorcery

Go make friends with your devs.


Service worker sorcery

Mm.


Service worker sorcery

And go and ask them.


Service worker sorcery

Basically, that should be your first step


Service worker sorcery

after you discover that there is a problem.


Service worker sorcery

Go to your devs.


Service worker sorcery

But that's a tricky one, because if you


Service worker sorcery

don't know what's going on, what are you


Service worker sorcery

going to talk about with them?


Service worker sorcery

Yeah, well, that's the thing.


Service worker sorcery

You don't necessarily know how they built the site.


Service worker sorcery

They know how they built the site.


Service worker sorcery

You know there's a problem.


Service worker sorcery

So go and talk to them.


Service worker sorcery

And you start from the first step.


Service worker sorcery

You make little breadcrumbs on the trail through the path.


Service worker sorcery

And you talk through the process of how


Service worker sorcery

does data come from your server and end up on the screen.


Service worker sorcery

So each little step along the way.


Service worker sorcery

And then you can body up and become like a detective tag


Service worker sorcery

team.


Service worker sorcery

Like, ooh, OK, that's cool.


Service worker sorcery

Get them to log each little stage in the JavaScript console


Service worker sorcery

log, something like that.


Service worker sorcery

And then you see where it fails to continue.


Service worker sorcery

And then it fails.


Service worker sorcery

So then when you're running through the live test again,


Service worker sorcery

you can see exactly the point that something's not happened.


Service worker sorcery

And you can ask them, well, how does you


Service worker sorcery

call in your main content from API forward slash cats?


Service worker sorcery

Exactly, from something.


Service worker sorcery

Yeah, and it's not coming through.


Service worker sorcery

You say, well, how exactly does that happen?


Service worker sorcery

What function does that happen in?


Service worker sorcery

And tracing it through that way, you


Service worker sorcery

can find the point it falls over.


Service worker sorcery

And then hopefully the devs will have a good, clear path


Service worker sorcery

of where to go.


Service worker sorcery

Yeah, fair enough.


Service worker sorcery

So in the first case, it was quite simply


Service worker sorcery

a case of moving all the calls to the APIs


Service worker sorcery

and everything out of that whole pretty string


Service worker sorcery

of the service worker logic.


Service worker sorcery

That's pretty cool.


Service worker sorcery

I like that you're basically approaching it


Service worker sorcery

from the perspective of, ooh, it's kind of murder has happened.


Service worker sorcery

Let's solve this together.


Service worker sorcery

And then you team up with the developers,


Service worker sorcery

because I've heard other people are having a different strategy


Service worker sorcery

than yours, where they're basically


Service worker sorcery

trying to figure out what the problem is and then


Service worker sorcery

going to their developers.


Service worker sorcery

I mean, it can work.


Service worker sorcery

The problem is with a lot of modern sites these days,


Service worker sorcery

they're built and they go through something


Service worker sorcery

like Webpack, the JavaScripts are sophisticated.


Service worker sorcery

It's a whole pipeline.


Service worker sorcery

So you can't just see, oh, it's this function.


Service worker sorcery

So at that point, you probably don't


Service worker sorcery

have the tools and the information


Service worker sorcery

to be able to say exactly where it's failing, just the outcome.


Service worker sorcery

Mm.


Service worker sorcery

And looking at a site that was partially indexed,


Service worker sorcery

I guess that must have been even harder to debug.


Service worker sorcery

It was, because a good deal of the stuff was indexed.


Service worker sorcery

Was there.


Service worker sorcery

And pages were indexed.


Service worker sorcery

Unfortunately, it was a little block of information.


Service worker sorcery

But it was quite the important block of information.


Service worker sorcery

One of the things one of the superpowers service workers


Service worker sorcery

give you is to work offline.


Service worker sorcery

And this was information that was


Service worker sorcery

relevant for people going out and doing jobs and, you know,


Service worker sorcery

curve.


Service worker sorcery

Right.


Service worker sorcery

Yeah, yeah, yeah.


Service worker sorcery

The devs thought, what a great way to do.


Service worker sorcery

To make that available offline.


Service worker sorcery

Yes.


Service worker sorcery

Which it does.


Service worker sorcery

Good thinking.


Service worker sorcery

And what they also thought is, we want to get loads of it.


Service worker sorcery

So they overtook the normal process of fetching


Service worker sorcery

and did it purely within the service worker.


Service worker sorcery

I did that in the past.


Service worker sorcery

Yeah.


Service worker sorcery

Not going to lie.


Service worker sorcery

I feel with the devs here.


Service worker sorcery

It was irrational.


Service worker sorcery

And it made sense.


Service worker sorcery

And it worked for the end user.


Service worker sorcery

Yeah, for the end user.


Service worker sorcery

Just unfortunately enough for search.


Service worker sorcery

Yeah, exactly.


Service worker sorcery

And unfortunately enough for search.


Service worker sorcery

And how did that manifest?


Service worker sorcery

And so you said you used Search Console and specifically


Service worker sorcery

the Coverage Report.


Service worker sorcery

Yeah, you can see.


Service worker sorcery

How did that?


Service worker sorcery

These guys are getting covered.


Service worker sorcery

They're getting indexed.


Service worker sorcery

But they're just not surfacing for things


Service worker sorcery

they really should surface for.


Service worker sorcery

Someone went in and did a search for a bit of this content.


Service worker sorcery

And it just didn't turn up.


Service worker sorcery

It was really quite unique.


Service worker sorcery

Didn't turn up.


Service worker sorcery

Their visibility.


Service worker sorcery

And then from that point where you see, OK, so it is crawled.


Service worker sorcery

Because if it's not crawled, it can't be indexed.


Service worker sorcery

It is indexed as well.


Service worker sorcery

It's just not serving in search results.


Service worker sorcery

How did you then figure out where to go next?


Service worker sorcery

Did you then head over to the devs?


Service worker sorcery

Or was there a step in between?


Service worker sorcery

The first step, before you head over to the devs,


Service worker sorcery

is you just go, is it just not ranking very well for this?


Service worker sorcery

Is it badly?


Service worker sorcery

Have you got a lot of competition?


Service worker sorcery

Because not everything's technical.


Service worker sorcery

It's ranking.


Service worker sorcery

Yeah, true.


Service worker sorcery

Might be a content issue.


Service worker sorcery

Yeah, exactly.


Service worker sorcery

So we had a look at the rendered HTML.


Service worker sorcery

If you look in Search Console, you'll see.


Service worker sorcery

I was about to ask, where's the rendered HTML?


Service worker sorcery

And use the URL Inspector.


Service worker sorcery

Don't do the live test, just the first step of it.


Service worker sorcery

And there's a nice little panel.


Service worker sorcery

No pretty pictures, but that's not really what matters.


Service worker sorcery

No.


Service worker sorcery

But you can have a look, is the HTML there?


Service worker sorcery

And sure enough, no sign of that.


Service worker sorcery

So that then gives you something to go to the devs with.


Service worker sorcery

I think you can even search in the rendered HTML.


Service worker sorcery

And then you type in the same unique content.


Service worker sorcery

And you're like, wait a minute.


Service worker sorcery

This can't show up, because it's not here.


Service worker sorcery

So then that gives you a clear path to go back to the devs.


Service worker sorcery

And basically, you repeat that process.


Service worker sorcery

How does that get from there?


Service worker sorcery

And it's just breaking down steps.


Service worker sorcery

That's nice.


Service worker sorcery

That's interesting.


Service worker sorcery

There's one thing, one opportunity


Service worker sorcery

that I think we haven't talked about.


Service worker sorcery

And that is actually the developer tools


Service worker sorcery

in your browser.


Service worker sorcery

Because you could go into the network inspector


Service worker sorcery

and try to find out, where does this piece of content


Service worker sorcery

come from?


Service worker sorcery

And I think there it says who or what initiated it.


Service worker sorcery

Which then you need to go to the devs with,


Service worker sorcery

because then it's going to be like app.min.js line one,


Service worker sorcery

which is a huge line of spaghetti code.


Service worker sorcery

And they're like, OK, great.


Service worker sorcery

DION ALMAEREN


Service worker sorcery

Interesting.


Service worker sorcery

You could use the developer tools in the browser as well


Service worker sorcery

to disable service workers.


Service worker sorcery

If you have a suspicion that it's the service worker.


Service worker sorcery

That's right.


Service worker sorcery

Yeah.


Service worker sorcery

It's a tricky one, probably.


Service worker sorcery

You just need to go in DevTools.


Service worker sorcery

If you go to the application panel.


Service worker sorcery

It is a bit hidden, yeah.


Service worker sorcery

You might need me to search to find that in the first place.


Service worker sorcery

And then there is bypass as well,


Service worker sorcery

if you find a service worker.


Service worker sorcery

There's a little tick box, bypass.


Service worker sorcery

It actually says bypass on this network.


Service worker sorcery

So it always goes on network, yeah.


Service worker sorcery

So it's just the network that you're on at that time.


Service worker sorcery

It will then start bypassing the service worker.


Service worker sorcery

But you get a great idea of what the site would look like when


Service worker sorcery

your service worker doesn't register or doesn't work.


Service worker sorcery

Ha!


Service worker sorcery

Interesting.


Service worker sorcery

OK.


Service worker sorcery

To recap, you had a client coming into your,


Service worker sorcery

actually two clients coming in with a site that


Service worker sorcery

wasn't indexed at all.


Service worker sorcery

And then one was indexed but was missing content.


Service worker sorcery

And both of that you could trace through Google Search Console


Service worker sorcery

and the URL inspection tools to be an issue that the devs


Service worker sorcery

needed to look at.


Service worker sorcery

And then you teamed up with the devs


Service worker sorcery

to actually work through that.


Service worker sorcery

How did the dev part of the process work?


Service worker sorcery

So you went to the developers and went,


Service worker sorcery

where's this coming from?


Service worker sorcery

Because it's missing?


Service worker sorcery

Exactly.


Service worker sorcery

Yeah.


Service worker sorcery

So you just get to explain step by step which way,


Service worker sorcery

well, a function calls that.


Service worker sorcery

And then what calls that function?


Service worker sorcery

And you trace it back to the origin point


Service worker sorcery

of someone coming in.


Service worker sorcery

And then that's where they can work from and fix that.


Service worker sorcery

I guess that should be not too hard to fix in some case.


Service worker sorcery

Well, in some cases.


Service worker sorcery

Certainly in the first case, it was very easy fix.


Service worker sorcery

It was literally just not work for the, yeah.


Service worker sorcery

The second case, it was a little bit more of the knuckling


Service worker sorcery

they needed to do.


Service worker sorcery

I can imagine.


Service worker sorcery

I've done weird things as well with Service Worker.


Service worker sorcery

Service Worker is a great but also dangerous tool


Service worker sorcery

if you're holding it wrong, as Gary would say.


Service worker sorcery

Am I holding it wrong?


Service worker sorcery

Awesome, Dave.


Service worker sorcery

This was a wild ride through the history


Service worker sorcery

of tracing a bug in Service Work or with Service Workers


Service worker sorcery

and Googlebot.


Service worker sorcery

Thanks so, so much for being here with me today


Service worker sorcery

and telling the story.


Service worker sorcery

And for all of you out there, do let me know in the comments


Service worker sorcery

what was your weirdest experience


Service worker sorcery

and how did the developers respond to it when you were


Service worker sorcery

asking them about what's going on there.


Service worker sorcery

And leave us a like and subscribe to the channel


Service worker sorcery

to see upcoming episodes and other content that we put out.


Service worker sorcery

Stay safe, take care, and bye-bye.


Service worker sorcery

Not creepy.


Service worker sorcery

How dare you?


Service worker sorcery

There is Ariel.


Service worker sorcery


Do keyword meta tags matter for Google Search?

And now for a question that came in from multiple folks


independently.


We've answered this question in variations over the years,


but I thought it would be good to have a modern look at it.


The question is whether the keywords meta tag matters


for Google SEO.


Matt Cutts, one of the early engineers at Google,


answered this question back in 2009.


Just to be clear from the start, the answer is still no.


We do not use the contents of the keyword meta


tag in Google Search.


It's fine, however, for you to use the keywords meta tag


if you want to use them for something.


It doesn't cause any problems with Google Search.


It's usually small enough to not change the overall page size,


so there's no strong reason to remove them from a page.


To find out more about meta tags that we do use in Search,


we have an article for you in our documentation.


Most of the time, you'll be looking at the description


meta tag and perhaps the robots meta tag.


Technically, the title element is not a meta tag,


but it's often treated as one.


These are all elements that can have an effect on the search


results for your pages.


And again, Google Search does not use the keywords meta tag.


Googlebot won't even notice if you leave tasty cookies there.


Sorry.


And that was it for today.


If there's something on your mind,


I'd love to hear from you with the hashtag AskGooglebot.


Bye.


English Google Webmaster Central office-hours from August 7, 2020

All right, welcome, everyone, to today's Webmaster Central


Office Hours Hangout.


My name is John Mueller.


I'm a webmaster trends analyst at Google here in Switzerland.


And part of what we do are these Office Hour Hangouts,


where people can join in and ask their questions around


their website and web search.


Bunch of stuff was submitted on YouTube already.


But if any of you want to get started with the first question,


feel free to jump on in.


Can I just ask a question?


Sure.


If you remember, John, I asked you some Hangouts ago


about the problem of Avicon having disappeared


since some months ago.


And I was suspecting that the route not indexable


may be a cause because I moved to a structure


for multi-language.


And I wrote down the name of the website.


I don't know if you have time to have a look because nothing


changed.


Nothing changed?


OK.


I passed it on to the team to check out.


So I'm a bit surprised that nothing changed


because it sounded like from their side,


they would take a look.


I will push again.


Maybe we can get something.


Thank you very much.


Sure.


Another short question.


In the same site, I have pages that


have specification and reviews of products.


And there is one main page with the list, of course,


of all the products.


And I have separate pages that do not show all the products


but filter for category.


OK?


So I have one page filter just type A of the products


and other list in just type B of products.


And each of these pages have a canonical URL to themselves.


Of course, if you put other filters,


the canonical points to the basic product page filter


at the category.


And in Google search, one language is fine.


I mean, all these filter lists are indexed.


While in English, almost no pages are indexed.


And Google says that these are duplicated pages.


So the same page in another language


that are considered as duplicated.


And I don't know if there is anything I can look at.


So is it with different languages the same page?


Or is it with the filters, with the categories, and kind of?


Wait.


The main list, the overall list, is fine in both languages.


The filtered pages, so the pages that


show the list filtered by category,


are normally indexed in one language


and are not indexed in English.


And in English, Google search says


they are all duplicated pages with wrong canonical,


because the canonical points to themselves.


And instead, Google says that it should point to the main list.


OK, so they are considered as being duplicated.


I don't know if it's being clear.


OK.


I think that can make sense.


I mean, it's hard to say without looking at your site.


But I think that that can make sense.


So our systems, when they come across setups like this,


where there is a possibility for a lot of URLs to be generated,


and the URLs end up pointing at very similar content.


For example, if you have a category page,


and you can filter by color, and you can filter by size,


and all of these things, then the products that you show


are essentially kind of just the same things, different orders,


different selection of the same products, essentially.


And our systems try to recognize those kind of setups


and to figure out which parameters in the URL


are not important for the site.


And then they will kind of skip that and focus


on the canonical URLs, or the ones that they


think are canonicals instead.


So my guess is our systems looked at the setup


that you have there and thought that we can save you time


by focusing on the canonical URLs.


So one thing I would look at there


before trying to kind of fix this problem


is if it's really a problem.


So in particular, are these landing pages


that you want to have indexed, are they critical for indexing?


Is it something that is very visible


or would otherwise be very visible in search?


Is it the case that the product pages


themselves are still indexable?


So usually from the category page,


you link to the product page.


And the products are the ones that you usually would focus on.


If those products are all indexed normally,


and if the pages that we fold out as canonical


are not critical for your website,


then I would just leave that and kind of let


it be resolved like that.


On the other hand, if the products themselves


are not being indexed, and if the category pages that you're


trying to get indexed are really critical for your site,


then I would double check to make sure


that the URL structure that you're using


is one that does not allow Google to kind of run


into the situation where we start to ignore things.


So one example that I sometimes see


is that you have parameters in the URL that


can be swapped out for any text, and they still


lead to the same page.


For example, you have maybe product name


equals a long descriptive name, and then you


have the ID as well attached to the end.


And as long as the ID is there, then that page


shows a normal page.


And that's something where our systems would say, oh, well,


probably this product name is irrelevant,


and we can just pick one and focus on that.


Yeah, the URL does contain a variable,


because these pages are filtered by compatibility.


I mean, it's a list of accessories,


and the category list is the list of accessories


that are compatible with a specific product.


So I would be interested in having an index,


because if you search for accessory


compatible to a specific problem,


then you have that specific filtered page.


And so the idea is that I have to check


that if I put a random ID for compatibility,


I should return 404.


Or, yeah.


And you can also check in Search Console,


there is the URL parameter handling tool.


I think it's in the old part of Search Console, where


we show the URL parameters that we've noticed on your website


and the ones that we've started to ignore.


And you can change the settings there and say,


well, actually, this is one that I think is important for me,


and we'll pick that up.


OK, thanks.


Sure.


All right.


Any other question before we get started with this?


I've got one, if now's a good time, John.


Go for it.


Mine's a bit of a conundrum in terms of user experience.


So it has to do with Flash.


So we run a large digital magazines website,


lots of its user-generated content.


So we've got about 2 million Flash magazines


that we're essentially going to retire,


because by the end of the year, most browsers


don't recognize them all anyway.


And so we've got the choice now, purely


through our server setup, we've got the choice of,


do we remove all of that content?


And essentially, we'd love to 404 it,


but the way that we've got it set up,


because a lot of these are files, is we can 405 it,


which is a very ugly access denied page of death.


Or we have the option to directly replace the file,


give the user some kind of information,


i.e., this content is Flash, we've retired it now,


go back to the main page, and then canonicalize that content.


But obviously, we're talking about a scale of 2 million


pages that suddenly, overnight on our website,


become identical with the same message.


Just wondering from a user experience side,


but then also obviously from a crawling and indexing side,


what would be the best route to go down?


So in the long run, from an SEO point of view, both of these


would be equivalent.


So what would happen is the 405, we would recognize as an error.


We would drop those pages.


That's pretty fine.


If you serve a HTML page instead,


and the HTML page is the same content as other pages,


then what would happen is we would pick that up


as a soft 404.


And then over time, we would treat it similar to the page


not found.


The difference that you would see probably


from a practical point of view is


that we would continue crawling the HTML pages longer


before we kind of slow down there.


Because we were like, well, looks like a normal HTML page.


We'll double check a while.


And probably, we would just recrawl those more often.


But if you used to have PDF files there or Flash files


or whatever, then it's probably not the server load


that you're worried about.


So that's something where for a longer period of time,


we'll be recrawling that.


Practically speaking, we'll treat them the same.


So my recommendation in a case like that


would be to focus more on the user experience side


and say, well, I'll put an HTML page here.


And Google will treat it as a soft 404.


Soft 404 isn't great, but it's essentially what you want.


You want Google to figure out, OK, this page is gone.


Yeah, and that is essentially it.


I guess what we wanted to do from the user's perspective


is when they land, kind of give them more information.


Unfortunately, with our server setup,


you can't customize any kind of 405 messaging


to explain why this content's been removed.


So OK, cool.


Brilliant.


Thank you.


Cool.


Sure.


OK.


Cheers.


All right.


I think someone else had a question.


Yeah.


This is not an old topic that we've discussed


in the previous Hangout.


Basically, if you remember, we have this weather website,


and we have the issue that very weird canonical


are being selected.


And we've been going back and forth with the team about it


since basically end of June.


But still, really, topic couldn't be progressed a lot.


We are still in trial and error mode.


And so I was wondering if there's anything


that we can do to sort of speed this along,


because now is our high season.


And business is already pressuring me quite a lot


to not get this fixed, but to take some radical measures.


We are really talking about relaunching the whole site


on the different URL, because it's in such a broken state


that we are losing almost all of our relevant search traffic,


which is horrible for us.


Can you drop some sample URLs maybe in the chat here?


Then I can pick that up afterwards and double check


with the team.


I can also post the webmaster thread.


It already has almost 30 replies.


So we are back and forth a lot.


But it seems to be a difficult issue to fix.


So I appreciate that there is a lot of people trying


to fix it already.


But it's really business-wise, it's killer for us.


It's really important.


Sure.


I mean, if you can post maybe a link to a thread


or a link to the pages where you're


seeing this problem in the chat here,


then I can pick that up afterwards.


Oh, fantastic.


Cool.


Thanks.


All right.


Let me run through some of the submitted questions.


And we can get, or do you want to go, Chris?


Oh, yeah.


I'd rather ask you, actually.


It'll probably be easier.


We run quite a nice career advice platform.


And we've been building this for like 12 years now,


sorry, 8 to 10 years.


It previously allowed open submission of guest posting.


And people would come in, log in, and get


paid for actually providing content.


And we never really had much editorial moderation


in that process.


It was very minimal.


The last three, four years, we've stepped up the quality.


We're trying to follow Google's best practices, the Webmaster


guidelines.


And we seem to run into a little bit of an issue


on the older content and some bad outbound linking


in the past.


So we have a manual action now for outbound unnatural linking.


And during the submission process,


we tried to follow all the recommendations.


But it seems the tools we were using


didn't uncover all the links on the first audit.


And then the second audit, we found issues as well.


And we've learned really painfully over the last five


months now that maybe we should have taken more time


on the initial reconsideration.


And we are where we are now.


Now we're waiting three months, I think,


from the last response, which was a processed response.


And we had no note from the reviewer telling us


what to fix.


So we're kind of at a dead end here, wondering what to do.


Do we wait longer?


Or do we keep following up?


Because I've seen advice, don't follow up and do follow up.


And we don't want to aggravate you guys


and spam you with reconsiderations.


But I'm just wondering, what do you


recommend in such a long wait for five months here?


Yeah, that seems pretty long.


So I think, so in general, with reconsideration requests,


it's not something where you get kind of pushed forward


if you do another reconsideration request.


So just continuing doing that is probably not going to help.


But it sounds like something is stuck.


So if you want, maybe post your URL here in the chat.


Then I can take a look at that with the team


to see what might be happening there.


Would really appreciate that, yeah.


And the last response you got was


that it was being processed?


It said it was processed.


And the note said underneath that it's


either adjusted or revoked.


And if it remains in Google Search Console,


it means it's just adjusted.


So it still applies to the whole site, the manual action.


And we've seen our traffic deteriorate with COVID


since the spring, really.


So it's kind of we're in a desperate state


now to get something done.


I've even contacted a consultant.


And they said to try to reach out to you.


So I'll post you the link anyway now in the chat.


Yeah, if you can post it now, I can double-check quickly


to see kind of roughly what the status is.


It's not always the case that I can see exactly what's


happening, but maybe I'll get something.


Let's see, that's the correct link.


Let's see.


I don't know what the current status there is.


So I see we got a lot of duplicate reconsideration


requests from you, but one is still kind of open.


Yeah.


OK.


I adjusted the message somewhat, but I


wanted to give you all the evidence each time.


But then I read maybe I should have customized


the whole message each time and explained


exactly what was done.


And usually what happens is when we


get another reconsideration request for the same site


and one is still pending, we'll try


to process the earlier one first.


But it seems like that's from May,


and it's quite a bit past May now.


So I'll double check with the team, see if we can.


If you could push it along, that would


be amazing because we're just waiting here,


and it's just a bit frustrating.


And we're already struggling with COVID


and losing a bit of business and traffic as well.


So we really appreciate it.


So usually with these kinds of manual actions, what happens


is we just devalue the outbound links on your site.


Yeah.


Because we're not sure.


If we can't figure out which ones are actually good ones,


then we might fall into a safe mode type thing.


But that wouldn't affect the ranking of your website.


So if you're seeing things around the ranking


of your website differently now, then that


seems like something worth attacking separately.


I think it's good to get this resolved because then


your website is really in a clean state again.


But if you're seeing significant changes


in ranking and visibility in search,


that would probably be something different.


Well, we dropped, I think it was about 35%,


a couple of weeks after the initial notification came.


So that was when it was alarming that we've been hit penalized


from the manual action itself.


I think after we submitted it, the first response we sent


and we got rejected shortly after because of the issues


with our audit, we saw an immediate drop.


I think it was on Friday the 13th of April,


I believe, where we saw that dip.


But this COVID came at the same time as well.


So it was quite difficult for us to think


we've been either double hit by both factors, or it's Google,


or it's COVID.


We couldn't really differentiate.


And now I've got either the decision


was to even follow everything because we feel we've done,


we've now followed everything which is of any possibility


of being low quality.


So we don't really want to do that


because I wanted to ask you a follow-up question if I can.


Would you recommend not following all outbound links


on a site?


Because from my understanding of SEO,


this will lower the relevance of the platform.


I don't think it would lower the relevance,


but it's unnecessary.


I would try to link naturally within the web.


And that's essentially fine.


My general feeling here is really


if you're seeing a significant drop in search,


that would be unrelated to the manual action.


That would be something that would be separate.


So one thing I've sometimes seen,


I suspect it's not the case in your situation,


is that when a site has a lot of unnatural outbound links,


then sometimes it's part of a bigger network


where it's like everyone has all of these unnatural outbound


links and a lot of support of that website


was due to the unnatural outbound links.


And if the web spam team takes a look at that network


and kind of neutralizes the whole network, then of course,


those sites will also lose that support.


So that's where you might see a drop in visibility.


My guess, just I don't know, from talking with you


is that probably you're not a part of a crazy link network


where it's like the support to your site


has also been neutralized.


But what I would kind of focus on here


is try to kind of leave the unnatural outbound links


manual action kind of running.


I'll see if we can get that resolved on our side.


But I wouldn't assume that that will fix the visibility


issue of your site.


So I try to continue looking at other things


and try to find ways to really significantly take


things a step further.


SPEAKER 3 OK, thanks a lot.


Well, it's stabilized for now.


So I think it maybe was due to COVID.


So yeah, thanks for that.


Really helpful.


JOHN MUELLER Sure.


OK, let me go through some of the submitted questions.


And we'll have more time for other questions along the way.


Do you know how Google determines how and why they


show a photo of a product on one website's organic listing


versus another website that doesn't receive it?


For example, if you Google use Tonda Civic,


nearly half the organic results have


a photo next to the website description, but half don't.


When I look at the schema and images of those and those


without, they're virtually the same.


What am I missing?


So we do try to pick up an image for particular pages


if we can find one.


And we do try to show that in search.


However, we also need to make sure


that there is some kind of a reasonable balance with regards


to how many images that we do show in search.


So it can certainly happen that for some pages


we show an image, for other pages we don't.


Maybe for other queries, we would show an image


for those pages too.


To kind of make it more obvious to us


which image you want to have shown,


I would kind of go back to the normal structured data.


For example, with the article markup,


or I forget which other variations of the article


or web page markup that you can use there


where there is a primary image on the page


that you can specify.


And by doing that, you make it a little bit easier for us


to pick that up.


That said, even without this markup,


we can usually figure out what a relevant image for a page


might be.


And we can usually try to show that where appropriate.


A 301 redirected 500 article pages to a new URL structure.


The website has about 2,000 pages in total.


The pages started to get crawled and indexed


and recovered.


The traffic dropped the experience during those days.


But a few days later, due to a technical bug,


the URL switched back to the previous URL structure.


And traffic from those pages is almost gone.


Other pages and page types of the site


were not hit by this, only the articles.


The fix took place the same day, plus we


modified the canonicals and backlinks and internal links


as well to the new structure in the following days.


Is this mess with the bug and Google indexing the new URLs?


And suddenly, those are removed and back again after fix it.


Might that trigger a recrawl for the entire website,


which might mean it might take weeks to recover?


It is possible that this will trigger something


on our side where we try to recrawl the website overall.


Usually, the triggering of a recrawl on our side


essentially just means we try to get an updated picture


as quickly as possible.


And usually, that's due to us recognizing


when there are significant changes on a website.


So that could be, for example, you changed the whole URL


structure, or you move from one domain to another.


Then we try to get that reprocessed as quickly


as possible.


It's not the case that the website will stand still


or be dropped out of search during that time.


It's just we're trying to crawl a little bit faster


to catch up again.


With regards to this situation where you move URLs,


and then those new URLs disappear,


and then you add them back again,


it kind of depends a little bit on what happened during that


bug stage on your side.


If those new URLs disappeared completely,


and the redirects were still in place,


then it would have been like you were redirecting to a 404 page.


And in cases like that, we will think,


well, you removed these pages.


So we will drop them from the index.


So in those cases, it probably takes a little bit longer


than normal for us to pick that up again


and to index all of those pages again,


because we have to get back into when we look at your web page,


we have to consider again, well, actually, these pages


might not be gone completely.


We'll double check them again.


And we'll kind of get back into the normal crawling speed


for those individual pages.


On the other hand, if the redirect just dropped


and essentially it looks like it moved back to the old URL,


then those are cases which we can usually kind of cover


fairly well, because it just looks like, well,


you temporarily moved them here, and then you


moved them back again, and then you moved them here again.


For our systems, we don't really care.


I think it helps to make up your mind which URLs you want.


But essentially, this moving back and forth thing,


that's something that we should be able to deal with.


So that probably gets resolved fairly quickly.


But in either of these cases, even in the worst case scenario


where we thought you removed those pages


and we kind of crawling those URLs less frequently,


with a website of your size with 2,000 pages in total,


we can crawl a lot of content fairly quickly.


So that's something where I would suspect in the worst


case within a week or so, that will settle down again.


And things will be pretty much OK.


And at any rate, when it does settle down,


it's not that Google systems will think your website is bad


because you had this technical problem.


It's essentially just, well, we have these pages indexed now,


and we can rank them normally again.


There is no kind of bad feelings left behind


from a kind of a technical glitch like this.


So these technical issues happen to every site.


So you're not alone with that, and our systems


have to deal with that all the time.


I need to 301 redirect a page from a subdirectory


on a domain to a subdirectory on a new domain.


Do I need to have the new page indexed by Google


before making the redirect to it?


No.


You can redirect to a completely new URL.


That's usually the case how things are done.


It's like you take one URL, you redirect it to another one,


and then we recognize the new URL,


and we focus on the new URL.


So it doesn't need to be the case


that you have to first get that page indexed and then set up


to redirect.


It doesn't harm if you do that either, so whatever works.


An article of mine was ranking on the first page of Google.


There was a typo in the URL.


I did a 301 redirect to the correct URL,


and Google was still ranking the old URL.


So I requested re-indexing of that page,


and within five minutes of the request,


the page disappeared completely and couldn't be


found using the site operator.


After three days, the page got back in Google Search,


but now it's no more ranking and can only


be found with the site operator.


Can you tell me how long it will take to get back


into the original position?


I don't know.


So like you mentioned, I also don't think this is a penalty.


It's not like a black hat practice or anything.


This kind of thing can happen.


Usually, the cases where I have seen something


like this where a page disappears completely


is more due to the site owner using Search Console


to remove the old URL, and then we


run into a situation where maybe we make the old URL canonical,


but you told us to remove it so we won't show anything.


I don't know if that's something that happened here.


But in any case, it sometimes happens


that things fall out of our index, and usually over time,


they come back in and they start ranking normally again.


So this is something where I would


try to just give it a little bit more time to settle down again.


In Russia, we have two powerful search engines, Google


and another.


Can I separate the site on two domains?


Visit from Google sent to the first domain


and from the other one to the other domain?


The second domain is closed for Google and robots.txt


but open for other search engines.


The idea is to try to recover Google traffic


but not risk losing traffic from the other search engine.


Before moving, it was 52% from Google and 48% from another.


Will it work, any bans, because the two domains


with similar content but one is closed for Google


and Google Chrome users will have different behavior


on opening one domain?


So I don't know if this makes it any easier for you to do.


But at least from our side, if you


want to do something like this, that's perfectly fine.


If you prefer to block a part of your website or one


of your domains and say, Google should not index this domain,


that's perfectly fine.


That's your decision to make.


The thing to kind of keep in mind


is that there is more to your website than just your domain.


So in particular, links on the web


are things that we try to pick up that pretty much all website,


well, all search engines try to pick up.


And if you have two separate domains,


then you will have some links going to this domain


and some links going to that domain.


And if you block one domain for one search engine,


then that search engine will only see a part of the picture.


And they might assume that actually this website maybe


is not so great after all because only


the small part of the links are going there.


And that's something where potentially you're


making it a little bit harder for your website


than it needs to be.


So that's kind of what I would watch out for there.


The other thing is also with regards


to having two websites for two search engines,


you're just making everything much more complicated


from a technical point of view.


How do you deal with users?


How do you deal with users from different locations?


All of these things still kind of fall in here.


If you wanted to do maybe ads at some point,


how do you deal with that?


Do you run ads separately on the different domains?


How do you promote those websites?


If you want to publish something really fantastic


and you want to spread the word about that, how do you do that?


These are all things which, from my point of view,


make it a lot more complicated than it needs to be.


And I don't know which search engine you're referring to.


Well, I can guess.


But my general feeling is that most search engines kind of


look at similar things.


So it's not that if you have headings on one site,


that one search engine will say, oh, this is fantastic,


and the other one will say, oh, this is terrible.


But rather, search engines try to look at pages,


and they look at it in slightly different ways


and try to understand those pages in different ways.


But it's not that they would say,


this page that one search engine says is fantastic


is now terrible.


So they kind of have similar ways of looking at pages.


So from that point of view, I kind of


worry that you're making it a lot more complicated


than it needs to be, and you're probably finding a problem


where there is none.


When taking a mobile-friendly test for the same page


at a different time gap, it shows three variant results,


like partly loaded, not mobile-friendly page,


and sometimes showing mobile-friendly page


with 17 resources unloaded.


What is happening behind the crawling process?


Is that the JavaScript not loading properly,


or are the tracking codes making the page really slow


while rendering?


So this can sometimes happen.


What happens with the mobile-friendly test


in particular is we know users of the tests are impatient,


and we try to bring back the results as quickly as possible.


However, at the same time, we also


know that websites have limited capacity.


So we try not to overload their service.


So what can happen is if you have a very complex page that


uses a lot of embedded resources,


a lot of different things, maybe even from different locations,


different servers, then it can happen


that we're not able to completely load that page


in time to do the full test.


So what will then happen is we will load as much as we can,


where we're sure that we're not overloading any servers,


and we will try to run the test on that part of the page.


So if you run the test and you see


that there are different JavaScript files or images


missing, then that's generally a hint that that is happening.


And that's generally something where


I would say maybe wait again a little bit longer


and try that test again at some later point


to make sure that you're getting a real picture of how


that page is actually crawlable and renderable.


And if you're seeing that it never


is able to be processed properly,


then I would double check to see if there may be things


that you can simplify on your pages,


maybe combining different kinds of JavaScript,


maybe removing some of the different tracking setups


that you have on the page.


Maybe you have other things that significantly slow things down.


A good way to look at that is to look at the waterfall diagram


that you can see in Google Chrome


if you go to the developer tools.


I think it's in the network settings in the developer


tools.


What happens is you reload the page when you're there,


and then it shows how all of the different resources


are being loaded and when they're being loaded.


And usually, when you look at that


and you compare it to other pages that you have


or other pages on other people's sites,


you can kind of guess where you might


be going a little bit too far.


It's not that there is any specific limit on our site


with the number of resources or the time that it takes to load,


but it is something where you can look at some pages,


and maybe they have 100 or so resources.


And other pages, when you load them,


they have 10,000s of resources that


need to be loaded in order to render the page.


And then you can imagine that somewhere between there


is probably a reasonable range to aim for.


So if you have thousands of resources


that are needed to load a page, then I


could imagine that the mobile-friendly test


at some point was saying, well, this is like we're


running out of time.


We have to make a cut off here.


Another tool that you can use which gives you this waterfall


diagram in a more neutral way, in the sense


that it's not tied to your computer or your settings,


your internet, is webpagetest.org.


And there you can use different device types.


You can look at the screenshots.


You can see how long it takes to load


different parts of your page.


So that's kind of the direction that I would go there.


Let's see, second part of the question,


a page without an H1 title, will it


still rank for keywords, which is in the H2 title?


Of course.


I mean, will it still?


I don't know if it will still, but it can.


It can, absolutely.


So headings on a page help us to better understand


the content on the page.


Headings on the page are not the only ranking factor


that we have.


We look at the content on its own as well.


But sometimes having a clear heading on a page


gives us a little bit more information


on what that section is about.


So in particular, when it comes to images,


that's something where headings and the context of that image


helps us a lot to understand where


we should be showing that image in image search.


Because by design, images are not text.


We don't automatically know what we should be showing it for.


And that combination of the image plus the landing page


is something that depends quite a bit on the text of the page.


And when it comes to text on a page,


a heading is a really strong signal


telling us this part of the page is about this topic.


And whether you put that into an h1 tag or an h2 tag or h5


or whatever, that doesn't matter so much,


but rather kind of this general signal


that you give us that says, well, this part of the page


is about this topic.


And this other part of the page is maybe


about a different topic.


So that's generally what I would think about there.


Is it good to have structured data WordPress plugin


or rather do it manually?


Both approaches work fine.


I think from purely a practical point of view,


I would personally go with a plugin that does this for you.


Because then you can focus on the content.


You don't have to focus on the technical implementation.


If you're using something like WordPress,


you're already focusing more on the content


rather than how do I make an HTML page


or how do I serve my HTML page from the server.


You're already focusing more on the content, not


the technical implementation.


That said, in the end, if it's valid structured data,


we don't care where it comes from.


If it comes from a plugin, from your theme,


if you added it manually, if you use JavaScript to add it


after the page is loaded, all of those different types


of structured data are perfectly fine.


Is it possible that a website has


been penalized in its traffic for less than 20 minutes


of 503 service temporary unavailable


and returned to previous levels a week later?


No.


I don't think that would happen.


So usually with 503s, what happens


is we will not index the content that you serve us


when we see your 503 response code.


And we will assume that this is a temporary issue


until we recognize that it's a permanent issue.


So if you're talking about 20 minutes or maybe half a day


or a day or so of 503 response codes from your server


because it's down, because something is blocked


or something needs to happen, that's


definitely something we can deal with where we say, well,


we'll try again.


And essentially, during that time,


we will not change the indexing of those pages.


However, after a couple of days, when we still


see a 503 status code, we will assume


that this is not a temporary issue,


but maybe your server is down.


Or maybe these pages were removed,


and your server is set up in a way


that it serves a 503 instead of a 404 response code.


And what will happen then is that page by page,


we will drop those pages from our index.


But if you're talking about a time frame of 20 minutes,


that would definitely not happen.


And it would definitely not be the case


that we would demote that website in ranking


during a time when we're seeing 503s,


because 503s are a very common thing on the web.


And it's something that you should


be doing if your server is slightly overloaded.


Or if it can't deal with the responses,


then 503 is the correct approach there.


And that's not something that we would kind of penalize a website


or hold against it in any way.


After the 5th of May, our site lost their rich snippets


and was also removed from the recipe carousel.


For three months, we're looking for a reason with no luck.


We did tests, different URLs with LD plus JSON,


schema.org, minimum properties.


Previously, recipes were fully described.


And now we switch to schema microdata only.


Here's an example of a recipe.


An interesting thing, some of the recipe collections


which group together another type of URL and article


marked up only with the necessary schema.org recipe


properties, which didn't have them removed from the recipe


gallery.


Only removed only recipes grouped by URL recipe slash


asterisk.


I don't know.


This seems like a very specific question.


I'll take a look to see if there is something specific


happening here.


But generally speaking, if you had structured a rich


response, if you had rich results that were shown


for your website and they disappear from one day


to the next, then unless something specifically changed


in the requirements on our side for that type of structured


data, which can happen from time to time,


but we try to keep it really rare


and we try to let you know about it,


then that's more a sign that our algorithms re-evaluated


your website and we're not really sure about the quality


of your website.


So that's something where I wouldn't


focus so much on the technical implementation


of that structured data.


If you're sure that it was correct before,


if it passes the validators that we have,


then that's generally OK.


I would focus more on the overall quality of the website.


So I don't know this website.


It's hard for me to say that this is not a good high quality


website.


But that's kind of, from our point of view,


where we see these kind of changes happening,


where it's like, well, we kind of re-evaluated everything


around the website.


And we're like, ah, we're not really sure


if it meets the bar for what we would like


to show in the rich results.


And what can sometimes make this confusing


is that sometimes a website might be kind of on the edge


there, and we will re-evaluate that after some time as well.


And if during that time you're kind of fiddling


with the structured data and fiddling with it


in a way that kind of breaks the structured data,


then suddenly we'll be like, oh, we


would like to show rich results from this website.


But now the rich results are broken,


so we don't have anything to show anymore.