And at some point, it'll be that when we manually


English Google Webmaster Central office-hours from Sept 20, 2019

look at these pages, you can't tell


if it was written by a human or written


by some kind of advanced machine learning setup.


And at that point, does it really


matter if it was generated automatically or not?


From a user's point of view, it has the same value.


So with that in mind, I think at some point in the future,


we will have to revisit this guideline


and find a way to make it a little bit more granular


in that it kind of differentiates


between these totally spammy uses of auto-generated content


and the actually pretty useful uses of automatically generated


content.


So one example that I've seen before


is with regards to earthquakes, for example.


I don't know if this is actually still running,


but I believe at some point, this


was set up by some news site or some government agency


where if their sensors detected that a big earthquake took


place in a certain location, they would automatically


generate a page for that with some automatically generated


content around that, which would be something


that we could find in search.


So before any human reporter is able to actually sit down


and write something up and say, well,


this happened in this location, and it was this strong,


and likely had this effect.


And if you're in this region, you


should watch out for this and that.


That's something that could be generated automatically,


and that does have a lot of value to users as well.


So that kind of differentiation between the different types


of auto-generated content, I imagine


that's something that we'll see more and more coming up


as a topic over the years.


So I think, especially if you're looking at cases where


a site has 9,000 pages that are generated with a template,


and you look at those pages individually,


and you think, well, OK, they were generated by a template,


but they're still actually pretty useful,


then that's the kind of thing where I could imagine


that the web spam team at some point would say, well,


do we need to take action on this just because something


happened to make these pages available,


or do we need to take action on these pages based


on the content that they're actually providing


and the value that they are providing to users,


or that they're not providing to users?


So that, I think, will be an interesting thing to follow.


I think we're slightly at time.


So what I'll do now is just stop the recording.


You're welcome to stick around a little bit longer if you'd


like to continue chatting.


But I'll stop the recording so that it's


kind of a reasonable length.


Thank you all for joining, and I wish you all a great weekend.


And if you want to stick around, feel free to stick around.


And otherwise, feel free to jump on in in one


of the future Hangouts.


Bye.


SPEAKER 2 Thanks, John.


Have a nice weekend.


Google Search News (May ‘20) - COVID-19 Search Updates, Web Vitals, and more

Hi, everyone, and welcome back to the Google Search News


series.


I hope life is treating you reasonably well wherever


you are.


I'm your host today, John Mueller,


here from home in Switzerland.


With this show, we want to give you


a regular summary of what's been happening around Google


Search, specifically for webmasters, publishers,


and SEOs.


If you find these useful, which I hope you do,


and if you'd like to stay up to date,


make sure to subscribe to the channel.


Now that you're here, I miss you.


It's been a while since our last Google Search News.


The coronavirus has stirred things up a bit.


Everything's a bit more complicated and confusing.


Like many of you, we're currently all working from home.


On the one hand, this means my commute to my desk


is now extremely short.


On the other hand, this means our usual recording studio


setup is out of reach.


So to help keep you informed about Google Search,


we decided to start recording from home.


We're not sure when things will get better with regards


to recording.


So in the meantime, we'll do more videos like this


from home.


And now, on to the Google Search News.


Today, we'll take a look at COVID-19-related updates,


new speed metrics, mobile-first indexing, and more.


So let's check it out.


Many things have changed as coronavirus


started making its rounds.


For Google, this meant adding and modifying


some parts of Search, as well as helping


webmasters tackle the new challenges in various ways.


The details are linked below.


And here's a brief overview.


With in-person events being canceled,


we added support for postponed, canceled, and online events


with structured data.


Government and official health websites


can now show special announcements directly


in search results using structured data


and a separate Search Console tool.


As businesses temporarily closed,


we noticed that some sites were handling the closure badly.


So we created guidance for handling closures


in a search-friendly way.


For other sites, the issues were more with regards


to additional server load.


So we created some help for dealing with that.


And finally, to help official organizations


to make their information as accessible as quickly


as possible, we set up a separate help forum.


We know there will continue to be challenges that


arise as the world turns.


So I'm sure there will be more to come in this regard.


As a side note, the changes in events


also affect our events as well.


For the meantime, we've put our Webmaster Conference series


on hold and have started sharing some highlights


from individual sessions on YouTube in the Webmaster


Conference Lightning Talks playlist.


Time spent being at home feels like it's


passing both extremely quickly and extremely slowly.


What is time?


What day is it?


Well, I don't know either.


However, when it comes to the speed of web pages,


Chrome recently launched a new set of metrics


that we call the Core Web Vitals.


It provides a clear set of foundational metrics


for website developers to help build and improve


online experiences.


These will be introduced across a variety of tools,


like Lighthouse, Chrome DevTools, and PageSpeed Insights.


Google Search Console currently offers an experimental speed


report, which highlights the previous metrics used


across these tools.


With the announcement of Core Web Vitals,


the team is planning to update the speed report


to help site owners identify opportunities


to improve user experiences on their sites.


The Core Web Vitals includes the following metrics.


Largest Contentful Paint, or LCP in short,


this measures perceived load speed


and marks the point in the page load


timeline when a page's main content has likely loaded.


This metric is dependent on first contentful paint


and time to first byte, which you might have heard of before.


First Input Delay, or FID, this measures responsiveness


and quantifies the experience users


feel when trying to first interact with the page.


Communicative Layout Shift, or CLS,


which measures the visual stability


and quantifies the amount of unexpected layout


shift of visible page content.


Because the web continues to evolve,


we expect to update these on an annual basis


and to give you the tools you need to help


make your websites fantastic.


We'll keep you informed as time goes by.


We mentioned mobile first indexing in our last episode


as well.


Mobile first indexing is a process


of Google indexing the web using a crawler that


acts like a mobile device, like a smartphone.


This matches what users commonly see when they browse.


About 70% of all search results currently shown


are from mobile first indexing, and we expect this number


to rise significantly once some of the larger websites


are able to be moved over.


This is an amazing achievement that's


only possible due to a lot of work on websites,


making them work well for mobile devices.


We're continuing to move forward with mobile first indexing


as our systems recognize that sites are ready.


As a goal, earlier this year, we decided


to switch everything over to mobile first indexing


in September of this year.


We picked that date before coronavirus was widespread


here, so it's possible that this date might change.


Sites continue to improve with regards to mobile first


indexing, even in times of COVID-19, which is a great sign.


To make it easier for sites to get ready for mobile first


indexing, we added more documentation in our developer


docs.


If you're unsure of your website status with regards


to mobile first indexing, Google Search Console


will show you how we're primarily indexing your pages.


These are some pretty big changes,


but there is a lot more.


Here's a brief run through to help keep you up to date.


In the search developers documentation,


we added a page that lists all updates to the documentation.


This is not a show anymore.


No, wait, not that.


But it will help to keep you informed of all changes,


for example, in structured data elements.


We added documentation about licensable images,


if that's something that you have on your site.


JavaScript and structured data, how do they work together?


Check out more in the developer documentation.


Search Console now has a revamped removals report,


about review-rich snippets and updated site new features,


and some cool new user interface changes, such as better


exports.


There is always more happening around search


than we can cover here.


If you'd like to stay up to date,


I'd recommend subscribing to this channel,


checking in on our blog, and following us on Twitter.


That's all for now, folks.


Thank you for joining us, and thanks for your patience


as we navigate recording from home.


If you have any questions or comments,


feel free to add them right here or reach out to us on Twitter.


And now, on to the weather.


Let's be realistic.


It's not very useful to do a weather report,


because nobody is going outside anyway.


Let's hope that things get better soon.


In the meantime, mark your calendars for June 10


to join us for the next Webmaster Conference


Lightning Talk on rich results and Google Search Console.


Special files in robots.txt #AskGoogleWebmasters

Today, I'll be answering Anthony's question regarding


robots.txt.


Should I disallow asterisk.css, disallow php.ini,


or even disallow.htaccess?


Thanks.


No.


I can't disallow you from disallowing those files,


but that sounds like a bad idea.


You mentioned a few special cases, so let's take a look.


Asterisk.css would block all CSS files.


We need to be able to access CSS files so that we can properly


render your pages.


This is critical so that we can recognize when a page is


mobile-friendly, for example.


CSS files generally won't get indexed on their own,


but we need to be able to crawl them.


You also mentioned php.ini.


This is a configuration file for PHP.


In general, this file should be locked down


or in a special location so that nobody can access it.


And if nobody can access it, then that


includes Googlebot 2.


So again, no need to disallow crawling of that.


Finally, you mentioned.htaccess.


This is a special control file that can't be accessed


externally by default. Like other lockdown files,


you don't need to explicitly disallow it from crawling,


since it can't be accessed at all.


My recommendation is not to just reuse someone else's robot


sex file and assume it'll work.


Instead, think about which parts of your site


you really don't want to have crawled,


and just disallow crawling of those.


I hope that answers your question,


and stay tuned until the next episode of Ask Google


Webmasters.


Google Search News (Nov ‘21) - Title links, State of Technical SEO report, and more!

Hi, everyone, and welcome back to the Google Search News.


I hope life is treating you reasonably well wherever


you are.


I'm your host today, John Mueller,


here from Google Switzerland.


With this show, we want to give you


a regular summary of what's been happening around Google


Search, specifically for website owners, publishers, and SEOs.


If you find these useful, which I hope you do,


and if you'd like to stay up to date,


make sure to subscribe to the channel.


It's good to see you all again.


I have a bunch of updates for you today.


They include Core Web Vitals, Titles, Continuous Scrolling,


and more.


So without further ado, let's jump right in.


All right, we're back with more about the Core Web Vitals


and the Page Experience Ranking change.


I know, we've mentioned them a few times,


but there is an update I'd like to mention.


Just to recap, this is a relatively new ranking change


which takes into account the user experience of web pages.


It includes elements like HTTPS and aspects


around speed and responsiveness.


The measurements are based on what real users see, not


from lab tests.


The update is that we're going to expand this to also include


desktop search results.


Previously, it was only focused on mobile users and mobile


search results.


The change includes everything apart from mobile friendliness.


This means that speed, responsiveness, as well as


HTTPS and the intrusive interstitial guidelines


will be used for the ranking of search results shown


to desktop users.


The rollout will begin next year in February 2022.


We plan on expanding Search Console reports to help


site owners with this as well.


On this subject, the Chrome Developer Relations team


has been publishing fantastic articles about Core Web Vitals


on their website, web.dev.


Whether it's explorations of future metrics or tips


on improving complex websites, you'll find it all there.


So if you like technical details or want something


to pass on to your developers, check that out.


I'll add links to all items mentioned here


in the description below.


One neat thing we've been seeing about Core Web Vitals


is how content management systems and hosting


platforms are showing off their work on these numbers.


With the metrics, they can quantify


improvements they've been making for sites hosted with them.


As a site owner, it's great to use a popular platform where


other developers can dial up the speed for you.


And finally, I want to be clear that while we at Google


talk about this change often, it's


not the primary or most critical search ranking factor.


Having a fantastic website is so much more


than just having a fast website.


The relevance of your content is critical.


The page experience ranking factor exists,


and it's more than just a tiebreaker.


But as a site owner, you need to focus on the whole picture.


And now over to changes in the search results.


Over the last few months, we've made changes


in how the title link text is generated in our search


results.


Title links are the links to your pages which


we show in the search results.


Usually, they're similar to the page title elements


that you provide.


The initial change in August resulted in a lot of feedback


from site owners.


We heard loud and clear that it wasn't as good as we had hoped.


I'm sorry for the trouble.


Thanks to the great feedback in the help community,


the team was able to significantly improve


the algorithms and launched an update a short time afterwards.


Since this last update, the feedback


seems to have settled down.


So I think it's looking a lot better.


We have two blog posts to help explain this change


and updated documentation about title links and snippets.


For example, one of the changes made


was that the text shown no longer changes by query.


If you're curious, I recommend you check the documentation


for all of the details.


Feedback is always welcome.


If you see unexpected title links for your pages,


please let us know in the help community.


Thank you so much for all of the input given already.


And one more from the search results.


We recently started letting users


scroll through multiple pages of search results on mobile.


It's initially limited to certain regions


and may expand over time.


This change makes it easier for users to see more results


and to better select the pages they're interested in.


For website owners, this will likely


mean that you'll see an increased number of impressions


as it'll be easier for users to find your entry in the search


results.


The rise in impressions can sometimes


lead to a lower overall click-through rate


in your reports, though.


Keep this in mind if you're seeing unexpected changes


in your site's Search Console performance report.


Overall, I think this is a great change


and helps users to find more options in search.


I look forward to seeing it here in Swiss search results


at some point, too.


Just in, a new section landed in our search documentation.


Recently, we added a comprehensive guide


for e-commerce sites.


If you're selling anything online,


I strongly recommend taking a look.


While I wouldn't be making bigger changes just


before the holiday season, it can


be useful for longer-term ideas.


For example, we mentioned the kinds of structured data


that might be appropriate for shops, the interactions


with Google Merchant Center, ideas for URL structure,


and ways to launch a new website.


This is our first step into search-specific documentation


for e-commerce sites, and we'd love your input.


You can find the feedback link on all pages in our search


documentation, and the team reviews everything


that's submitted there.


One of the neat things about SEO is


that there's so many different tracks that


can lead to success.


Websites can focus on different things


and rank similarly in search, but there's


still all quite unique.


Because it's tempting to just focus


on what worked for you so far, it's


good to take a step back and see what others


are doing from time to time.


Earlier this year, Erij Abouali and Patty Moogan


ran a survey on technical SEO and recently published


the results.


It includes comments from experts on all of the topics.


If you're curious how other SEOs are organized


within their company, how they work, what they focus on,


it's worth checking that out.


I found it quite insightful.


I'll drop a link to it below.


And now for a handful of smaller updates.


We recently published an update to our Quality Raiders Guide.


This guide is used by quality raiders


when they review potential algorithm changes.


Because the web is ever evolving and users rightly


have high expectations from search engines,


we update the guide from time to time.


Quality raiders do not make the search results.


Instead, they give us valuable insight


into where we can improve our algorithms.


The guide is quite long, but interesting


if you'd like a peek into what our search quality


team thinks about.


On that subject, we currently have a position open


for a search quality analyst.


We're looking for someone with a flair


for analytical, quantitative, and technical problems


with a passion for search to join the team in California.


If you'd like to find out more or if you know someone


who would be a great fit, I'll add a link


to the description below.


And last but not least, if you work with local businesses,


take note.


Google My Business has rebranded and is now


called Google Business Profile.


Your business profile can now also


be edited directly in Google.


Just search for the name, and you should find it there.


And that's all for this time, folks.


Thank you for watching this episode.


It always feels good to me to look back and pull together


the things that recently stood out.


I hope you found it useful and insightful as well.


If there are elements you'd like to have covered differently,


please let me know in the comments below


or drop me a note on Twitter.


For more of these episodes, make sure to subscribe


to the channel.


We're regularly publishing new videos here,


and I'm sure there's something coming up


which you will enjoy as well.


And as always, I look forward to seeing you all again


in one of the future episodes of Google Search News.


Bye.


English Google SEO office-hours from February 18, 2022

All right, welcome, everyone, to today's Google Search Central


SEO Office Hours Hangout.


My name is John Mueller.


I'm a search advocate on the Search Relations team


here at Google in Switzerland.


And part of what we do are these Office Hours sessions,


where people can join in and ask their questions


around the website in the web search.


It looks like we have a bunch of questions submitted,


but we also have a ton of people raising their hands already.


So maybe we'll just jump right in and start


with the first folks.


Let's see, Nirav, I think you're on top of the list here.


NIRAV SHAHANI-KUMAR Hi, John.


Thanks for the opportunity.


My first question is, does Google Search Central


consider image packs, video packs, local listing


into the rankings, or those are just kept


and the URL are counted?


JOHN MUELLER We try to include everything that


includes a link to your site.


So if there is a block of images and one of those images


is from your site and it links to your site,


then that would count.


Or if there's a local listing that is shown


and there's a list to your website there,


NIRAV SHAHANI-KUMAR So if there are people also ask questions,


that is also considered into the ranking, right?


JOHN MUELLER If that link is shown,


then that would be considered, yes.


NIRAV SHAHANI-KUMAR OK, thank you so much.


My second question is, so this is like a case study.


We had a page for electric bikes and scooters.


OK, and later on, we understood that electric scooter is also


a very big search when it comes to volume.


And electric bike and electric scooter as a product


are two different things.


So we thought of creating two separate pages.


At first, we had just one page.


And now it's almost like six to eight months.


We used to rank well for electric bike and scooter,


which is in top five, when we were having just


the electric bike page.


But now, after having electric scooter and electric bike


page, still the electric bike page ranks,


and electric scooter doesn't rank.


We have tried a number of things,


like trying to separate the content of the two pages,


generate more and more internal links,


giving electric scooter links from electric bike page,


but still it doesn't show up in search.


So should we wait for some more time


so that Google can understand the intent?


Or will it be wise to merge those pages again?


That's more of a strategic question, almost,


because essentially, you can take one page


and you can split it up into multiple parts.


But that also means that each of those individual parts


will kind of have to battle it out on their own.


And sometimes it makes sense to have fewer pages


so that you can kind of have fewer really strong pages.


Sometimes it makes sense to have pages that are specifically


target on individual items.


And finding that balance is something


that is essentially up to you, and it's


more of a strategic question.


And I think the things that you're doing there


sound like the right things.


But whether or not this will lead


to kind of success for that specific query is hard to say.


And it might change over time.


OK, so we have already given three to four months.


So should we, will it be wise to wait for a few more months


so that Google can think about it?


It is more like a sufficient time to do this thing.


I can't say.


Like, if nothing changes on the web, then I wouldn't just wait.


But if you're continuously working


on improving these pages and kind


of their context and their content,


then maybe giving it more time makes sense.


But that's ultimately up to you.


Sure, sure.


Thank you so much, John.


Sure.


All right, Praveen.


Hi, John.


How are you?


Hi.


So my question is about product reviews update.


And this is basically for my own understanding


that I want to learn more about this thing.


And I just wanted to understand how


Google identifies whether a page or site is


related to product reviews.


So let me give you an example.


Like, there are case studies that tell you that there are


many affiliate sites that, because they


write about product reviews, pros and cons,


they compare products.


But for example, there is an e-commerce site


which is selling pens, fountain pen, ball pen,


that kind of stuff.


And they also have a blog where they review their own products.


They do write about pros and cons of their products,


compare different products.


So it's an e-commerce site, but the blog


is also talking about product reviews.


So can Google say that, OK, this is also


product reviews and can be analyzed by product reviews


update?


Product reviews can impact that content.


I mean, the recommendations we have for product reviews,


I think they would be relevant for any kind of product review.


So I wouldn't necessarily try to see,


like, does Google think my site is a product review site


or not?


And then I will do these good practices.


But rather, if you think these good practices would


apply to your content, then just do those good practices.


So that's kind of how I would see it, not like,


does Google think I fit into this category?


Therefore, I must follow these steps.


But rather, do these steps make sense for my content?


And if so, then just do that, yeah.


SPEAKER 2 OK, just a follow-up question on this.


The same e-commerce site, they have these product category


pages where they have listed all the products, like pens.


But it's a standard practice for most of the sites


that they create a lot of content on these category


pages, like 2,000, 3,000 words of content.


And that content also talks about product reviews.


They're just saying that these type of pens you can buy,


pros and cons.


It seems like a product review page more than a category page.


So I tend to get confused here.


It's a product review page or it's a category page


based on the content.


Usually, things like category pages


are pretty clear to understand.


But again, I would think more about whether or not


these recommendations make sense for that piece of content


or not.


And just because Google might get confused


that it could be a review because you


have a lot of products on it doesn't necessarily


mean that it makes sense to follow those steps.


But a lot of times, those recommendations


still make sense.


SPEAKER 1 OK.


OK.


Just one more question.


It's about indexing API.


So the Google Docs, it mentioned that the indexing API


should be used for pages like job posting or broadcasting


events.


Is it possible that we can try this API


for different type of content, like some news articles


or blog content?


That kind of thing.


I mean, people try it.


But essentially, that's like what we have documented


is what we use the API for.


So if you don't have content that


falls into those categories, then the API really


isn't going to help you there.


SPEAKER 2 I just want to understand.


If you give it a try, will it impact negatively in some way?


SPEAKER 1 No.


No.


But I mean, if you don't have job postings,


then there's nothing to do with that API.


SPEAKER 2 OK.


No problem.


Thank you.


SPEAKER 1 All right.


Isabel.


SPEAKER 3 Hello, John.


Hi.


Well, our question is about our main brand cable.


We were talking about this strain behavior two weeks ago.


It's about when we have a brand like InFace.


We are in the first position in the search engineer.


When we add our brand, Power Planet,


our website is filtered by Google.


We would like to know why Google filter our website when


the organic position is the first one when it's alone.


When we add our brand, it's filtered.


If you can maybe drop some of these examples into the chat,


then I can pass those onto our team.


I think I passed the domain, the name query thing


onto the team already, but I haven't turned anything back.


So if you can include some examples there,


that would be really useful.


SPEAKER 4 This is the brand when we looking for in Google,


InFace.


We are in the face in Spain.


We are in the first place.


And when you add InFace, Power Planet,


our result is filtered.


This behavior is in a lot of brands,


even in products and categories.


And we don't know why we make a lot of changes in the website


in order to improve it.


The behavior continues.


OK, and this is specific to Spain,


or are you seeing it also in other countries?


SPEAKER 5 In Spain, it's the most relevant.


But in some cases, the behavior is in Portugal


or in other countries.


But in Spain, it's the most relevant


and happens in the almost brand.


OK.


I'll check back with the team to see what they found so far.


Thank you.


Thanks.


All right, Vahid.


Hi, John.


Have a good time.


Hi.


I have some questions about EAT.


In quality raters guidelines, the author expertise


are important.


So do you think it's also important for real algorithms?


How do you mean?


I mean, EAT is just mentioned in quality raters guidelines.


But I want to know if real algorithms also


care about EAT factor like author expertise.


I would assume that there is some indirect kind of work


done to try to do similar things, yes.


I mean, we put this in the guidelines


so that we can kind of guide the quality testers


to double check these things.


And if we think that it's something important,


then I would assume that folks on the quality, search quality


side also work to try to understand


that in a more algorithmic way.


But I wouldn't see there's like an EAT score


and you have to get five or something like that on it.


It's more kind of like trying to understand


the context of the content on the web.


And that's a very, I don't know, fuzzy area.


OK, thank you.


In some article, I see people are speaking


about on-link brand mention.


I want to know your opinion in this case.


Do you think it's also important for algorithms


on-link brand mention?


How do you mean brand mentions?


It's like another website in article


speaking about my website brand but doesn't link to me.


I don't know.


I think that's kind of tricky because we don't really


know what the context is there.


So I mean, I don't think it's a bad thing just for users


because if they can find your website through that mention,


then that's always a good thing.


But I wouldn't assume that there's


like some SEO factor that is trying


to figure out where someone is mentioning your website.


OK, thank you.


And what about user reviews or comments?


Do you think it's also a ranking factor for an article


or product?


Like comments on the bottom of an article?


Yeah.


So what I think is really useful there with those comments


is that oftentimes people will write about the page


in their own words.


And that gives us a little bit more information


on how we can show this page in the search results.


So from that point of view, I think


comments are a good thing on a page.


Obviously, finding a way to maintain them


in a reasonable way is sometimes tricky


because people also spam those comments


and all kinds of crazy stuff happens there.


But overall, I think if you can find a way


to maintain comments on a web page, that gives you


a little bit more context and helps


people who are searching in different ways


to also find your content.


Thank you so much.


And for the last question, do you


think different type of SSL is also important for SEO?


No.


Just free SSL or not?


Free SSL is perfectly fine.


So the different types of certificates


is more a matter of kind of like, I don't know,


what you want to do with a certificate.


But from our point of view, we just watch out for,


is this a valid certificate or not?


And all of these certificates are valid.


OK.


Sure.


I'll say.


All right.


Wow.


I think.


Hi, John.


It's Vova.


Yes.


Great.


Thank you.


So I have a question regarding multiple job posting


on different domains with job posting structured data.


As I have understood in the guidelines,


it's not so good to post multiple times structured data


on the same page, let's say.


So as an international job provider,


we have a corporate structure with multiple brand websites,


but also multiple subdomains for one brand.


The question is, is it problematic


if the same job postings are posted on the same root domain,


but different subdomains with structured data


from the job posting?


So let's say one of the same job is one on jobs.domain.de,


and the other one is on www.domain.de.


It's the same job, maybe different styling,


but same structured data on both subdomains is posted.


Is there any problem with that?


I don't know all of the details around job postings,


but I would assume that's perfectly fine,


because it's very common to have the same job hosted


on different websites, and those different websites could have


structured data on them as well.


So from that point of view, just having


it hosted different times on the same website


or in different subdomains, that should be perfectly fine


as well.


I don't know the details of all of the guidelines


around job postings, though.


So it might be that there is some obscure mention


that you should only list at once on each website,


but I would be surprised if that were a problem.


OK.


So do you think it will affect Google for Jobs,


and it would appear two times on Google for Jobs,


one for the one subdomain, one for the second subdomain,


right?


I don't know.


I don't know how they would trip,


because usually we do try to dedupe different listings,


and we do that for all kinds of listings.


So if it's an image or if it's a web page or anything else,


where if we can recognize that it's the same primary content,


we will try to just show it once.


I would assume that we do the same thing for Google Jobs.


I don't know the details there, though.


Thank you.


Sure.


All right.


Corinna.


Yeah, hi, John.


I have a question regarding internal duplicate content.


So I have the content of a PDF file, a case study.


I submit it to my website.


Now I want to present it as well in a HTML blog article.


Does this have any negative impact


for my site because of duplicate content?


So we wouldn't see it as duplicate content


because it's different content.


It's like one is an HTML page, one is a PDF.


Even if the primary piece of content on there is the same,


the whole thing around it is different.


So from that level, we wouldn't see it as duplicate content.


I think at most the difficulty might


be that in the search results, it


can happen that both of these show up at the same time.


And whether or not you want that to happen,


that's more almost like a strategic question on your side.


So from my point of view, I wouldn't see it as a negative


when it comes to SEO.


But maybe you have strategic reasons


to have either the PDF or the HTML page more visible.


So would then they compete against each other?


They could, sure.


I think for the most part, PDFs will probably


be less visible just because they're


less tied in with the rest of your website


in that in your internal linking,


you'll link to the web pages.


And then from one of those web pages, you'll link to the PDF.


So they'll be a little bit kind of de-emphasized there


from internal linking.


But they could appear in the same search results.


And they could kind of compete with each other there.


That would be bad, right?


Well, I mean, it depends on what you want.


Because users will see that it's a PDF in the search results.


And if they want a PDF, then maybe that's


the best choice for them.


But that's ultimately up to you.


So if I want the HTML to be indexed,


I would need to set a canonical on the PDF.


Yes.


You can.


I believe you can set the canonical on the PDF


with HTML headers.


You can definitely also use noindex


in the headers for PDF files.


OK.


Yeah, that's it.


Thanks so much.


Cool.


Thanks.


All right, Nick.


Hey, John.


Thanks for having us.


My question, my main one is centered around the idea


of paginated content.


So if I have, say, a long discussion thread with maybe


100 or more comments, it's probably


intuitive to split it over multiple pages


so the length of the initial page


isn't too long for people to scroll.


So the question is, let's say a new comment


is posted towards the end of the discussion thread.


It gets added onto the end, which


could appear, say, on page four or page five or beyond,


because it's the newest comment.


And then across all pages of the discussion thread,


the date it was updated will reflect


the most recent activity.


However, the most recent activity


does not appear until page four or page five, for example.


I'm just trying to get an idea of the best way


or whether Google understands to crawl in through each page


or whether the most recent comment needs


to be featured more prominently.


Any recommendation along those lines?


I think that's ultimately up to you.


So that's something where I would


try to think about which of these comments


you want to prioritize.


I assume if something is on page four,


then we would have to crawl page one, two, three first


to find that.


And usually, that would mean that it's


kind of like a longer way from the main part of the website.


And from our point of view, what would probably happen there


is we would not give it that much weight,


and we would probably not recrawl that page as often.


So that's something where if you're saying, well,


if it's on page four, then it's probably not that critical,


then that would be kind of how we would see it as well.


Whereas if you say the newest comment should


be the most visible ones, then maybe it


makes sense to kind of reverse that order


and show them differently.


Because if the newest comments are right on the main page,


then it's a lot easier for us to recrawl that more often


and to give it a little bit more weight in the search results.


So that's kind of, I guess, up to you,


how you want to balance that.


Sure.


And that's true, because some of the messages are timely,


and we would want Google to know, oh, this


is the newest content being added


onto this series of pages.


Am I correct in understanding?


Remember those old tags link rel equals next and previous?


Those were depreciated, correct?


Yes.


OK, so that doesn't even matter.


And then for a canonical URL, I already


have several views of a thread, for instance.


It could be sort of the default sort is conversation order.


But there's also sorting the newest messages first,


as you kind of alluded to, but that's not the default.


And then there's also sorting by the top voted,


like in order of what was voted up the most.


And however, in each case, the canonical


is always on the default sort, because I'm always


concerned about duplicate content across pages


when it's simply viewing the same content in different ways.


Does Google place a lot of weight on the canonical,


or is there still a chance Google might crawl it


in those other sort orders?


I would assume that we sometimes crawl it in the other sort


orders, but we would probably use


the canonical for the most part.


Because for canonicalization, we take into account


different factors, and that includes rel canonical.


It also includes things like internal linking.


And I assume the internal linking will also


go to that default sort order version.


So probably most of those signals


align already on the default sort order version.


So for the most part, I would assume we just pick that one


and stick with it, but I could see our systems trying out


the different other versions from time to time as well,


just to make sure that there's nothing that we're missing.


Because we don't know what canonical


might be specified on the other sort order pages,


so we might try those out.


I assume for indexing, we will probably


use the default one, and for crawling,


you would probably still see some activity on the index.


OK, that makes sense.


Thank you.


And then the last quick question,


do you know if Googlebot is advanced enough


to handle infinite scrolling yet, or at least something


where content keeps building on itself?


A little bit.


But so I don't know.


I thought we had a page on that, but I


think we might not actually have finished that up.


But essentially what happens when we render a page


is we use a fairly high viewport,


like if you have a really long screen,


and we render the page to see what the page would show there.


And usually that would trigger some amount


of infinite scrolling in whatever JavaScript methods


that you're using to trigger the infinite scrolling.


And whatever ends up being loaded there,


that would be what we would be able to index.


So that's something where depending


on how you implement infinite scroll,


it can happen that we have this longer page in the index.


It might not be that we have everything


that would fit into that page, because depending


on how you trigger infinite scroll, it might be that,


oh, you're just loading the next page kind of thing.


And then we might have two or three


of these pages loaded on one page with infinite scroll,


but not everything.


So that's something where I would recommend


testing that with the inspection tool


and just seeing how much Google would actually pick up.


SPEAKER 2


Cool, thanks.


All right, Stefan.


SPEAKER 3


I have an issue with, I explained it


in the comments on the YouTube page already,


where we have a website which has been


ranking for several years.


It's a translated version of the main website.


We took it offline for about five months


and then revived it.


And ever since, the new website has


shown in Google Search Console to have around 40k


valid crawled pages.


Yet, if you look into simple search for Google


and you type in the site name, it only


shows like four to six pages.


And if you go to the internal links page of Search Console,


it shows the root page.


It shows that the root page has 40k internal references.


And then it says there are no other pages.


So the question is, are we doing something wrong,


or is Google doing something wrong?


I don't know.


It sounds like one of those things


where I'd have to look at the specifics.


I don't know if you added maybe the domain name


or some of the details of the pages that you have indexed


there.


Not yet.


But I could put it in the chat or in the posting


here in the chat.


If you can put it here in the chat,


then I can copy that out.


This is the domain where it is about.


It has a sister domain in Dutch, which is ranking fine.


But the German one is just strange, very strange.


And it was offline for a couple of months?


Let me see, the main site has been online since 2005.


The translated German version was online from 2010


till the end of 2020.


And it has been offline from December 2020 till June 2021,


serving 404s.


Yeah.


OK.


It feels like it should be catching up in the meantime.


But I can pass that on to the team


to double check if there's anything on our side.


It has been like this from August till now.


So it is showing only one valid page


with 40k internal links without other pages.


So it's kind of strange.


I'll take a look.


Thanks.


Cool.


All right.


Let's maybe jump to some of the submitted questions.


And I'll get back to all of the many people


with your raised hands as well towards the end.


And I have a bit more time afterwards as well


to kind of extend if we have need for more questions.


Let's see.


Is there a good ratio or consideration on Google's side


when it comes to amount of pages?


In other words, are positions of high traffic ranking pages


hurt by many, let's say 50% of total pages on a domain


not being indexed or being indexed but not


receiving traffic?


From our point of view, there is no specific ratio


that we would call out for.


How many pages a website should have


or how many indexable pages a website should have,


that's ultimately really up to you.


What I tend to see is that fewer pages tend


to perform better in the sense that if you


can concentrate the value of your content on fewer pages,


then in general, those few pages tend


to be a lot stronger than if you were


to kind of dilute the content across a lot


of different pages.


So that's something where usually I


would recommend having fewer pages rather than having


more pages.


And that kind of plays across the board in the sense


that from a ranking point of view,


I can give these pages a little bit more away


from a crawling point of view.


It's easier for us to keep up with these.


So especially if you're starting off with a new website,


I would recommend starting off small,


focusing on something specific that you want to achieve,


and then expanding from there and not just going in


and saying, well, I have 500,000 pages


and I want Google to index them all.


Because especially for a new website,


when you start off in such a big number of pages,


then chances are we will just pick up


a small sample of those pages.


And whether or not that small sample


is the pages that you care about most, that's kind of


questionable.


Let's see.


When using URL inspection in Search Console


on established highly visible pages,


how concerning is it that the referring pages listed


are long retired microsite domains?


That would not bother me at all.


So in particular, the referring page in the inspection tool


is kind of where we first saw the mention of your pages.


And if we first saw them on some random website,


then that's just where we saw them.


And that's kind of what we list there.


It doesn't mean that there's anything bad with those pages.


So from our point of view, that's


purely a technical thing.


It's like, well, here's where we found it.


It's not a sign that you need to make sure


that it was first found on some very important part


of your website.


So if your pages are indexed, I think


that's the important part there.


I wouldn't really worry about where they were first found.


I usually use the referring page when you run across pages


that you think, why did Google even find this page?


Or where did this come from?


If there are weird URL parameters in there,


or if there's something really weird in the URL where you're


saying, well, Google should never have found


this page in the first place, then looking at the referring


URL is something that helps you to figure out,


where did this actually come from?


Is this something that I did?


Or maybe some random person on the internet


dropped a broken link to my website somehow,


and that got picked up.


That can help you a little bit to figure out,


is it something you need to fix, or is it just


the way the internet is?


Our CMS platform is now automatically


generating the sitemap file with a different file name


than before.


We 301 redirected the old sitemap to the new one.


We will, keeping the 301 redirect,


permanently cause any issues with crawling or indexing.


The redirect is fine.


I think that's a good practice to have.


What I would also do is just make sure


that the new URL of the sitemap file is directly submitted


in Search Console, or at least in the robots.txt file,


so that we can go directly to the sitemap file


and that we don't have to go through the redirect.


I don't think that changes anything significantly.


It just makes it a little bit cleaner


in that we can go directly to the sitemap file


and get all of those URLs.


It's not going to change your site's ranking.


It just makes it a little bit easier, a little bit cleaner.


Let's see.


Question about the indexing API and e-commerce sites.


I think we touched upon that before.


The indexing API is really just for job postings and these live


events.


In the Search Console report, 97% of the crawl requests


are refresh, and only 3% is discovery.


How to optimize this and let Google discover more pages?


We don't really have any guidelines


on how that balance should be and how


you should try to tweak it.


In general, it's kind of normal for, especially,


an older, more established website


to have a lot of refresh crawl.


Because we will look at the amount of pages


that we know about that grows over time,


and the amount of new pages that comes in


tends to be fairly stable.


So it's pretty common, especially for a website that


is established and just slowly growing,


to have a balance like this that most of the crawling


is on the refresh crawling and not so much on the discover


crawling.


I think it would be different if you had a very short-lived


website, maybe, I don't know, classified


or local news where you have a lot of new articles that


come in, and the old content becomes irrelevant very


quickly.


Then I think we would tend to focus more on discovery.


But especially if you have something like an e-commerce


site where you're just growing the amount of content


that you have slowly, and most of the old content


remains valid, I tend to see that the amount of refresh


crawling there, that's probably going to be a bit higher.


During the last few weeks, I've noticed a huge drop


in crawl stats from 700 to 50 per day.


Is there a way to understand from the Search Console report


what could be the cause of this drop?


Could it be source page load?


How can I correctly read the crawl request breakdown?


So on our side, there are a few things


that go into the amount of crawling that we do.


On the one hand, we try to figure out


how much we need to crawl from a website


to keep things fresh and useful in our search results.


And that kind of relies on understanding


the quality of your website, how things change on your website.


We call that the crawl demand.


And on the other hand, there's the limitations


that we see from your server, from your website,


from your network infrastructure with regards


to how much we can crawl on a website.


And we try to balance those two.


And the restrictions tend to be tied to two main things.


On the one hand, the overall response time


to requests to the website.


And on the other hand, the number


of errors, specifically server errors


that we see during crawling.


So if we see a lot of server errors,


then we will slow down crawling because we


don't want to cause more problems.


If we see that your server is getting slower,


then we will also slow down crawling


because, again, we don't want to cause


any problems with the crawling.


So those are kind of the two main things


that come into play there.


The difficulty, I think, with the speed aspect


is that we have two essentially different ways


of looking at speed.


And sometimes that gets confusing


when you look at the crawl rate.


So specifically for the crawl rate,


we just look at how quickly can we request


a URL from your server.


And the other aspect of speed that you've probably run into


is everything around Core Web Vitals


and how quickly a page loads in a browser.


And the speed that it takes in a browser


tends not to be related directly to the speed


that it takes for us to fetch an individual URL on a website


because in a browser, you have to process the JavaScript,


pull in all of these external files, render the content,


kind of recalculate the positions of all


of the elements on a page.


And that takes a different amount of time


than just fetching that URL.


So that's kind of one thing to watch out for.


If you're trying to diagnose a change in crawl rate,


then don't look at how long it takes for a page to render


and instead look at just purely how long it takes to fetch


that URL from the server.


The other thing that comes in here as well


is that from time to time, we, or, well,


depending on what you do, we try to understand


where the website is actually hosted in the sense


that if we recognize that a website is changing hosting


from one server to a different server, that


could be to a different hosting provider, that


could be moving to a CDN or changing CDNs,


anything like that, then our systems


will automatically go back to some safe rate


where we know that we're not going to cause any problems


and then step by step increase again.


So anytime you make a bigger change on your website's


hosting, then I would assume that the crawl rate will drop.


And then over the next couple of weeks,


it'll go back up to whatever we think we can safely


crawl on a website.


And that might be something that you're seeing here.


The other thing is that from time to time,


our algorithms to determine how we classify websites


and servers, they can update as well.


So it can certainly happen that at some point,


even if you don't change anything with your hosting


infrastructure, that our algorithms will try to figure


out, well, oh, actually, this website


is hosted on this server.


And this server is one that is frequently overloaded.


So we should be more cautious with crawling this website


so that we don't cause any problems.


And that's something that also settles down automatically


over time, usually over a couple of weeks.


And if that were the case, then probably things


will settle down and get back into a reasonable state again.


The other thing that you can do in Search Console,


you can specify a crawl rate.


I believe it's in the setting per site.


And that helps us to understand that you


have specific settings specifically for your website.


And we'll try to take that into account.


The difficulty with a crawl rate setting


is that it's a maximum setting.


It's not a sign that we should crawl as much as that,


but rather that we should crawl at most what


you specify there.


So usually, that setting is more useful for times


when you need to reduce the amount of crawling,


not when you want to increase the amount of crawling.


And finally, there's so many things that come into this.


Finally, one thing that you can also do


is in the Help Center for Search Console,


we have a link to reporting problems with Googlebot.


And if you notice that the crawling of your website


is way out of range for what you would expect it to be,


then you can report problems with Googlebot


through that link.


What you need to do there is specify


some of the IP addresses of Googlebot


when it tries to crawl your page and give some information


on the type of issue that you're seeing.


And that could be that Googlebot is crawling a lot less


than it could, or it could be that Googlebot


is crawling way too much, or Googlebot


is crawling all of these totally irrelevant URLs,


and they make totally no sense.


And all of these reports go to the engineering team


that works on Googlebot, and they tend to go through those


and try to figure out what they need to tweak on their side


to improve the crawl.


For the most part, you won't get a reply to the request


that you send there, but they all get read by the Googlebot


team, and they do try to either figure out,


do we need to do something specific for this one site,


or is this perhaps something that we can improve


for these systems overall?


So many things about crawling.


Let's see.


We recently added Web Stories to my website,


which is focused on marketing a destination.


I put the name of the country, Jordan, in the title.


The good news is I got thousands of impressions on them.


The bad news is that it drove my overall click-through rate down


to 0.5%.


I am worried that this low click-through rate


is going to affect my search results for the whole website.


Any advice?


It will not affect the search results for your website.


So that's kind of like the first thing to be really clear about.


It's not going to be that some pages have a very low


click-through rate, therefore we will rank this website lower.


That's not how it works.


So from that point of view, I would look at these Web Stories


if they work well for you.


And if they work well for you, what you're trying to achieve


with your website, I would keep them.


If the Web Stories themselves don't work well for you,


then maybe tweak them or find a different format to use there.


But I would not worry that the performance of the Web Stories


somehow relates to the ranking of the rest of your content,


because that's definitely not the case.


I think with Web Stories themselves,


doing SEO for them is sometimes tricky,


because there's very little text on them.


So if you're getting impressions for them,


then you're already doing some really good work.


But in general, these are essentially HTML pages.


They're presented in a slightly different way in the search


results and in Discover.


But it's not going to be that those Web Stories will


negatively affect the rest of your website ranking.


I see you also have your hand raised.


So maybe I'll just go over to you


if there's anything more that you'd


like to add to the question.


I think you're on mute.


Here we go.


Hi.


Thank you for reassuring me.


I was really worried about that, because I've


seen things that tell me that if the click-through rate


for your site is low, that you'll end up actually


in a negative reinforcement loop where you keep


dropping in the search results.


No.


That's definitely not the case.


Yeah.


And then the problem, though, is with SEO,


because there aren't any plugins or any way


to determine a keyword for a Web Story.


Do you think this is something that will happen?


I mean, it really would help me a lot.


Um, I don't know.


So we include them in Search Console


and as kind of normal pages as well.


So there's a little bit of information there.


But I do think it's challenging just


because there's so little textual content on these Web


Stories.


And from a search ranking point of view,


we almost need to figure out a little bit better


how we can recognize this small amount of information


in a Web Story and rank them appropriately.


Because they're very often really nice to look at.


They give a lot of information to people.


But they just don't have a lot of text in them.


Right.


Well, this is the strange thing.


I mean, I'm a very niche market doing one little camp


in the desert.


And I'm ranking six for Jordan.


That's cool.


I don't know how this happened.


But it's not actually helping me at all.


Or not yet.


Yeah.


I mean, one of the things you can,


or what I would also recommend doing,


is making sure that you embed the Web Stories well


within your website so that there are links from the Web


Stories to the rest of your website too.


I'm afraid to do that or all the competitors will do the same.


Well, I mean, the advantage there


is if people go to your Web Story,


then they can still move on to the rest of your website.


And that's kind of what you want them to do.


OK, thank you so much.


Cool.


Good luck.


Thanks.


Let me see.


Some more questions here.


How quickly does Google consider the effect


of internal linking and rankings in terms of link juice?


Is it recommended to highlight new content with channeling


link juice to it?


Wow.


I don't know.


Link juice is always one of those terms


where people have very conflicting feelings about,


because it's not really how our systems look at it.


With regards to internal linking,


I do think this is one of the most important elements


of a website, because it's a great way for you


to tell us what you would consider


important on your pages.


And especially, like most websites


have a home page that is seen as the most important part


of the website.


And especially links that you can


provide from those important pages to other pages


that you think are important, that's really useful for us.


And it can be that these are temporary links too.


For example, if you have an e-commerce site


and you link to your new products from your home page,


then that's a really fast way for us


to recognize those new products and to crawl and index them


as quickly as possible and to give them


a little bit of extra starting weight when you're starting off


with them.


But of course, if you remove those links,


then that kind of internal connection is gone as well.


And with regards to how quickly that is picked up,


I would assume that's essentially picked up


immediately as soon as we recrawl and reindex


those pages.


So it's not that there's any kind of an extra latency


that we would add where we would say,


well, this internal link was just now added.


Therefore, we'll look at it in a week.


From our point of view, if we see an internal link


and we kind of can follow it, then we'll


go off and follow it as soon as we can.


The next question I have in the list is super long.


And we're kind of low on time.


So maybe I'll just jump back to some of the live questions


here.


So many people still with raised hands.


Let's see, Tafur.


TAFUR OZLINGOBA-RODRIGUEZ-MARTINIYA Hi, John.


I have two short questions.


So basically, the first one is about a smaller market


that doesn't have so much competition.


So it's not United States, right?


So we're making a lot of upgrades on our site,


like adding frequently asked questions.


We're getting good organic links on our pages.


And we have more content than our competition.


But we're firmly placed on the fifth position.


And we can't move now for months.


Basically, my question is, do you


have any kind of advice that what we can do


or what we can try to get our ranking up?


I don't have anything specific to say,


like in general, if you're in position five


and you're doing a lot of things,


you should also do this.


When it comes to search and rankings,


there's just so many different things


that come into play there.


So it's not really like there's this one trick


to get past position five.


Essentially, it's a function of all of the competition


that you have there, all of the other signals


that we've collected for your site, for your content.


So it's almost like keep doing good things


is kind of the best advice.


Yeah, because some results in front of us


are just translations of, say, English sites, which


is automatic.


It's not even content that is deliberately made.


Yeah.


Yeah, so the second question is, we


started a new page about two months ago.


And we have pages that are discovered but not indexed.


So my question is just, how long can we


expect for that to last?


And yeah.


That can be forever.


It's something where we just don't


crawl and index all pages.


And it's completely normal for any website


that we don't have everything indexed.


And especially with a newer website,


if you have a lot of content, then I


would assume that it's expected that a lot of the new content


for a while will be discovered and not indexed.


And then over time, usually it kind of shifts over


and is like, well, it's actually crawled or it's actually indexed


when we see that there's actually


value in kind of focusing more on the website itself.


But it's not guaranteed.


So from that point of view, it's not


that I would say you should just wait a little bit


and then suddenly things will get better


with crawling and indexing.


It's more that as I continue working on the website


and making sure that our systems recognize


that there's value in crawling and indexing more.


And then over time, we will crawl and index more.


OK, thank you very much.


Cool.


Hazel.


Hey, John.


Good morning.


So I've got one question about the subdirectory way


to view the different countries.


So we are thinking about view different English version


pages for different countries in a way of subdirectory.


As we researched, we found that some competitors,


they are doing the subdirectory in this way.


Like their category page is like www.example.com


with subdirectory like Canada-English version


and with their category page.


And both their page content and products URL


on this category page all starts with that country subdirectory.


So we are thinking about doing it in this way.


But as we discussed with our tech team,


they said it requires maybe more infrastructure.


In infrastructure, so maybe we think


if we can only set the display URL in subdirectory,


but the product URLs on the category page


is the exact same at the English site.


Will it work?


Do you think it's a good way that we can give a try or?


How do you mean?


I don't understand the workaround


that you're looking at.


So we are thinking about our category page


is starts with subdirectory.


But the product URL on the subdirectory page,


like the internal links, still the English site


without subdirectory.


Yeah, in this way.


Oh, OK.


So the category page would be per country,


but the actual products themselves


would be the same URL for all countries?


Yeah, exactly.


Do you think it will work?


It may be easier for Googlebot?


I don't know.


I think it's suboptimal because it's almost


like you're creating half of an international setup


and not the full international setup.


And what I would try to look at there


is whether or not the traffic from individual countries


to those category pages is sufficient enough for you


to make this kind of a change in the sense


that if the category pages themselves are not


super visible in the search results, then probably


by adding the international aspect just to the category


pages themselves, I kind of worry that the effect that you


would see from internationalization


would be so small that it's not worth the effort.


So I think it's hard to say in general for any website.


It's almost more something where what you could do


is in Search Console, look at, kind of filter


for the category pages if they have a specific URL pattern


and see how much traffic they get from individual countries.


And then based on that, try to consider


what would change if the traffic from maybe the top three


countries went to individual country versions


and just for those category pages to try to figure out.


I could imagine it will be visible in the search results,


but is it visible enough to be worth the effort


that you put into it?


Because it could very well be that you


put a lot of work into this, and the change in the category


pages for visibility internationally is minimal.


And then you decide, oh, I don't want to do anything.


For internationalization, because it's


such a small effect.


So I don't know.


From my point of view, I would be


worried that you put a lot of work into it


and don't actually get a lot of new value out of it


if you just take the category pages


and not think about what the primary pages are


for the website.


And it might be that the category pages are actually


the primary pages, and putting the internationalization work


into that is actually the most important part.


But I think that's something you need to figure out first.


AUDIENCE 1 Sure.


Sure.


We'll take your advice into consideration.


So as for this geotargeting, besides using Halfline,


how does Google figure out what geotargeting


to use for this specific website or this specific subdirectory?


We try to group URLs by clear patterns that we can recognize.


And that's, for example, by subdomain or by subdirectory.


So if you have the country in the subdirectory in a higher


place in the path, then it's a lot easier for us


to say everything under this path is for this country.


Everything under this other path is for another country.


And you can also verify individual paths


in Search Console and specifically say,


this path is for this country or this path


is for another country, which makes


it a little bit easier for us.


In practice, I don't hear a lot of feedback from people


saying that this makes a big difference.


So I don't know if it's actually something you need to do,


especially if it's a more complex setup.


But I would try to make it so that it's


as clear as possible which country is


relevant for the individual URLs with a clear path in the URL.


I think there was a question someone submitted as well


about using the country as a URL parameter at the end.


Theoretically, you can do that.


I think for our systems, it makes it a lot harder


to recognize which URLs belong to which country.


So I would see that as being less likely for us


to pick up for geotargeting.


Obviously, if you're using hreflang,


then that's less of an issue there


because you can do that on a per-URL basis.


SPEAKER 2 OK.


I see.


Thank you.


So another two questions is about crawling.


So we are a huge e-commerce site.


And as we checked our core report,


and we found that there are a huge amount of URLs


in the discovered but unindexed that tab.


And about twice than the index pages.


So is this an addiction of our site problem?


And how can we identify the problem and solve it?


Because we think it's a huge issue.


There are so many index pages.


It should be index pages, but not index.


And we also checked many times for our basic technical points,


like robots.txt and maybe mental.txt in code source.


We didn't find anything wrong with that.


So yeah.


Yeah, I think it depends on what those pages are


and how you use them within your website.


So in particular, we find URLs of all kinds of URLs


across the web.


And a lot of those URLs don't need to be crawled and indexed


because maybe they're just variations of URLs


we already know.


Or maybe they're just some random forum or a scraper


script that has copied URLs from your website


and included them and are broken away.


And we find all of these links all the time on the web.


So it's very normal to have a lot of these URLs


that are either crawled and not indexed or discovered


and not crawled just because there are so


many different sources of URLs across the web.


So what I would do is, first of all,


try to maybe download a list of a sample of those


so that you can look at individual examples


and try to classify which of those URLs


are actually ones that you care about


and which of these are ones that you can ignore.


And anything that looks kind of really weird as a URL,


I would just ignore it.


I don't think you need to do anything specific for that


unless you can tell that this URL was


found within your website.


So that's kind of the main classifications


that I would take there.


And the ones that you do care about,


that's something where I would try to figure out


what you could do to better tie these in your website


with regards to things like internal linking.


So if these are individual products or categories that


are just not being found, try to figure out


what you can do in a systematic way


to make sure that all of these URLs are better linked


between each other.


Because especially with a larger e-commerce site,


it can get tricky because you kind of can't look at every URL


individually all the time.


But sometimes there are kind of like tricks


that you can do where you say, well, anything that


is first level category, I link to it from my home page.


And I make sure that my first level category has at most,


I don't know, maybe 100 items or 200 items


so that you have kind of like a little bit of a forcing


function in terms of what you give Google to crawl and index.


And based on that, you can kind of like build it out


a little bit more systematically.


My guess is that you're probably doing a lot of this right.


And it's just that with a very large website,


it's kind of normal that we just can't pick up everything.


So to some extent, I would kind of just accept that as, well,


Google can't crawl and index everything.


One of the things you can do to help mitigate that a little bit


is if you recognize, for example,


that these are individual products that are not


being crawled and indexed, then make sure


that at least the category page for those products


is crawled and indexed.


Because that way, people can still


find some content for those individual products


on your website.


So it's not that people are not finding


your website for those queries.


It's more they're going to the category page


rather than directly to the product page, which


to a large extent is also fine.


I see.


So yeah, we'll try a few separate methods.


And yes, as we checked a few samples in our GSC backend,


most pages are our product pages.


And we also have some category pages that are unindexed.


So basically, these pages are the most we care about.


And we want them to index.


I think there is no URL like some unimportant in the GSC


backend.


The Google discovered these pages.


They all discovered these important pages,


but they just don't index.


Yeah.


So one thing I would also try, which I don't know,


perhaps you're already doing, is to see


if you can crawl your website yourself so that you


have a little bit more direct data of how a website like yours


can be crawled.


And there are various crawling tools out there


for very large e-commerce sites.


I think it's tricky just because there are so many URLs.


But maybe there is a way for you to limit it


to one segment of your website.


And by crawling the website yourself,


you can also see which of these URLs


are linked very far away from the home page


and which of these are linked closer to the home page.


And based on that, sometimes you can tweak the site structure


a bit to make sure that things are reasonably close


or reasonably stable with regards to the distance


from the home page.


Oh, I see.


That's another method that we didn't like,


yeah, we didn't think of.


So based on this situation, do you


think if we continue to add new product pages


and add new category pages to our website,


will these new pages be ranked further back in the Google


query?


Like, they will rank at a later date?


Yeah.


It depends on how you implement them or integrate them


within the rest of your website.


So just because something is new doesn't


mean we will crawl it less frequently.


It's more we try to figure out, like,


is this prominently linked within your website?


So is the internal linking kind of telling us


this is important or not?


And if it's important, then we will crawl and index it,


even if it's a new page.


So just because something is new doesn't mean


that we won't crawl and index it.


So it's more like how you put it into your internal linking


that matters.


Oh, so like if we view new pages like in the Google crawl


query, maybe they are not at the later position


than this existing non-index.


Yeah.


Go to here.


We'll try the methods you said.


Thank you.


Cool.


All right.


I am going to pause the recording here.


I still have more time.


It looks like we have millions of people raising their hands.


So we will try to see how many we can get through.


Thank you for sticking around for so long.


If you're watching this on YouTube,


you're welcome to join us live in one of the future episodes.


I tend to post the link for these a couple of days


ahead of time in the community section of our channel.


So feel free to jump in there if you're


keen on asking questions in person.


Otherwise, thanks again for all of the questions


that were submitted.


Thanks for all of the live questions as well.


Google Images: Best Practices for Search (WMConf MTV '19)

Hi, good morning. Welcome everyone. My name is Francois Cepice. I'm a product manager on Google Images.


So I want to start a little bit different than the other presentations, give you a quick overview of the actual product.


There's some slight nuances to Google Images versus web search that will then lead into the product changes that we've made recently


and then talk a bit about sort of optimizing your page within that context.


So virtually, our users come to image search not just to find an image.


They actually come to get things done in the real world and it's just a way for the brain to foster process the information


when they're doing a search. So not exactly sure what they want.


Images scrolling, so the infinite scroll is a very good way to find something. Users come for many different use cases.


So shopping or to redesign a bedroom,


inspiration, get some work done. So there's lots of different use cases and we've already changed the product to reflect these use cases.


So take an example.


Someone's looking for a kid's tire swing. What you see on the right there is what the product looked like about two years ago.


It was just images. There was a lot of context.


And so if you want to try and get something done, we're just giving you a bunch of images. It's hard for the user to navigate.


So what we did was a couple of changes to the product. Firstly,


we added sort of text snippets in the domain below the images.


So let's say that now I want to buy a kid's tire swing. I want to make one.


Maybe I want an image of one. Maybe I just want to see how to use one.


There's all these different ways.


It's much easier for the user to navigate. In the back end what we also did was we changed from


ranking not just the images, so like okay, these are the most relevant beautiful images for the user,


but like really what are the best landing pages given what they want to do?


So we're fundamentally changing the way that we rank the results and


to help users really find the great content behind the image. It's not about the pixels, but what's the page behind that?


We've introduced structured metadata, so products, recipes, videos.


So if you click through to the viewer, what we call the images viewer, the second click,


we will then surface that information to the user. And so in general, it's much easier for the user to get things done.


So there's some product changes in line with that that we made in the last three or four months.


I'll just walk through in case anyone was curious why we did that.


So firstly on the desktop, we hadn't really redesigned it in about five years and we introduced a couple of new things.


Firstly, we moved the viewer to the side panel.


It used to be when you clicked on an image, the image took over the whole of the screen.


And so the focus of the user was really on the image and the pixels.


We've moved it to the side. So what did that allow us to do? It moves much more clear context for the user.


It's not just about the image, but it's what the content is behind it. You know, we're also surfacing the metadata


and all the published information now for the user.


And so this we've seen is much easier for users to get things done and you know,


they're clicking through more to the publisher sites.


Similarly to the mobile view, we've also modernized that. We made some aesthetic changes that just made it more clear.


Okay, here's the image. Here is the website where it appears. Here's the context where it appears.


And we've made a more prominent visit button. Again, it's just sort of helping our users to indicate to them,


you're trying to get something done. There's a great page behind this image.


Here's all the ways that you can go there. And again, we've seen a significant increase in interactions with pages downstream from the image.


And then recently for AMP pages, we call swipe to visit, which is if your page is AMP'd.


There's now a much lower friction method for users to actually get to the web page.


So there's a little bar, tray bar at the bottom. You can swipe up and very easily go to the web page.


And that way, you know, the user can go back. And so this has seen an increase in people going to this and seeing those web pages,


as well as we've seen a slight increase in AMP adoption for this feature.


So that was a very quick breeze through some of the product changes.


Let me just get to some of the optimization best practices. I just have sort of three here.


Firstly, yes, we do support structured data. Today, specifically reps, recipes, videos, and products.


Each of our viewers will have a slightly different treatment based on this. So this is a second click experience.


And it's really great for users to be able to understand what's the great content behind the image on the web page.


So we support them for those three categories today.


So in general, it's good to use descriptive titles and sort of captions and file names to give us a bit more context.


What is this image actually about? We use some of the information such as, you know, the descriptions and titles that we surface to the users.


And we've made all these changes where the user is much more clear to them.


These are the pixels, but this is the context which it appears. This is the page title.


And so we can tell you users are coming to use Image Search not to find images but to find web pages.


So you can help them by optimizing through these different means.


So that really helped us to surface the right image for the right query for the users.


And then the last thing is just to use high quality images.


We want to show our users not just the relevant images but also beautiful inspirational content.


And so using that next to text, you know, not just an image by itself but within that context, we're trying to help the user to get things done.


And then, you know, as a standard practice user, mobile friendly pages, all of that will lead to better results.


And so lastly, I'm sure you all know this, but just to reiterate, in Search Console you're able to check what traffic you're receiving from Image Search specifically.


There's a way for you to filter that and so people see the sort of standard metrics you see on web search.


So if you aren't doing that today, we're not optimizing your pages for Image Search, I encourage you to do so.


English Google SEO office-hours from March 12, 2021

Hi, everyone, and welcome to today's Google Search Central


SEO Office Hours.


My name is John Mueller.


I'm a search advocate on the Google Search Relations team.


And part of what we do are these Office Hour Hangouts,


where people can join in and ask their questions all around


search and their websites.


So many people joining us today.


It's fantastic.


Cool to see a bunch of new names and faces as well.


So that's always neat.


And a bunch of things were submitted on YouTube as well.


But as always, maybe we can get started


with some first questions.


And I think what probably works easiest


is if you're live in here, just use the Raise Hand feature.


And then I'll try to go through in that order.


And wow, OK, lots of people raising hands already.


OK, Praveen, maybe you can get started with your question.


Hey, hi, John, how are you?


Hi.


So it's a bit weird and a long question, so please don't mind.


So this website we have, which is a global website,


and they have pages in multiple languages,


in English and Japanese and Korean.


So this website has implemented HRE offline tags


and all that stuff.


And they're doing pretty good in Google


in all the local regions.


And the problem is that in South Korea,


there is a local search engine called Naver.


And Naver has problem with these HRE offline tags.


It doesn't understand them.


And this website has these languages.


They have used directory system, folder system


for multiple languages.


They are not using subdomains or separate domains.


And this makes it even harder for Naver


to understand that this website is in multiple languages.


So what it does is it shows English version


to the Korean audience for Korean queries.


And that's a problem.


The management had decided, why not


we should make a separate website for Korean audience


in only Korean language.


And the concern from Google's point of view


is that if we allow that website to be indexed by Google too,


can that new website impact performance


of the existing global website?


Is there a possibility that because Google can figure out


that both these websites belong to the same domain


and they are basically targeting the same audience.


But that new website won't be just copy-paste.


It will be the website with new user interface,


new fresh content, and everything will be new.


The only concern is, can the new website


impact performance of the existing website?


Because that existing website is doing pretty good on Google.


I think if it's a completely separate website,


then it's a completely separate website.


So that's no concern from that point of view.


If it's just on a different domain


and it's essentially a part of the same set of pages,


then you can still use hreflang between


those different domains.


So hreflang doesn't have to be just within one domain.


So that might be another option where you could say,


well, they don't understand hreflang,


but if it's on a different domain, it works for them.


And Google understands hreflang and can do it between domains


as well.


Then you can combine those two options.


SPEAKER 2 We actually don't want to confuse Google with,


we have so many pages on the system


and then one is on the subdomains.


So we just want to keep things simple.


So the only concern is the new website


should not impact the performance of the existing


website.


I think that's clear.


Yeah, I think what will happen is


you will compete with yourself, with your existing website.


And if you're OK with that, that's fine.


But it sometimes makes it a bit harder


to rank well for some queries because both of these versions


will have some amount of signals and value in Google's eyes.


And that means maybe both of them


will be kind of mid-range with regards to ranking.


But if these are queries where your site is very visible


anyway, then probably it doesn't make a big difference.


But essentially, that's up to you.


It's not that the web spam team will look at this and say,


oh, two websites is too much.


SPEAKER 2 Sure.


Thank you.


That's it.


Sure.


Xiao Chen.


Oh, I cannot pronounce your name.


Sorry.


SPEAKER 3 Thank you.


I also typed my question here.


So my question is, a while ago I was reading one


of Google's guide.


They say that do not index internal search results


because you generally not search user intent well.


I agree with that.


It is an informational site because people


want the landing on the page that directly


enter your question.


However, I'm not quite sure it's also


the same for e-commerce site.


For example, when you are searching for, for example,


slim computer, and the first result


is actually Amazon's internal search result page.


And as a customer, I won't be happy seeing that


because it did provide a lot of options for me to buy.


And so my question is just that for very transactional intent


keyword, e-commerce site is also not


recommended to index internal search page?


So I think that's something where


you could argue that these are more like category pages


or where you could say, well, it's more like a category page


and it's indexable.


The tricky part with internal search pages


is that it's often an infinite space


in that you can enter any random word


and it'll create a web page for it.


And that's something you want to avoid.


But if there are specific topics where


you say this is an important topic,


then you can definitely keep that indexed.


It's not that the web spam team is reviewing these and seeing,


oh, internal search, it should be spam kind of thing.


It's more a best practice for crawling and indexing


that you provide pages that are useful for users.


And if these are internal search pages that


are more like category pages, that's fine.


I see.


Thank you so much.


So would you recommend only pick some of those internal search


page to index while block the majority of them?


Yeah, I think if you can do that, that's what I would do.


Like, take a list of keywords where you're saying,


these are OK.


And everything else, just block those.


Just to avoid the situation that people create


random pages for random words.


Or also what I've sometimes seen is


people create pages for words that you don't want to rank for.


Like, I don't know, maybe they're


searching for medical terms and your website


is about computer products.


Then you don't want to rank for that content.


Sure.


David.


I think you're muted or your microphone's not working.


Yes, no, I was muted.


Sorry.


Thank you.


OK.


I have a few questions, but I think they're quick.


So I'll try to get through them.


Number one is, if enhancing Core Web Vitals,


let's say, reduces the, we get rid of a bunch of old scripts


and reduces the page size in half,


would that correlate with an increase of crawl rate budget?


Maybe.


So there are two things there.


On the one hand, if we can access your HTML pages


faster, we can probably crawl more.


So that would kind of go into the crawl budget


side of things.


The other part is, if we can render your pages faster,


if there are JavaScript pages, then we


can also get through them faster.


But it all depends on the demand as well.


So for crawl budget, we have kind


of the capacity of the website and the demand from our side.


Got it.


Now, the other question is a little more difficult.


Basically, there's the amount of pages


that are being indexed by Google in the coverage report.


And then if I go to the internal link report in the link


section, I see that all the links that


are in the header and footer are the top ones


with the same number, pretty much.


So I know that it's collecting those as internal links


to report on, regardless of its effect or impact.


But the number is a tiny fraction


of the total indexed pages.


And when I do a site search in Google,


it's somewhere around the range of the total index pages


in the coverage report.


But then I go to the internal link section, and it's,


I'm talking about indexed is 6 million,


and in the internal links is 30,000


for all of those header footer pages.


And those are on every one of those pages.


So what would that hint to mean?


Nothing really useful, probably.


It's just for these reports, we take a sample


of the pages from your site.


So they're not meant to be kind of on the same level


as the indexed coverage report.


Got it.


So it's just to get an idea of which internal links are


found more often than others.


Exactly.


OK.


And then the next thing is she mentioned it


about the URL parameters for search pages.


If you're dealing with a site that the search is set up


that it autocorrects and redirects to a proper search


that we want that's in our site map that's relevant,


is that a redirect that will hurt,


like, in terms of crawl budget?


Because it could be anything.


Someone could input anything, even


though it will find the right page probably


the most relevant on the site.


Is it going to impact anything?


And how would be the best way to implement?


How are you redirecting there?


I would have to speak to the dev team, but I don't know.


They have their own internal link system,


like algorithm and autocorrect.


So usually with redirects, we follow,


I think, a small number of redirects immediately.


And it's kind of seen as a part of the same request.


And I think the limit is something like five, something


like that, where if there are five redirects in a row,


we'll just follow that and look at the final content


and treat that as one request.


If it takes more than five steps,


then we would treat that as separate requests.


But usually for something like this, you would have one step.


And you'd be kind of on the category page


or something like that.


And that's perfectly fine.


OK.


All right.


Thank you.


Evan.


Yeah, so I think I was a little faster on the call,


on the hands up button.


But my colleague, Derek, could probably


explain the question more succinctly than I can.


Derek, if you want to go ahead.


Sure, can you hear me?


Yes.


Great.


So our question has to do with Core Web Vitals.


I'll post it here in the chat.


Basically, we want to know how it relates to AMP.


So we have enhanced our site for AMP.


And 90% of our mobile traffic goes to AMP pages.


But now we're addressing Core Web Vitals stuff.


But in our Core Web Vitals dashboard,


looks like only 50% of our pages are hitting good marks there


for mobile.


So we spend a lot of time doing AMP.


Are we going to be penalized for the pages that we have that


are not AMP that are performing poorly on the Web Vitals side?


I don't think you would be penalized for that


in that sense.


But what happens on our side with regards to the Core Web


Vitals is we use the Chrome User Experience report data


as kind of the baseline.


And we try to segment the site into parts


what we can recognize there from the Chrome User Experience


report, which I think is also reflected in the Search Console


report.


And based on those sections, when it comes to specific URLs,


we will try to find the most appropriate group


to fit that into.


So if someone is searching, for example,


for AMP pages within your site, and you have a separate AMP


section, or you have a separate AMP template


that we can recognize, then essentially, we'll


be using those metrics for those pages.


It's not so much that if there are


some slow pages on your site, then the whole site


will be kind of seen as slow.


It's really for that group of pages


where we can assume that when a user goes to that page,


their experience will be kind of similar.


We will treat that as one group.


So if those are good, then essentially,


pages that fall into those groups are good.


The tricky part, I think, with the AMP report


is that we show the pages there, but we don't really


include information on how much traffic


goes to each of those parts.


So it might be that you have, let's say,


100 pages that are listed there, and your primary traffic


goes to 10 of those pages.


Then it could look like you have all of these 90% are slow,


perhaps if those 90 other pages are slow,


and those 10 that people go to are fast.


But actually, the majority of your people


will be going to those fast pages,


and we'll take that into account.


So two follow-up questions there.


One, you mentioned that you segment these results


into different sections, right?


So when I look in the Search Console, you're right.


I can see that they're grouped together,


but I see the AMP and the non-AMP pages separately.


So were you taken to, if you do a search result


and I have maybe a site that's not AMP canonical,


but you're showing an AMP result for,


it will use the results from the AMP section of that page,


even if the search result itself is for the non-AMP version,


but it has the AMP enhancement?


Sorry, does that question make sense?


JOHN MUELLER-HENNEKENSEN-SANTOSIOSKIS


I just kind of got confused with that last part.


It's like, it's an AMP page, but we're showing the non-AMP page?


Or, yeah.


Well, so there's an AMP and a non-AMP version, right?


And in the search result, you're showing the AMP version,


but it's not AMP canonical.


So is my, if that makes more sense.


Yeah, I think what, I kind of have to look


at the specific situation.


My understanding is we would follow the canonical


and use it based on that.


OK, so if it's not AMP.


I mean, usually what would happen is we would fold the AMP


information, the AMP signals, into the canonical itself.


Got it.


OK.


And then I guess the second follow-up question is,


does the AMP results in the Chrome User Experience


reports take into account the benefits of the AMP caching?


Because I can imagine just like going to the AMP page


would be much slower than viewing the cached version


from the.


I mean, what we do is we take into account what users see.


So if it's a valid AMP page and we can serve it from the cache,


then we'll use that.


Whereas if it's a kind of not a valid AMP page


and it's still an AMP page and pretty fast,


but we can't serve it from the cache,


then obviously we don't use that.


Yeah, totally.


I was just asking because the Chrome User Experience


report I know takes from just people browsing the internet


from Chrome.


So I don't know.


I wasn't sure if I was able to associate the cached AMP


pages with those origins.


So that was my question.


Thanks.


Cool.


Fantastic.


Max.


Hi, John.


Another AMP-related question.


If we're changing AMP URLs, do we


need to redirect from the old one to the new?


So the canonical URLs are staying the same.


We're just changing the structure of the AMP URLs.


Ideally, yes.


In practice, it's not as critical


as with normal URLs on a website.


Because usually what happens with the AMP URLs


is we refresh the cache fairly quickly.


And we will notice that the canonical URL and the cache


page and those URLs change fairly quickly.


So it's more something where I'd say


you have that time period between when


we know about the URL, when we show it in the search,


and when we've updated it, which might be, I don't know,


a couple days' time.


So it's definitely less critical than with the normal pages.


But ideally, you would still redirect those as well.


OK.


And I have a second question related to AMP again.


So I think it's been stated a few times


that there isn't a direct ranking boost, a ranking


factor from AMP.


However, last year, we removed our AMP pages.


We redirected AMP URLs to our canonical pages.


And then we saw a decrease in traffic straightaway.


And over six days, it was a 10% decrease in organic traffic.


So we redeployed it.


We turned it back on because we didn't


want to keep losing traffic.


Is there an obvious reason why that might happen?


I don't know.


It kind of depends on the site.


So on the one hand, I think there


might be an effect of things just kind of like still


pointing at the AMP pages and kind of getting lost