page experience as well as many other foundational pieces to an outstanding, fantastic, modern


How to improve Cumulative Layout Shift for a better page experience

website.


Search Console, Page Experience, and Core Web Vitals all have the same source for your


page's CLS result, Crux or the Chrome User Experience report.


You can read all about what Crux is in the links below.


Unfortunately, Crux doesn't update instantly.


It can take up to a month after you publish your page for it to start getting field data.


But you don't need to wait that long to get an idea of how the page you're currently working


on will perform with CLS.


Using tools like Lighthouse available in the links below, as well as directly in your Chrome


DevTools, you can quickly get the same field data we have in Search Console for any page,


as well as generate lab data.


Lab data is a best guess effort as far as what field data will likely look like.


It's based around average mobile devices and average mobile internet speeds.


Keep in mind, however, that lab data is a guess.


It can be helpful to point you in the right direction, but field data is the only value


that page experience, and therefore search, is going to actually look at.


If your users have high-end devices or unusually slow connections, you are more likely to see


a larger difference between lab and field data values.


Lab data can be really helpful, but once you actually publish your page, I think it's a


bit more useful to actually collect CLS data via our own analytics.


Just like the other Core Web Vitals we discussed in the previous episodes, we can measure CLS


with our own JavaScript using a performance observer.


This is a relatively new browser API that lets us subscribe to events that happen in


the browser related to performance.


We don't have the ability to directly observe the cumulative layout shift value, but we


do have all the right pieces we need to figure it out for ourselves.


In this case, we're going to be subscribed to the layout shift performance event.


Now every time the browser has a layout shift event, our code gets called.


We can use all the information here to figure out what the CLS is going to look like for


this user.


Once our function is called, it gets an entry list of every layout shift that has been observed.


We can call get entries on this entry list to get an array of entries so that we can


iterate over each individual one.


From there, we can check if an entry had a recent input.


It's a handy property provided by the browser on every layout shift entry.


If had recent input is true, then we are in that first 500 millisecond grace window and


can just ignore this entry.


But if has recent input is false, then we can grab entry.value and add that to our CLS.


Now you may have noticed that this counter just keeps getting added to.


Didn't we talk about session windows earlier?


Well you legalized Stram and you're right.


In fact, this is how CLS was originally calculated.


It's the sum of the entire session of a page.


But based on feedback and continued research, CLS was evolved to use the session window


methodology we discussed earlier.


You can read more about it in the links below.


So if you wanted to track the CLS value that matches the same logic that exists today,


we can update our code to track it with just a few more variables.


In the interest of time, there's a link to the updated version of that code in the description


below, if you're interested in seeing how the specifics of the tracking session window


would work.


Now that we know how to collect the CLS value, we still need to trigger an analytics event.


The specifics of how we will actually go about doing that are going to be purely on your


own infrastructure.


But the specifics of when we do it is a lot easier to cover.


The browser has another event called visibility change.


This is triggered whenever a page's visibility changes, like when it's minimized, closed,


or navigated away from.


We can use this event to expand our earlier code to include reporting.


We listen to the visibility change event, and if the visibility state is hidden, then


we will call take records on our performance observer.


This will give us a list of any entries that haven't been processed by our code already.


We can then process each entry just like we would normally.


And then we call send CLS to the server.


This is a completely made-up function that you'll need to implement in order to work


on your site.


If you have any questions about this, please leave comments in the section below.


I'd be more than happy to help anybody implement this on their site today.


The only oddity left with our code is that in order to make sure that CLS is always being


sent when folks leave the page, we're using the visibility change event.


It's great to get the right information, but it does come at the cost of multiple analytics


events potentially being sent for the same user.


If someone has our page open, but then switches tabs, and switches back, and then again, and


again, and again, well, multiple events will be sent to the server every single time that


switch happens.


We'll need to filter these values on the back end in order to get the largest value for


each user session.


Remember, there can only be one CLS value.


For more in-depth information about how this is accomplished, you should definitely be


checking out the wonderful WebVitals.js library.


It's linked in the description below.


It was written by the Chrome team's Phil Walton.


It wraps up the complexities of tracking and reporting Core Web Vitals, and is a great


source if you aren't already familiar with it.


So we know what CLS is, what it should be, how to measure it, how to track it, and well,


if you're already at or below 0.1 as a goal, then you're good to go.


Send me a message.


I can personally congratulate you.


But if you still want to hang around for a bit, I promise you that we get into the good


stuff, how to improve the cumulative layout shift scores.


Well, let's get to it.


Specifically, all layout shifts can be traced back to really a handful of causes.


These include images that don't have inline dimensions, iframes or embeds that don't have


inline dimensions, dynamically injected content that doesn't reserve space with explicit dimensions.


Are you seeing a pattern here?


And finally, web fonts or icon fonts that cause a flash of unstyled or invisible text.


Let's start with probably the most common issue that I've seen, images.


If you recall the example we looked at earlier, an image that didn't include a width or height


attribute, either inline or with its own CSS, will initially just be a placeholder node.


It's something that's completely different than the size it will eventually be when it


is used.


And just like in this example, this causes the content to jump, triggering a layout shift.


Luckily, this is a really straightforward fix.


The best, most modern way to address this is to explicitly include the height and width


attributes on our image.


These attributes have existed for decades and used to be common practice.


But with responsive design and high definition screens, setting these values was often less


useful than using CSS or just letting the browser kind of figure it out by itself.


Nowadays, all modern browsers are automatically sizing based off of the aspect ratio, as in


how wide an image is divided by how tall it is, based on the width and height values.


That means that we can still style the image using responsive values like width 100% in


our CSS, but browsers will now also automatically reserve the correct height based on the aspect


ratio of the width and height we provide.


It's the best of both worlds.


But perhaps even best of all is that this will prevent these images from triggering


CLS.


There will be no layouts being shifted, even if the image loads on a slower connection.


Next up is iframes and embeds.


This can mean a lot of different things, add slots, widgets for a video player, you know,


just about anything that we load from another domain or another service is likely to be


loaded inside of an iframe or an embed.


We can't control what happens inside the iframe, but we can control what happens around it.


Just like our images, we need to reserve a space that's going to be taken up by the iframe.


By setting explicit heights and widths for the iframe's parent element, the browser is


reserving space for it, thereby preventing any kind of layout shift.


Occasionally, the ad network that we may be getting our ads from is not quite serving


us the right size for its slot.


It might be a couple pixels too big.


I like to explicitly set overflow hidden on those iframe parent containers to ensure that


they're not going to be overflowing into anything I'm not expecting.


This isn't always practical, though.


Sometimes you need to ensure that every single pixel of an ad is visible.


If that's the case, we're going to want to oversize that parent container ad slot and


make sure that we modify our design to account for this and make sure it still looks OK,


even if it's a bit bigger.


On the other side, you're going to want to avoid collapsing ad slots when no ad is returned.


See, removing all that space is going to cause just as much of a layout shift as well.


You're likely noticing that this is all about isolating iframes and embeds so that they


don't take up space that isn't accounted for in our design.


So whether we're going to be clipping unexpected content or just overestimating the size to


be sure that all of it shows, there's almost always a way to avoid layout shifts with dynamically


loaded content.


Not all dynamically loaded content is in iframes, though.


If your site has an infinite scrolling list of articles or product pages, or if you're


just doing any kind of lazy loaded content, you will want to make sure that you reserve


the space that it's going to take up with an explicit height and width.


Just like with iframes, the server is going to hopefully be returning something that will


be these dimensions, but ideally our pages are designed in a way to be flexible enough


to allow for content that's bigger or smaller than expected.


As you may have noticed, having a flexible but unchanging width and height on our page's


remote resources is key to a rock bottom CLS score.


But there's one more still common issue that we should look at.


Up until now, all of our examples can more or less be summed up as make sure all the


rectangles have explicit sizes that don't change.


But what do we do if it's the content inside those rectangles that change?


This is why web fonts pose a unique challenge to CLS.


We can't explicitly set a width or height on text, and so with a web font or icon font


that's causing a layout shift, we'll need to approach it from a different angle.


The simplest solution is don't use the font.


In the case of icon fonts, it's actually just much better to use SVGs directly today.


Not only do you get a greatly enhanced design capability, there's going to be much fewer


issues that are prone to icon fonts in modern browsers, and it's substantially better for


accessibility too.


But if a web font is causing the shift, and we definitely want to use it, then we need


to optimize how the font is being loaded.


I'll include some articles about optimizing web font loading and general font best practices


in the links below if you really want to go deep on the topic.


But a great place to get started is to ensure that any fonts that we are definitely using


are going to be preloaded.


Normally, web fonts are going to be lazy loaded, meaning that the browser downloads the HTML,


any JavaScript, and the CSS before it starts to download and process any font-based declaration


we declare inside of that CSS.


Link rel preload is a sort of hint we can give to the browser so it can start downloading


the font before it discovers it the normal way.


That means that it'll be ready much faster when it is discovered.


We want to make sure that we put these essential fonts as close to the top of the page's head


as possible to give the browser as much time as possible.


If the web font is more of a nice-to-have, another choice would be to use font-display


optional.


Font-display is a property we can set on font-based declaration.


It supports a few different values, and the one that we're using here is telling the browser


that the font is, well, optional.


When we use font-display optional, if the web font takes longer than three seconds to


load, the browser will just give up trying to load it, and whatever is already being


rendered is what's going to be shown.


So no more changes, no more content shift.


While we're already looking at our font-based code, we want to make sure that we aren't


already using font-display swap.


That more or less does the opposite of optional.


Font-display swap tells the browser to show whatever fallback font exists right away and


then swap it out for the font that we're requesting as soon as it finishes downloading, no matter


how long it takes to download.


If you use a web font that's slow to load on a substantial part of your page, well,


you can imagine how large your CLS score could end up being if you use font-display swap.


If you want to have the best of both worlds, I think the last bit of advice I could suggest


is to check out the amazing Font Style Matcher, included in the links below.


This site lets you take two different fonts and adjust various CSS properties on each


one, so that they more or less visually align.


Once you have a web-safe font that is close enough that it's a fallback, you can copy


the CSS directly and enjoy that shiftless-free page experience.


And that's page experience, folks!


We've covered the what, the when, and done deep dives on each part of it.


If you didn't see those videos, I'll make sure to include them in a full playlist of


this series in the description below.


I know that there's been a lot to this, and I can't tell you how much I appreciate you


coming along for the ride.


I hope that you've been able to pick up a few things you can use to push your sites


that much further.


If you have any questions, please leave them in the comments below, or reach out to us


directly on Twitter.


I look forward to hearing about everything that y'all build.


Google Search News (Jan ‘20) - Data-vocabulary.org, BERT, Search Console, and more

Hi, everyone, and welcome back to the Google Search News


series.


I hope life is treating you well wherever you are.


I'm your host today, John Mueller, here


from Google Zurich in Switzerland.


With this show, we want to give you


a regular summary of what's been happening around Google Search,


specifically for webmasters, publishers, and SEOs.


If you find these useful, which I hope you do,


and if you'd like to stay up to date,


make sure to subscribe to the channel.


Today, we'll take a look at Search Console,


mobile-first indexing, structured data, web search,


and some other news.


It's been a busy time, so let's check it out.


We're starting off with Search Console.


This month, we celebrated the two-year anniversary


of the new Search Console.


It's been a long process, creating new features


to replace the old ones, and all of the feedback


from passionate users along the way has been awesome to see.


Since our last episode, the Search Console team


has worked to make the data in the Discover report


much fresher, similar to how the Search Performance


report was updated.


Discover, originally the Google feed,


bubbles up content that users have expressed interest in


without them having to search for it.


We've also made the Index Coverage


report more comprehensive.


That's not visible for all sites,


but generally means that the data we show there


is as consistent as possible with the search results.


Finally, also in Search Console, we've


updated the messaging system.


Previously, messages were shown in a common inbox.


Now they're shown right next to the reports themselves.


This makes it easier to resolve issues


associated with particular reports on your site.


Mobile-first indexing continues to grow.


Mobile-first indexing is a process


of crawling and indexing with mobile devices


to match what most users would see when they use the web.


As we switch more sites over, we've


recognized more issues that need to be resolved by webmasters.


We've recently updated our documentation


to touch more on that.


Here's a short summary.


With mobile-first indexing, we will only


index the content that's on the mobile version of your page.


If it's your intention to show less content on mobile


than on desktop, that's what will be used for indexing.


This includes not only the text, but also images, links,


headings, and structured data.


For pages with images, remember the alt attribute.


Include high resolution images and embed the images


appropriately, prominently on the page.


Similarly, if your page uses ads,


then position the ads on mobile in a way that doesn't


block your primary content.


For sites that use separate mobile URLs, commonly called


m-dot URLs, make sure to reference the mobile URLs


within the structured data, like breadcrumb, product, video


objects, as well as in hreflang annotations.


There are more details in our documentation.


Going forward, we'll also be sending messages


about mobile-first indexing through Search Console,


should we recognize issues on your site.


We're continuing to work on mobile-first indexing.


Our longer-term goal is to be able to completely switch over


so that all crawling and indexing matches


what the majority of the web's users would see.


Moving over to news about structured data.


We use various forms of structured data


to better understand content on web pages.


Recently, we decided to stop supporting


datavocabulary.org structured data per 6th of April, 2020.


Google Search Console will now issue warnings


for pages using datavocabulary.org schema


so that sites know to switch over to schema.org markup.


The most common usage of datavocabulary.org markup


was for breadcrumbs.


If you added breadcrumb markup to your site early on,


double-check Search Console's enhancement reports.


If you see warnings there, it's probably time


to update your markup to schema.org.


To check your pages for other issues,


use the rich results test.


This test will flag any recognized issues


with regards to structured data.


Within Google Web Search, we regularly


make updates, thousands of them every year.


Our website on how search works has more if you're curious.


These changes are meant to help users


find relevant information, perhaps on your site.


Site owners can find out more about these changes


on our blog as well as on Twitter.


To keep you up to date, let's check out


some of the recent changes.


Late last year, we announced the use of BERT in Google Search.


BERT is a modern way for computers


to understand natural language.


We use it both for working out what the user was really


looking for as well as for better understanding


the pages in our index.


The Google Research team has written


about some of the work they've done on this as well.


For ranking, we've made regular updates,


including so-called core updates in December and January.


We call them core updates when they affect our ranking


algorithms at their core.


Sometimes, we also make changes to the visible design


in Search, which our passionate users recognize right away


because, well, it looks different.


Recently, we've experimented with a visible design


for desktop users that's similar to what we've


been showing mobile users.


We received a lot of feedback on these changes


and will continue to test and listen to feedback as usual.


Another change was to reduce duplication in our search


results.


Previously, we might show the same page both


as a traditional search result and also with a larger preview.


We call these featured snippets.


With this change, we're removing that duplication


and showing these pages just once.


This change does not affect how your site's data is collected


and shown in Search Console, where the average position


continues to be based on the topmost element for your site.


As a side note, and unrelated to this change,


if you have preferences with regards


to how much of a snippet you'd like to have shown,


we have documentation on the appropriate robots meta tags


for you to check out too.


We have a few more short updates for you.


Chrome has made some changes with regards


to how websites are handled when they incorrectly use HTTPS.


In particular, Chrome is stricter on its handling


of mixed content.


Mixed content is when a secure page on HTTPS


includes elements that don't use HTTPS,


such as images or JavaScript.


Mixed content is bad, and it makes pages insecure.


Independently, we've sent out notices via Search Console


to websites that use older, insecure versions of the TLS


protocol for HTTPS.


These older versions won't be supported by modern browsers,


so the server must update to a newer protocol version soon.


Regarding rendering, here's a short update.


As announced last year, we've moved


to using a modern version of Chrome for rendering.


One part of that change is currently still ongoing.


We're moving to a user agent similar to what users would


have when they use Chrome.


The user agent is a text each browser


sends to web servers, which generally


identifies the browser.


Going forward, we'll be updating the version number


as we update the version of Chrome used.


Some websites try to recognize Googlebot


based on the exact text of the user agent, which they'd need


to do differently in the future.


Finally, here on this YouTube channel,


we recently launched a new series


about Google Search Console.


Google Search Console is a free tool


that helps website owners, or in general, anyone


who has a public presence on the web,


to succeed with the help of Google Search.


The video series is hosted by Daniel Weisberg


and covers a variety of topics all around Search Console.


The first videos are out already,


and there's more to come.


That's all for now, folks.


Thanks for joining us.


And now, on to the weather.


The weather outside is cold.


It's winter here in Switzerland.


It looks like it's even going to start snowing soon, which


is great if you like snow.


Who likes snow?


I love snow.


There's something special about a day out in the sun and snow.


But if you don't like snow, the nice part about winter


Announcing the Virtual Webmaster Unconference

Martin, you know that all events are canceled this year.


Yeah, we can't have local Webmaster conferences right now.


But what about a virtual conference instead?


Oh, I like those.


We can join from any part of the globe at any time of the day.


But yet another online conference.


Yes, but no.


What if we made a virtual Webmaster unconference?


A virtual Webmaster unconference.


Now, how would that work?


You know, instead of just listening to speakers giving


presentations, we have a few Googlers in the community


propose topics for discussions that then everyone attending


the event can vote on.


And then they participate in the discussions which


have received the most votes.


So no presentations?


Correct.


It will be a series of interactive discussions.


That sounds amazing.


How do I sign up?


This is happening on August 26.


So visit this URL and fill in the registration form


before August 19.


Cool.


I'd better do this before going to sleep.


See you very soon at the virtual Webmaster unconference.


Virtual Webmaster unconference, August 26.


Don't forget to register at goo.gle


slash virtual Webmaster unconference before August 19.


English Google Webmaster Central office-hours from June 26, 2020

for technical problems.


All right, welcome, everyone, to today's Webmaster Central


Office Hours Hangout.


My name is John Mueller.


I am a webmaster trends analyst at Google in Switzerland.


And part of what we do are these Office Hour Hangouts,


where people can join in and ask their questions


all around their websites and web search.


Tons of stuff was submitted already.


So I'll see if I can get some answers into things


that we don't get to in the comments as well.


But if any of you want to get started,


you're welcome to jump in with a cross-question.


I can try one, I guess.


All right, go for it.


So more of a theoretical one, let's say


you have a store or maybe a site running classified ads,


something that's fairly big.


It's doing very well.


And it kind of targets a lot of topics.


So if you have a marketplace or maybe classified ads,


you might have pages regarding jobs,


pages regarding selling fashion items, and things like that.


And let's say you're doing fairly well for most


of the topics you're targeting.


But most of the websites that rank in the first positions


are dedicated to that specific topic.


So maybe it's hard to beat a fashion store with your fashion


section of your marketplace when you have all


these other sections.


And what would happen if that website decides, well,


let's just create a new website dedicated to just the fashion


section that we have.


And we run two websites in PEL or something like that.


And that would mean the content is kind of the same.


So one product uploaded somewhere,


it shows up on the other website as well.


How does Google treat this other than just two websites


that have similar content?


Does Google like the same business


running those two websites?


Does that play any role in how Google


decides what to rank, what to index, and things like that?


I don't think we would do anything special


in a case like that.


We would essentially try to treat it


as two websites with regards to whether or not


that makes sense.


I think that's a bit of a different question,


because sometimes it makes sense to have one really strong


website that includes a lot of different topics.


Sometimes it makes sense to focus a little bit more.


And finding that balance is more, I'd say,


a marketing and kind of strategic problem rather


than SEO problem.


Because what won't happen is that if you're


ranking at number three, then suddenly you're ranking


at number two and three.


I don't think that would just happen automatically.


But rather, usually you'd probably, I don't know,


try to link to your other website


or maybe redirect some pages to that new website that you have.


And that kind of dilutes a little bit


of the value of that big, strong website that you have


and creates a little bit more value


in that more concentrated website then.


And whether or not that works well for search


or that works well for users, that's


really hard to say ahead of time.


With regards to two websites like that,


I don't see a problem.


If you're going in the direction of,


like, I'll create 10 or 20 websites like that,


then that's something where I could


see the kind of quality algorithms kicking in


and saying, oh, this looks a bit like doorway sites.


We need to be careful here.


But if it's just two websites, if you


decide to go to three websites at some point,


I don't really see a big problem there.


MIHAI APERGHISLAMOV


They've also built websites dedicated


to either the jobs market or the real estate market.


So anything somebody posts an ad on,


a classified ad on the job section


of the main classified site, it automatically


appears on the job-specific site as well.


So it's like duplicating the section of your site


and creating a whole other website for it.


And I mean, they both seem to rank well.


I mean, both sites seem to rank well for keywords related,


let's say, to the job market and things like that.


So I was wondering whether Google


has any issues with that or might be any issues that somebody


should take into account.


I don't think we would have any issues by default.


I think it's really more a kind of a strategic question.


Do you want to go down that direction or not?


There's a lot of work involved with splitting things off


and kind of duplicating things, finding


the right balance between how much you duplicate,


how much you don't.


But maybe that trade-off is overall positive for that site,


for that situation.


I don't know.


It's hard to say.


MIHAI APERGHISLOOO-MUNDEZ-SINGH CAMPBELLO-MUNDEZ-SINGH


If it's duplicates, let's say, the same pages,


the same content, classified ads show up on both websites,


are there situations where we might decide that, well, we've


already seen this content, and it's exactly the same


on this other website.


Doesn't make much sense to show both websites.


Yes.


A really simplified way to look at it


is if the snippet would be exactly the same in the search


results, then it makes sense for us to filter one of those out.


And that depends a little bit on the query.


If people are searching for something that is kind of more


specific to one of these websites or not,


that's usually kind of the simplified approach


to look at it.


If you think that the snippet will look exactly the same,


then we'll filter one of them out.


If the snippet would be different,


then we would show them both.


And if Google does decide to filter one of the sites out,


is there any way to see that in Search Console?


I don't think so.


I mean, other than you just don't have any impressions


for that query or whatever.


Like, if you have two websites, and you look at the query


statistics, you can probably see from the impressions,


like, does it match?


Does it not match?


MIHAI APERGHISLOO-MUNDEZ-SINGH CAMPBELLO-MUNDEZ-SINGH


But it would show up in the Coverage section.


So the Coverage section would still


show the websites as being indexed?


Yeah.


They would be indexed normally.


And if someone searches for, I don't know,


a keyword from the classified ad plus something


from the location or something from the title,


then we would just show that one.


Or maybe if they search for something slightly more


different to the other one, then we would show the other site.


So indexing would be fine.


It's really just showing in the search results


that would be different.


With regards to indexing, what would be problematic


is if the exact same page were used on both of these sites.


If you use the same layout, the same HTML page,


if that was exactly the same on both of these sites,


then we would see that as duplicates,


because it's exactly the same thing.


Cool, thanks.


Sure.


All right.


Any other questions before we jump into the submitted ones?


Hi, John.


Hi.


I have a question about singular and plural form of keyword.


So one of our clients' keyword is Garden Shade Sydney


and Garden Shade Sydney.


Now, for Garden Shade Sydney, the category page,


Garden Shade category page, ranks on Google.


But for the singular form, Garden Shade Sydney,


one of the blog posts is ranking on Google instead


of the category page.


Why is it different?


Both keywords are same, just singular and plural.


So why is that different?


Yes.


We, so it's hard to say without looking into the exact details,


but we would see those queries as being different.


I think kind of the first step there.


And when we see them as being slightly different,


then we might think that one or the other of these pages


makes more sense to show.


So usually, with singular and plural,


we do recognize that they're synonyms, more or less.


But we also recognize that maybe there


is something kind of unique to one of them


or to the other one, such as if you're looking for a plural,


maybe you're looking more for like a list or a comparison


page or maybe a category page of different kinds of these items.


So that's something where our systems


try to take that into account.


And it can result in slightly different results being


shown for one or the other.


It's a bit tricky when you're in that situation.


You're like, oh, but I want my other page


to rank instead of this one.


And you don't want to remove the page that you currently


have ranking.


That's something where you, I don't know,


you can't really force that other than to tweak things


subtly, that you kind of make sure


that the right words, the right phrasing is on these pages,


that you link them internally, properly.


But that's sometimes kind of tricky.


It's also worth keeping in mind that just because when


you take a step back that these words or these queries sound


very similar and they seem very much the same,


it might well be that users do treat them


as different queries and do expect different kinds of results.


So if kind of like before just jumping in and saying,


oh, I need to have the same page rank for both of these,


maybe check with some other people to see,


does it make sense to change this,


or is this something where it's actually not that bad?


Or another thing you could do is the page


that's currently ranking, put some kind of a call


to action on it and say, hey, if you're looking for this,


also check out this other page.


And another question, John.


So we have a client who they are a startup management company.


They have a business in five states in Australia.


For each state, they have different websites,


and they have one corporate website.


Now, their corporate website has a block,


but the other website doesn't have any block.


Now, the corporate website, the block of the corporate website,


it gives link to the state website.


Will there be any problem?


So what's the last part?


OK, so the block of the corporate site


gives link to the state website.


Will there be a problem?


OK, so the blog just links to one of them or to all of them?


It depends.


They don't give deep links every time.


When it's necessary, they just give a link


to one of the state sites.


If it is a big event to that site.


I don't see a problem offhand with that.


I think this kind of setup that you have,


one big corporate site for kind of the overgroup,


and then the individual locations


or the individual subcompanies, separate websites,


and them kind of linked together,


I think that's pretty common.


That's not something I would consider problematic.


Thank you, John.


Sure.


All right, let me run through some


of the submitted questions.


And as always, if you have any comments or more questions


along the way, feel free to jump in.


And towards the end, hopefully we'll


have some time for other questions from you all.


Our website and our content is quite usually


getting copied by someone else.


They basically just create a new domain


and copy the whole website on it.


Is there something apart from disavowing these sites


that we can do?


Can we submit these sites as duplicates


so Google won't harm our results and sites as a result?


I think this is always kind of tricky.


The general approach that I would try to take here


is to see if there's something from a legal point of view


that you can do to essentially tackle the problem at source.


Because if the website is no longer up,


if it's no longer being indexed, then that's


essentially a solution for search as well.


So I would double check with whoever


you work with on a legal side with regards


to things like DMCA and copyright


to see if there's something that you can do there.


With regards to kind of like the location of the website,


if they're hosted in other countries, all of those things,


that might be an issue with regards


to kind of the full legal process.


But with regards to how Google kind of processes


these reports, generally that's less of an issue.


So that would be kind of the first step


that I would take there.


And alternately, what you could do is, of course,


submit these with the Web spam report.


With the Web spam report, it's important


to keep in mind that this is not something


that we go through one on one and kind of manually review


everything and throw all of these sites out.


For a large part, these are used essentially


to train our algorithms to improve over time.


So just because you submit something to the Web spam


report, I wouldn't expect that to kind of be


removed automatically after a couple of days


or something like that.


But those are kind of the approaches


With regards to disavowing those sites,


if they're linking to your website


and you really don't want that link to be taken into account,


then using the Disavow tool is fine.


If they don't link to your website,


if they're just copying your website,


then there is nothing really there to disavow.


Hi, John.


Can I ask a question to that?


Sure.


So there might not be a possibility


to take those legal steps you're talking about,


but when it comes to Google, is there some possibility Google


will harm our website, or can Google


see that our website has history and is much bigger,


has much more authority than the copies?


And so therefore, it's basically OK?


I don't know about basically OK, because you submitted


this question.


It sounds like you were seeing some problems.


But yes, we do usually try to figure that out.


It's really common that sites get copied on the web,


and it's important for us that we


can work out which of these versions


should be shown in search and which ones


we can kind of filter out.


All right.


So basically, the only step we can take


is to submit the website to the spam report, and that's it.


Right?


Yeah.


I would still double check to see if DMCA works,


just to kind of make sure that you're


covering all of the bases.


In particular, the report through Google,


because that is something that, from our side,


the legal team reviews.


And if it's really a one-to-one copy of your website,


then that's something where maybe they'd be able to help


resolve that.


Mm-hmm.


It's MCA?


DMCA, yes.


Oh, DMCA.


OK, thank you very much.


Sure.


OK.


Is a category counted as duplicate content


if it's indexed on Google?


No, not necessarily.


On a category page, you might have individual items


that you have on your individual item pages,


but that doesn't mean that it's duplicate content.


I think duplicate content is kind of a tricky term anyway,


because people often frame it as like, oh, this


is bad for your website.


This will harm your website.


Just because something is recognized as duplicate content


or if parts of a page are recognized as duplicate content


isn't necessarily a bad thing.


It basically just tells us, well, we've seen this before,


and if someone is searching for something in this block of text


that's duplicated, we need to find out


which of the pages that have that copy of text


are the most relevant.


And it's not the case that a website


will be thrown out of search just because it


has some duplicate content.


Because essentially, every website


has some amount of duplicate content that's kind of,


I don't know, it's almost impossible to avoid.


We've been doing quite a bit of speed optimization on our page,


and we managed to get a consistent over 90 score


in our performance in Lighthouse.


But this is only without our ad stack enabled.


Once we turn it on, we drop in the performance scores


to below 10.


We've debated this at length with our ad provider,


but they insist there is little they can do.


If we want monetization, we've done a bit of research


and found that many face similar challenges


with different ad stacks.


So here is my question.


Is anyone in the 90 plus range for speed in Lighthouse


that has ads, it feels pointless to further optimize


the page for speed with the limitation


that the ad stack prevents?


And then there are some examples.


So I didn't check these pages explicitly,


but essentially, when it comes to speed,


we do take the pages into account the way


that we get them.


So if you have ads on these pages,


and the ads are the thing that makes the pages slow,


then we see those pages as being slow.


So that's something, I don't know,


just kind of worth keeping in mind.


With regards to the absolute score there,


it's also worth keeping in mind that, especially the 0 to 100


score that you have, I think, in PageSpeed Insights


is something that's more of a guideline for finding


different types of issues.


And it's probably worth more looking


at things like the Core Web Vitals,


where you have a little bit explicit numbers to work on.


But I don't know offhand which websites have really high scores


and use ads, but the Chrome User Experience Report data,


that is something that's public.


And I believe you could even download that


and you can double check different sites


from your country and see how they are performing.


You can also use the HTTP Archive


that also has a full Lighthouse performance reports in there.


So you can look at things over time for other websites


if you want.


So that's something where, from my point of view,


if you're worried about one specific ad provider, that's


something that you can often test across different providers


and see how other websites are performing.


So just to add on that, because so far we haven't really


found anyone in our industry nor in our country


region that leverage ads and have reasonable Lighthouse


speed, basically, grades.


So it seems really frustrating because we've


spent so much time on optimizing.


And really, the only thing that seems to be preventing us


is actually the only thing that makes us money on the page.


So it's a pretty frustrating trade-off, to be honest.


Yeah, I can understand that.


Yeah, that's always tricky.


But on the other hand, the good part


there is for the things that you're competing


with with other people, they have the same problems, right?


So it's not exactly because we're also


competing with government agencies that don't do ads.


OK.


Since we have a provider, basically,


we have government weather services


that don't need to monetize.


Then maybe you have to be more creative.


Now, I don't have any exact answer.


So it's not from our side that we're saying ads are bad


and you should remove your ad so that you get a good score so


that you can rank well in search.


It's really just we need to make sure


that when people search for something


and they click on a result, they find the answers


that they need on those pages reasonably quickly.


That's kind of what we're looking at.


And maybe there are other ways that you


can monetize things in a way that


doesn't prevent the primary loading of the page.


So especially if you're looking at, what is it,


largest meaningful content?


I forgot what the names are.


There's like so many.


That might be something where maybe deferred loading


of the ads would be an option.


We are doing that.


OK.


It's actually been a large discussion with my ad provider


because they always want to be in the head.


And so we've been debating that.


But still, we are getting these bad Lighthouse scores,


which is surprising to me.


But I'm also wondering if the scores are perfectly correct


here.


But really, we've been basically pushing back the ad stack


as much as we can.


Yeah.


I mean, the score from 0 to 100 is kind of arbitrary


because it's kind of a weighted score of different factors


that are involved there.


It's probably easier to look at exact scores, which


are based more on kind of timing and those kind of things,


especially when you're kind of fine tuning the optimization


there.


Because things like, oh, was this served from a CDN or not?


It doesn't really matter as long as it gets there fast enough.


I think you're also going to use those as a ranking


signal in the future.


So that has us additionally scared in a way.


I mean, we do use speed as a ranking factor on mobile


already.


It's more a matter of is this page reasonably fast or not?


With regards to the core web vitals, the new ones


that we announce, they're part of the page experience score,


which is kind of what we plan to use with regards to speed


and ranking at some point in the future.


We kind of haven't announced a date for this yet.


And we'll announce the date at least, I think,


six months before we actually turn it on so that you


have a bit of time to figure those details out if you feel


that you need to do something there.


And the page experience benchmark is based on, I think,


a percentile from the site's point of view.


So it's not that you need to optimize every millisecond,


but rather you need to be in the green range.


And then if you're in the beginning or the end


of the green range, that's more or less OK.


Thank you.


Sure.


I think you're going to be, I ran the site


through Lighthouse.


So one thing that I think you can do,


I ran the ads version, the one with the ads.


So I don't see ads as being an issue in the sense


that you're not showing above the fold ads, at least


on mobile, from what I noticed.


So that shouldn't affect the largest contact flow paint


metric, which just that is like 25%


weighted in the score of the overall performance score.


But what I noticed is that you're


showing a cookie kind of banner.


So that shows up a lot later, and that


affects how LCP, the large contact flow paint, is calculated.


Because your LCP, first contact flow paint, is like one second.


But your largest contact flow paint is almost seven seconds,


and that's only because Google, or I mean,


PageSpeed Insights, Lighthouse calculates when


that cookie pop up shows up.


So it would be.


I think you're not seeing the worst of it


because you're in a geo area where we don't have


our primary ad stack.


So you will be seeing something else


that I'm seeing because I'm in a region where there's


a lot of stuff happening in the ad stack,


and you're just in a backfill mode.


I guess you're not hospitalized, I suppose.


I tested with PageSpeed Insights as well,


so I don't know if for the US you also have a different.


Yeah, so with regards to location,


that's also a good point.


We use, essentially, or the plan is


we use the Chrome User Experience report data, which


is based on the normal users that are accessing your site.


So one thing you don't have to worry about


is whether or not people in the US


have fast access because that's where Google is based,


but really where your users are based.


That's where you need to watch out for speed.


And I guess it sounds like that's


the place where it's the hardest on speed, which is tricky.


Yeah.


Andres, if you can start the thread on the Webmaster forums,


I can happily take a deeper look into it


and find some suggestions.


Sounds good.


Thank you.


Awesome.


Cool.


All right.


Does having three open and close HTML tags in a page


negatively affect Google's ability


to determine the main page content of a page?


No.


So this is essentially a matter of kind of like,


is there a valid HTML on the page or not?


And that's not something that we would use with regards


to ranking at all.


The only time the valid HTML or not plays a role


is if you have specific meta tags or specific attributes


that need to be in the head of a page.


And because the HTML is so invalid


that the head of the page is kind of slipping


into the body of the page, then we


might not be able to confirm that these tags are actually


in the head.


So for example, if you have hreflang attributes


on the page and the head of the page


is so broken that we think it's a part of the body,


then we would assume that these hreflang attributes are not


actually valid because we can't find them


in the head of the page.


So that's kind of the one situation where valid HTML


would play a role.


But otherwise, it wouldn't play a role at all with regards


to ranking Google.


Let me just correct.


One other place where that would play a role


is when you're looking at things like structured data


where you need to have the HTML structured in a specific way


so that we can extract the structured data properly.


But that's kind of like if it's not,


it would result in that structured data not being


valid, which essentially would be the primary problem.


It's not that the HTML has to be valid,


but rather the structured data has to be valid.


Is it possible for a domain to be punished


if a site removes a lot of articles


and makes them 404 due to some legal action or author


requests?


No.


So removing content is essentially perfectly fine.


It's part of the normal web.


The only kind of punishment, if you will,


that you will see is that obviously once this content is


removed from our index, you won't rank for that content


or for the content that used to be there.


But it's not that the rest of your site's content


would be ranked worse just because you


have a lot of 404 errors.


404 errors are essentially part of the normal web.


It's not something which we would take as a sign


that the website is low quality or bad or anything like that.


Could the change of address tool be used in some way


for migration where you're consolidating two domains down


to one, trying to think outside of the box on this?


I could see a situation where possibly you


could use the change of address tool like this in the sense


that you could probably set up the configuration of your sites


to make it so that we would process the change of address


tool there because there are some requirements that we look


out for with the change of address tool,


such as that you have a one-to-one redirect from the old


site to the new site.


But you could theoretically set something like that up.


Whether or not that makes sense, I don't know.


Because the tool is really meant to transfer essentially


all of the signals from one site to the other site.


And the idea is kind of to replace the new site


with the old site.


And if you're consolidating multiple domains,


then it's not that you're replacing the new site


with the old site's information, but rather that you want kind


of like everything added together.


And that's something where I could imagine you might be able


to use a tool kind of like from a practical point of view.


You can access it.


You can click OK and make it kind of activated.


But I don't know if you would have any positive effect


over just redirecting normally.


My feeling is probably you would see some weird effects,


but I don't know if overall you would have kind of something


positive out of that.


We fixed some really old product URLs with 301 redirects


and see some huge spikes in backlinks.


How does Google react to this?


We're seeing some substantial decline in ranking after that.


Is it related?


Usually changing URLs on your website, fixing old URLs,


redirecting them to make them valid,


that's perfectly fine.


That's not something that I would say would be problematic.


It wouldn't be something that we would say is a sign of spam.


Definitely not because you're improving your website.


My feeling is the decline in ranking


is probably unrelated to fixing the URLs on your website,


but maybe something that was happening anyway


and is kind of independent of that.


So I would continue to improve the URLs on your website,


fix them with redirects if you notice


that they were kind of like linked in correctly,


those kind of things.


If there's a domain whose archives shows


some different themes and content


than the current theme and content,


should we consider that kind of domain


is good for backlink purpose?


So I don't know what you would consider


good for backlink purpose.


That's kind of the part where I'm a little bit stuck.


But essentially, just because the cache page


looks slightly different than the current page,


that's not a bad sign.


That's usually a sign that the website is being worked on,


that things are changing.


That's perfectly normal.


With regards to good for backlink purposes,


it sounds like you're dropping links on other sites,


and that's kind of the bad thing.


It's not so much that the website is being worked on,


but rather, if you just go out and you add links


to other people's sites, that would be kind of not so cool.


Since a couple of months, the favicon of my site


disappeared from mobile search results


while it keeps being shown in Search Console.


I checked the Google guidelines, and everything seems OK.


The only thing that changed in the same period


is that I changed the URL structure from www.site.info.


to www.site.info.en, since I have multilingual versions, all


with their own site map.


So the root is no longer being indexed.


It 301 redirects to slash en.


Could that be the reason?


I don't know.


That's actually an interesting question.


I haven't looked into how that would be handled.


So I think on the one side, one of the things


I've noted with favicons is that it just takes a lot of time


to be processed.


So if you make changes with the favicon, if you fix something,


if you change the URL, then that's


something that takes quite a bit of time


to actually be reprocessed.


So if you made this change, I don't know, a month or two ago,


it's possible that that's still being processed.


But on the other hand, it feels kind of weird


that if you don't have a root URL,


but rather kind of like you're redirecting


to some lower level page, if that's the only change


that you made, then that seems like something


that we should be able to deal with.


I am online.


OK, cool.


Can you drop me a link to your site, maybe in the chat?


And I can pick it up afterwards and double check with the team.


Yeah, because one of the requirements for the favicon


is that the root must be crawlable by Google.


And my actual root, so the domain,


is not indexed because I have two home pages,


one for one language and the other for another language


and so on.


So if you type the basic home page,


you get a 301 redirect to the default language


that is the English version.


So actually, the real root is not crawlable by Google,


but the English version, yes.


So that was the question if it is this.


Yeah.


Yes.


I don't know.


It's the first time I hear about this configuration,


but I'm happy to take a look with the team.


Yeah, but is it a bad configuration


having just a home page on different language


and not the naked domain?


It's essentially fine.


It's not something that I would say is problematic.


I think sometimes it might confuse tools


if you're running tools over the website


and you don't really have that root home page.


But from a search point of view, this


is something that a lot of websites do.


So one thing you might want to do is look at hreflang


and use the x-default for the primary version that you have.


But that's, I don't know, kind of like small optimization.


It's not something that is critical


that you need to do.


I will drop you the line.


Thanks.


Sure.


I want to know that I update my website every day


with a new post, and Google indexes the post, but not


the keywords of that post.


So can I know the reason behind that?


I don't know.


It's hard to say just based on that question.


So if we're indexing the contents of a page,


then we essentially have that page indexed.


And we could show it in the search results


for any searches that are made for that content.


What might happen, or I guess depending


on the keywords that you're targeting,


is that if you create something completely new,


that it doesn't immediately rank for competitive keywords.


So that's something that feels kind of normal


from that point of view.


So maybe it's a matter of just taking more time,


working out ways to improve your content overall,


and kind of growing your website from there,


rather than just creating new content every day


and hoping to rank for the keywords


that you have in that new piece of content.


But it's not something where I would


say we have any systems on our side that say, oh, we


will index all of this content, but we won't show it


for any of the keywords for it.


Because if we want to kind of index it,


then we want to index it with the ultimate goal


of showing that in search.


It doesn't make any sense for us to index something


that we wouldn't want to show.


My website is in Marathi, an Indian language,


and we'll look at index for the English translated queries.


So if your website is completely in one language,


then generally we wouldn't be showing it for queries


made in other languages.


There are always, I guess, some exceptions to this


and some situations where we do kind of this kind of thing


where we recognize that someone is searching in one language,


but we can't find any content in that language.


And therefore, we will translate that query


into maybe another local language


and try to see if we can find something there.


I believe that's something that we've done.


I don't know if we still do it in India in particular, where


maybe there is not a lot of local content yet,


but if we translate the query into English


and we can show English pages using Google Translate perhaps,


then that's something that might make sense there.


But in general, if you're working on your website


and you want to rank for queries in a particular language,


then I would strongly recommend that you actually also


write your content in that language.


Using things like hreflang lets you


connect those different language versions.


But essentially, having content in a particular language


version, if you want to be shown for queries in that language,


that's really important.


And that's especially important when


it comes to more competitive terms, where


there is a lot of content already in one language,


then you really need to make sure


that you're putting your best foot forward,


that you're creating the content as ideal as possible so


that, on the one hand, our systems can look at it


and say, oh, this matches exactly what the user was


looking for.


And users, when they go to your pages,


they're like, oh, this is fantastic content.


This is something I need to recommend to my friends,


for example.


My website is only restricted to the US demographic.


Can I bring links for my site from outside of the US country?


Again, kind of this, can I bring links to my website?


That sounds a lot like I would like to randomly create links


on the web pointing at my website,


and I want Google to think that these links are natural, which


is kind of more the problematic aspect there.


With regards to restricting a website to the US,


that's something you can do.


In general, we crawl websites from the US in most situations.


So if we can access the web page,


we would be able to index it.


I think it's a bit, I don't know,


tricky or not always that great of a user experience


when people outside of the US click on a search result


like this, and they essentially can't access the website.


But sometimes you need to do things


like this for legal reasons.


I guess the problematic alternative to this,


just to mention that as well, is if you


have a website that is being crawled from Google


bot in the US and you want to block users in the US


from accessing the page, then Google bot


wouldn't be able to access it either,


and we wouldn't be able to index it.


So that's kind of if you were in the other situation where you


needed to block users in the US.


But with regards to links in general,


if people are linking to your website,


then that's something that we take into account, regardless


of where those links are located.


For multilingual blog, which is best, subdomain or subdirectory,


and how to configure that in Search Console?


Oh my gosh, I don't know if I want to start this fight again.


Essentially, when it comes to multilingual websites,


we don't care what the structure of your website is.


So that's something where if you feel


it makes more sense to do it on different subdomains,


then go for it.


If you feel it makes more sense on subdirectories, go for that.


If you want to use URL parameters, that's fine too.


So for different languages, any URL structure


works as long as you have one language per URL.


So as long as you don't have this fancy setup


that your website on the same URL


returns with different language content,


depending on the user location, because we would not


be able to index that.


On the other hand, if you were targeting multiple countries,


so not multiple languages, but multiple countries,


then we need to have a clear structure of the website


for individual countries, which means, again, subdomain


or subdirectories would be the approach to take.


And in Search Console, you would need


to specify or verify those separate sections


and specify the geotargeting of those sections.


So for multiple languages, any structure you want is fine.


For multiple countries, you need to use subdomains


or subdirectories or different domains


even, if that works for you.


I have a client that has one of their navigation options


as a JavaScript dynamic URL.


So if someone clicks on media in their navigation,


they land on a slash media page.


But all the resources and the blogs


live on slash media type equals blog, et cetera.


I know Googlebot is smart when crawling JavaScript,


but some SEOs say it's best to have clean URLs with past


like slash media slash news.


How good or bad is it to have a navigation


this dynamic for SEO?


I also read about kind of hashbang on Google's blog.


I feel my lack of computer science


made this topic a blur to me.


So on the last part, that's perfectly fine.


We strongly don't recommend using hashbang anymore.


So that setup is something I would not bother looking into.


You don't have to kind of worry about your computer science


skills for that.


With regards to dynamic URLs or not, I think,


as I understand it, the problem is less


that it's based on JavaScript, but rather that you have


the question mark in the URL, and then you


have type equals blog or kind of different parameters


within the URL rather than just files and directories.


And from our point of view, that's perfectly fine.


That's not something that you need to change.


So kind of these URLs with parameters


or URLs without parameters, from our point of view,


they're essentially all equivalent.


They rank exactly the same in search.


Sometimes it's even, like, if we talk


with the crawling and indexing teams at Google,


they tell us people should not try


to hide these kind of parameters because having parameters


in your URLs makes it a lot easier for our systems


to actually crawl and index them.


And the reason for that is that we


can recognize that these parameters are


unique elements of a URL.


And our systems can learn, does this parameter make sense?


Is this something that is critical for crawling


and indexing of the page, or can we remove it?


And by learning these kind of things across a website,


it's easier for us to crawl more efficiently.


So if you have kind of everything in a path of a URL,


then it's hard for us to say, well, this path element,


the fifth one in the line, is something


that we can drop or kind of simplify


for crawling and indexing because that


gets a part of the path.


And theoretically, it's something completely different.


Might lead to a different script on the server, all of that.


However, if it's a part of a parameter in a URL,


it's like question mark, type equals media, or I don't know,


page equals two, then that's something


that our systems can learn.


And they can understand that this is critical for the page


or this is less critical for a page.


And we can optimize our crawling and indexing based on that.


So it's definitely not necessary to go from parameter URLs


to kind of like just pure file name or directory URLs.


I think sometimes it makes sense to have cleaner URLs just


for users because sometimes users copy and paste


URLs and they share them or they link to them.


And having really nice and clean URLs


makes it a little bit easier.


For the most part, I feel URLs themselves in Search


are a little bit overrated because they're


less and less visible in the search results.


In browsers, they're less and less visible as well.


So that's something where things like using breadcrumb markup


will probably have a stronger effect with regards


to the visual identity in Search when


it comes to individual URLs.


So I guess I drifted off a little bit on a tangent there,


but essentially having these kind of parameter URLs


or having clean URLs is essentially fine.


There is no need to kind of force it one way or the other.


OK, we're kind of running towards the end


and tons of questions left.


Like I said, I'll try to go through these


and add some comments as well on YouTube for all of the things


that we don't get to.


But if any of you want to kind of jump on in with more


questions live, feel free to do so.


I also have a little bit more time afterwards if any of you


want to stick around and maybe ask some questions.


More off the record, I guess.


Hi, John.


Are you able to hear me?


Hi, yes.


Actually, I wanted to ask more about 301 and 404.


We have got more than 1,000 pages in our website.


That is 404.


And we would like to solve them.


And those pages are also getting link backs


from external websites.


And I wanted to know how we can solve that thing, because that


is out of our control, because there are external links.


And what should we do?


Should we add 301 redirects to all 1,000 pages


to the relevant pages?


Or whether, I mean, also, my concern


is whether this will increase our crawl buzzer or not.


So if they're already returning 404,


then we essentially don't use any of those links


to those pages.


It's not that they cause any problems.


Essentially, you can keep it like that.


If you've replaced what used to be there with new pages


or with different pages, then using a redirect


is the right approach there.


So for it to have some related pages, yeah.


Yeah.


So if the old page is one product version,


and then you stop selling that product, and it went 404,


and now you have a new version that replaces the old version,


then redirecting from the old URL to a new one


is perfectly fine.


I think that's good for users.


That's good for search as well.


What will happen then is any external links


that we're pointing at the old version,


they will be forwarded to the new version.


And that's essentially OK.


But just going in and saying, I have 1,000 404s.


I will redirect them all to my home page.


That's something where our systems, when they look at that,


they will say, well, it doesn't look like your home page


has replaced these pages.


It's more that you're doing this as a way of kind of having


a 404 page that just shows users the home page.


And in a case like that, we would probably


treat that as a soft 404, which means internally, we would say,


this is essentially the same as a normal 404,


and we would just drop those pages


and drop those links or anything that was associated with that.


So if you have a replacement, use a redirect.


If you don't have a replacement, using a 404 is perfectly fine.


And what about the crawl budget, whether it will increase


any big difference, even if the website had millions and millions


of pages that were like this?


I don't think you would notice a big impact on crawl budget.


SPEAKER 1 OK, fine.


Thanks, John.


JOHN MUELLER Sure.


SPEAKER 2 Hi.


Can I ask a question?


SPEAKER 2 Is there any negative impacts


on blocking with the robots.txt resources


that we think they are not to be indexed by Google,


like the cookies, JavaScript, some other pop-ups,


or other stuff that we think it's not worthy,


so we block with robots, or Google is not so happy that we


cannot download some resources?


JOHN MUELLER Ultimately, that's fine.


What's important for us is that we can render the page


and see, for example, that it's mobile-friendly or not.


So if, by default, you're blocking all of the CSS


and all of the JavaScript, and that makes your HTML page


unreadable on mobile devices, then we


would think this is not mobile-friendly.


But if we can render the page and we can tell,


oh, this is a normal mobile page,


and it's just one JavaScript file or one data file


that you access, you need to block it,


that's perfectly fine.


SPEAKER 2 Thanks.


JOHN MUELLER Sure.


SPEAKER 3 John, can I ask a question, please?


SPEAKER 3 It's a question about site links.


It was actually posted in the comments.


But is there anything we can do as webmasters about the text


that is underneath individual site links?


Because it's throwing out gibberish for one of my clients


and they're querying it.


JOHN MUELLER Oh, no.


OK.


If it's really gibberish, it would


be great to have some examples of that.


Usually, site links are based on the structure of the website


as we can recognize it.


So that means it's something where we maybe


found a link that was going to one of those pages


using an anchor text like that within the website.


That's something where if you're seeing pure gibberish,


then it sounds like we're not able to recognize maybe


the anchor text on some of those pages, which


could be maybe something weird with regards to JavaScript.


It could be maybe something weird with regards


to the encoding of the kind of characters on the page,


especially if it's something in, I don't know, non-English


or non kind of the 7-bit ASCII character set.


Then sometimes it can happen that we pick up


the encoding wrong.


And then the anchor text obviously will also be wrong.


What I've also sometimes seen is depending


on how you link pagination within a website,


sometimes we'll just take number seven


and use that as a site link because we think


this is particularly relevant.


Because maybe you're linking to all paginated pages


from all pages of the website.


Then we think, well, number seven is as good as number five.


So maybe we'll just link with number seven.


That can also happen.


But if the internal linking is really fine,


if you use kind of a crawler of your own to crawl the site,


you can tell the anchor texts are all OK.


And we're picking up something that really doesn't make sense,


then that would be something to pass on to the team.


SPEAKER 2 OK, I'll double check all that and send you


a link or a screenshot or something.


Cool, sounds good.


All right, getting kind of quiet.


OK, so maybe I'll just pause the recording here.


If any of you want to, you're welcome to stick around


for a bit longer.


Like I mentioned, I'll try to go through some


of the other submitted questions and add some comments where


I can to catch up a little bit there, since there's so much


stuff.


But in any case, I hope you found this useful


and would love to see some of you


again in some of the future Hangouts again.


Thanks for joining, and wish you all a great weekend.


Handling 307 Redirects #AskGoogleWebmasters

Today's question is from Lee McCoy.


Lee asks, how does Googlebot interact with HSTS 307s?


Well, in short, it doesn't interact with them.


307 redirects are generally not real redirects.


So what does that mean?


Well, when you make a site HTTPS,


you can optionally use HSTS.


HSTS tells users to only get the HTTPS version of a page.


So when a user enters a URL or clicks


on a link that would otherwise go to HTTP,


the browser remembers the HSTS and goes directly


to the HTTPS version.


To make it clear what's happening,


it acts like there was a redirect.


Chrome calls this a 307 redirect.


So if you use Chrome and you see a 307 result code with a tool,


it's not really there.


When it comes to Googlebot, we try


to crawl URLs with a fresh slate.


So we wouldn't keep the HSTS lists


and rather just directly access the HTTP URL directly.


If that URL redirects, which is usually


the case with an HTTP and HTTPS site, we would follow that.


So in short, Googlebot doesn't see the 307


that you'd see in a browser.


And that's fine.


I hope I got all the HTTPS and HSTS is right.


And I'll see you in the next episode.


Which is usually the case with an HTTP.


I hope I've got all the HTTPS and HSTS is right.


I hope I got all the HSTS.


English Google SEO office-hours from February 19, 2021

All right, welcome, everyone, to today's Google Search Central


SEO Office Hours.


My name is John Mueller.


I'm a search advocate on the Search Relations team at Google.


And part of what we do are these Office Hour Hangouts,


where people can join in and ask their questions around Google


Search.


And hopefully, we can find some answers along the way.


Lots of stuff was submitted already on YouTube.


So we can go through some of that.


But as always, if any of you want


to get started with the first question,


you're welcome to jump in.


Hey, John.


Can I start?


Sure.


OK, so you might have answered this question


in one of the previous meetings.


But somehow, I missed it.


This is related to core web vitals.


Just wanted to ask this web vitals thing


is going to impact your website on the domain level or the page


level?


For example, there is a website with 20 pages


with good vitals score.


And then there are other 20 pages


with average to poor scores.


So these four pages can impact rankings


for these good pages also.


Good question.


So the Chrome User Experience Report data


is available on different levels.


So that's something where, depending


on the website, the amount of data


that we have available to work with,


then we might be able to be more fine-grained,


or we might just have one score that we


need to use across the whole site.


And you can see some of that in Search Console


directly, where you see how many pages are grouped together


in this one data point that you have.


So this domain, does it include subdomains also,


or subdomain is separate thing?


As far as I recall, the Crux data


is kept on an origin level.


So for Chrome, the origin is the host name,


which would be on a subdomain and protocol level.


So if you have different subdomains,


those would be treated separately.


OK.


Thank you.


Sure.


All right.


Lee, I think you raised your hand.


Yeah.


This might be a silly question and maybe one


that you've covered beforehand.


But I work with a company that has a very large footprint


on the internet.


We have over 100 million pages that we're


trying to have indexed.


And obviously, that's a struggle.


But in Google Search Console, we have submitted a site map


index with over 1,000 different site maps contained therein.


Currently, in Google Search Console,


it's saying that we have submitted and indexed


3 million pages.


However, when I look at the site map coverage,


under that site map index, it is only


saying that it is indexed 40,000 pages.


And it's only recognizing one file out of 1,000 site maps


in that site map index.


So there's some discrepancy there,


because that one site map that it's recognizing


only contains 40,000 files.


My question is, we have some issues with Google crawling


our site and returning server errors,


like 500 errors and some soft 404s.


Would that contribute to Google stopping crawling or not


recognizing our site maps because it's getting stalled up


somewhere?


I guess I'm just a little bit confused


as to where to go from here as far as next steps.


Yeah.


I am not 100% sure.


But I suspect in Search Console, maybe you're


just seeing a part of the picture because of the way


that the site map files within the site map index files


are treated there.


So that's something where maybe you're just


seeing one of those site map files in Search Console.


But actually, if you added the site map files individually,


you could probably see data for the other ones.


And that's more of a reporting thing


rather than an indexing thing.


So it's more that you just don't see the details there.


With regards to crawling in general,


with a really large website, it sounds


like that's the case there.


It is kind of helpful for us to be able to crawl


as much as possible.


And that kind of flows into the general topic of crawl budget.


So there was a blog post, I think, now maybe three years


back from Gary about crawl budget,


kind of going into the different aspects


that we take into account there.


And on the one hand, we have kind of the crawl demand


from our systems, where we think,


oh, this is a fantastic site.


We need to crawl as much as possible from it.


And with a really large website, sometimes we


have mixed signals there.


Sometimes it's like, oh, there's a lot of content here.


But actually, we found it's not as useful.


So maybe we don't crawl as much.


I don't know if that plays a role there.


And the other side is kind of the technical aspect


that we don't want to cause problems on the server.


And that's kind of something where things like server errors


flow in, where kind of the delay on responses from the server


flows in, where if we see when we crawl


a certain amount of pages per day


that the number of server error goes up, then we'll back off.


And we'll kind of assume, oh, maybe we're


crawling too hard and causing these problems.


And we want to leave capacity for users too.


So we'll back off a little bit.


And similar with server response time.


So the rendering side, kind of the speed aspect


that most people focus on is usually less critical


when it comes to crawling.


But the response time, like how quickly does a server


respond to requests, that's something


that kind of on a technical level


slows us down from crawling.


Got it.


OK.


Well, thank you for the insight on that.


I do appreciate it.


Sure.


All right.


Michael, I think you have your hand raised too.


Yeah.


So I often notice that an established outlet


will republish a wire stories article, such as Reuters,


and then they'll leapfrog in search the original wire


service, whether it's Reuters or Associated Press.


Although services aren't spammy, they're very trustworthy,


what factors would contribute to that happening?


I don't know.


It's hard to say.


I think in most cases, we try to recognize situations


where exactly the same article is just being republished


and then to kind of treat that accordingly in search


by showing maybe the original or the one


that we think where it might have come from.


But there are lots of cases where we can't really


recognize that completely.


And then it's sometimes a matter of, well, this content is here,


but someone also wrote about the same topic in somewhere else.


And then we kind of have those two viewpoints.


So that's something where it's not,


I don't think there is anything technical or anything specific


that is happening there where it's like, well,


if it gets republished here, then we just take that one.


But any time you have content that is syndicated,


it can happen that our systems don't recognize that it's


like, we should be showing this version instead


of the other version.


Is there any difference between a press release


and any other type of content publishing?


It's just content, isn't it?


Yeah, yeah.


Usually it's just content.


I think to some extent, we probably


try to recognize press releases and understand


that these are pieces of content that are just republished


in lots of places and to try to act accordingly for that.


But otherwise, it's just content.


It's kind of like if I write a blog post or a news article,


it's essentially a piece of content for us.


The nature of press releases is that you


want it to be somewhere else.


I mean, the whole point of releasing a press release


is that it goes out into the world where people actually


read it.


No one goes to Reuters to read the news.


If there's a new iPhone release, for example,


which is always the most popular example,


you go to all of the bloggers and tech reviewers and Apple


itself to read that.


So it's the very nature of a press release


that you send it out into the world hoping that people


will read it elsewhere.


Yeah.


I don't know if Google News does something


slightly different than web search in that regard, though.


So that might be something that also kind of plays in there.


I know from, I don't know, some book about Google a long time


back where in the early days of Google News,


they definitely tried to recognize a situation where


people were writing about the same topic


or writing with the same content and trying to understand,


does that make this topic or this article more


important than other topics?


But within web search, it's really mostly a matter of,


these are different HTML pages, essentially,


and we find content there and we try to index it.


John, quick question, just jumping in.


First to say hello, long time no see.


Just while you're talking about syndication,


how might geolocation come into that if people are syndicating,


let's say, an English article across different locales,


Australia, US, Great Britain, could there


be an impact there with geolocation also on that?


I suspect that could be the case, yeah.


I mean, when there are situations


where we recognize that people are searching for a piece


of text that's included in a lot of other articles,


then we try to pick which one is the most


relevant one for the user.


And if we recognize that there are some pages or sites that


are particularly relevant for users in some locations,


then that's something that we might highlight there.


So that could also be playing a role there a little bit.


I don't know, if something is originally released in the US


and a UK news site picks it up and you're


also located in the UK, then maybe you


would see the UK news site.


All right, Robin.


Yes, hello.


So we have a bit of a conundrum here.


I just posted a question in the comments,


but I don't think you have responded yet.


I joined seven minutes late, sorry.


So we have a big site with a global footprint


and a lot of different English variant language sites,


which are located in different subdirectories on our domain.


And we are using self-referencing canonicals


and the correct hreflang tags for each of the countries.


So we are specifying that the Australian site, for example,


is using Australian English.


It is the fact that we do have a bit,


since the sites are virtually copies of each other,


we are using one English language master site


and then allowing each country to localize the country


as they see fit.


But in most cases, they don't do that


because we have, for example, product pages which


are identical in all countries.


So we can see in Google Search Console


that there are a lot of exceptions


due to Google either choosing another canonical


than the one that we have specified.


So initially, we thought this would be a huge problem


because we would not have any visibility,


any local visibility.


But I have done some tests where I either


tried to use an Australian IP to Google on Google.com.au


and I still, and the same for the American site,


and I still get the US sites for pages


which shouldn't be indexed.


So the page isn't indexed according to Search Console,


but it still is the URL in Google Search,


which is a bit strange.


So my question is basically twofold.


Is there any way we can get around


so we could have our pages indexed without Google Search


Console complaining too much even though the sites are


technically duplicates?


And if we can't do that, how is Google evaluating the ranking?


Because I think that some domains will have more links,


for example, external links.


The American site is much bigger than the Philippine site,


for example.


And so the American site would have much more links to it


if that page is not indexed, but the Philippines page


is the one that is indexed.


I see my theory is that the Philippine version would


have a lower Google rank even though,


but it's the one that's the template for our rank


on the search results page even though it


displays the American page.


Yeah.


I think this is a super complicated topic,


and we make it extra complicated, I think, in Search


Console, unfortunately.


So it makes it, I don't know, super confusing.


So basically, what is happening is


we're recognizing that I don't know your site,


so I don't know 100% sure if this is actually the case.


But usually, this is what happens.


We recognize that the content is the same.


So you have the same English language content


maybe for, I don't know, Australia, New Zealand,


or something like that.


And our systems will then understand


that these are duplicates, put them into one cluster,


and then we will pick one canonical URL


to use for indexing for that.


We do pick up the hreflang annotations on these pages.


So we understand the connections between the URLs,


but we still index them as one canonical URL.


And in the search results, when you search,


if we understand the hreflangs properly,


then we will show the appropriate local URL


in the search results.


But we will base it on the canonical index version.


So in the snippet, you might see signs that, oh, this


is that canonical version, but the URL


should be the right one.


And the ranking should be based on whatever for that local URL


should be relevant.


So it's not the case that we would pick,


like if we had the Philippine version as canonical


and all the links go to the US version,


then when we put them into a cluster for canonicalization,


then essentially whoever is shown gets those links.


So it's not that those get lost.


But the tricky part is in Search Console,


we report on the canonical URLs for the most part.


So in both the index coverage report


and in the performance report, we


will simplify things for you and show the canonical URL.


So if in Australia, like, well, take, for example,


a situation where you have the Philippine and the US site,


and we pick the Philippine version as canonical


for whatever reason, in the US, you


would probably still see the US URL


if we understand hreflang properly.


But in Search Console, we would report that


as being the Philippine URL.


So in the performance report, it will say, like, oh,


the Philippine URL is very popular.


And you look at the details in Search Console,


and it'll say it's popular in all of these countries, where


it's kind of the same content that's being shown just


with the local country versions.


And similar in the index coverage report,


we'll focus on the canonical URL.


So if you have, say, a separate subdirectory for US,


then it would look like maybe the US version is not


getting indexed at all, but the Philippine version


is getting indexed.


And that can be very confusing.


But essentially, from a ranking point of view,


that should be working out.


So if we understand the hreflang,


we'll show the right version.


It'll rank accordingly in search.


It's just everything with Search Console,


with the tracking there, is super complicated.


We see this a lot more in Europe with German language content,


where the German language countries are


right next to each other.


And in Switzerland, we have different currencies,


and we can't kind of import things


from Europe, all of those weird extra situations.


And still, we kind of have that canonical issue there as well.


So it's not just specific to English things there.


I don't see this changing in Search Console


in the near future.


So that's kind of something where


you have to keep that in mind and understand


that for your content, one version is being chosen


as canonical.


And the reports that you see in Search Console,


you kind of have to pick them apart


and try to understand them on your own.


To avoid that, the only thing you can really do


is make sure that these pages are not


recognized as duplicates, which would


be to make sure that they're really significantly


localized.


Which if it's an e-commerce site,


and you have the same product across all


of these different country versions,


that's probably not that feasible.


If it's mostly content that you have there,


and it's not a lot of pages, then maybe that


is something that you could be doing.


So those are kind of the options there.


SPEAKER 2 OK, thank you very much.


I think our worst worry was that our ranking was negatively


impacted.


But I did find that one of my UK colleagues


sent me a couple of URLs for changing.


And I think he Googled it, because he did not


send the UK versions.


He sent me the Philippine versions.


That's why I use that as an example.


Is there any way to help Google even more


beyond the hreflang tag to understand that these


are really localized pages?


SPEAKER 1 Not really.


Not really.


I think also with hreflang, especially


if you have a lot of different country versions,


then it is something that sometimes we


pick it up properly, and sometimes we struggle with it.


So that's something where anyone who has, I don't know,


worked on bigger international sites,


you'll see that lots of hreflang gets picked up and used.


And then some parts just Google gets confused and shows


the wrong version.


SPEAKER 2 Yeah, I think we have like 30 English versions


and 10 or so Spanish.


No, even more Spanish.


15 versions or something like that.


Spanish.


SPEAKER 1 I mean, one thing that I usually recommend


is to assume that users might be going


to the wrong version on your site


and to try to catch them on your site as well.


So show something like a banner on top, and it's like, hey,


it looks like you're from the UK.


You want the UK version.


And make it so that they can get there fairly quickly.


SPEAKER 2 OK, yeah.


Thank you.


Yeah, we've thought of that, but we


wanted to know if our ranking, foremost, was negatively


impacted before we did that type of action.


OK, thank you very much.


SPEAKER 1 Sure.


Let me run through some of the submitted questions,


and I'll get back to everyone who's raising hands as well


later on.


Don't worry.


Let's see.


I'm working on the biggest mom advice


site in our country, which was demoted with a core update.


They have a great brand, Search, and a community


of loyal returning users.


So my approach is going through the content that


is deemed the most relevant in Google for queries


from Search Console and making sure that when users come,


the content is the highest quality content,


solves the need behind the query, and is trustworthy.


Is that the right approach?


I think in general, that's a great approach.


And especially when you have a larger website,


you can't do everything at the same time.


But focusing on the parts that are


seen as the most relevant for your website, that


are the most important gateways to your content,


I think that always makes sense.


So that seems like a good approach.


There are various people within the SEO community


who have focused on the whole topic of EAT, Expertise


Authority, trustworthiness.


And they've written lots of really interesting blog posts


and case studies.


So I would also go through a lot of that,


just to make sure that you're not missing anything


obvious that might be relevant for your site as well.


Core web vitals and subdomains, I


think we talked about that briefly.


The Chrome User Experience report data


is based on the browser origin, which is the host name.


So it would be separate for a subdomain.


How long does it usually take to show a web story in Discover


after creating it?


The answer is, it depends, unfortunately.


So it's something where sometimes we


can pick up content very quickly after it was created


and crawl it very quickly, index it very quickly.


Sometimes all of that takes a lot longer.


And Discover, in particular, is yet another level


on top of that.


Because for Discover, we want to make sure


that we're actually recommending something that is really


appropriate for users.


Because users are not searching for something specifically.


So we have to be kind of almost extra careful


with regards to the content that we show in Discover.


So there, in particular, it can happen


that it takes a little bit longer for it


to start showing up in Discover.


It can also happen that it is never shown in Discover.


So it's not the case that all web stories are


shown in Discover.


What would be the difference between the new passage ranking


algorithm and the current featured snippet?


From my understanding, the UI part of this


is completely separate from the ranking aspect there.


When it comes to passage ranking,


it's more a matter of us going into the content


and recognizing that this page is relevant for something that


is kind of hidden away on the bottom of the page.


And then how we display it in the search results


is a completely different question.


So it's not that the featured snippet or longer snippet


or anything like that would be tied to the way


that we do ranking for those pages.


What's the best practice of guest posting?


And is it recommended by Google?


Oh my gosh.


OK.


So from our point of view, it's fine to go off and promote


your content on other people's sites.


So if you're creating content and you're publishing it


on other sites to kind of draw awareness to your website,


to what you're working on, that's perfectly fine.


It's just the links within these guest posts


should be nofollow because, essentially, you're


placing those links on other people's sites.


So from our point of view, that's


something where you're putting these links on other people's


sites, so those links should not be passing any page rank.


And that's essentially the main aspect there.


So going off and creating something


to draw awareness for what you're working on,


that's perfectly fine.


Just make sure that the links don't pass any page rank.


If we get links from a subdomain,


will they count as PBN backlinks?


No, not necessarily.


So PBNs are private blog networks,


which is kind of a technique that, I don't know,


some people, usually spammers, use


to try to create a network of sites that looks natural,


that all tend to link to something that you care about,


with the hope that when Google runs across this,


it thinks, oh, so many people are recommending this page.


We should show it higher in the search results.


And obviously, if you're creating these sites


and you're just putting links there,


we would see that as web spam, or at least


as links that are unnatural, and try to discount those.


So from our point of view, links from subdomains


are perfectly fine.


It's not that we would see subdomains


as a private blog network.


Sometimes sites use subdomains.


WWW is a subdomain, technically.


And PBN backlinks are essentially


something completely separate, where


if we understand that this is a part of a private blog network,


then we would just discount that by default.


And it doesn't matter if it's on a subdomain


or on a main domain or anything like that.


So those are kind of completely separate things.


Just to make it clear, I would not


recommend using private blog networks to promote your website


because that's essentially against our webmaster


guidelines.


What matters the most?


The number of unique referral backlink domains


or the total number of backlinks?


So I don't think we differentiate


like that in our system.


So from my point of view, I would


tend not to focus on kind of the total number of links


to your site or the total number of domain links


to your website because we look at links


in a very different way.


And we try to understand what is relevant for a website,


how much should we weigh these individual links,


and the total number doesn't matter at all.


Because you can go off and create millions of links


across millions of websites if you wanted to,


and we could just ignore them all.


Or there could be one really good link


from one website out there that is for us


like a really important sign that we should treat


this website as something that is relevant because it


has that one link.


I don't know, maybe from like a big news site's home page,


for example.


So the total number essentially is completely irrelevant.


Do you ever consider content published


across various websites sourced from the same press release


as duplicate content across those sites?


We talked a bit about that in the beginning.


Technically, if you're duplicating content,


then that is duplicated content.


We don't penalize duplicate content in our algorithms.


It's just that we would try to pick one version


and show that appropriately.


And the other things that we talked


about with syndicated content are kind of in the beginning


if you want to watch that on the recording.


My home page outranks my internal pages.


How can I fix it?


It's not necessarily something that you need to fix.


So this is something that just happens sometimes.


Sometimes internal pages rank higher than the home pages,


and people complain about that.


But essentially, for individual queries,


we try to recognize what is the most relevant result


from within your website or across the web,


and we try to show that appropriately in the search


results.


And sometimes, that's your home page.


Sometimes, that's a product page, might be a category page,


might be a blog post on your site about a specific product.


From our point of view, that's not something


that we would consider something wrong.


The thing that you can do a little bit,


though, within your website is to help


us understand which parts of your website


you consider to be the most important.


And the best way to do that is with internal linking.


So the clearer you can make it to us