Google Indexation Bug Starting Around Mid-April 2020

I've been having issues with indexing and according to my SC it started around the start of April. As it's a new site I never noticed until the end of May. Most pages on the site didn't get indexed until I manually submitted them with the URL tool. There are still 59 excluded pages out of over 200, and when I use the cache: operator it looks like they are not indexed. Hopefully it is fixed now and I'll see the rest of them pop back into the index.
 
Quick update. When I create a new page and request indexing through GSC, there is a delay of a couple of days until the page is shown as indexed (according to GSC). At that point, the new page doesn't actually appear to be indexed or shown in the SERPs (despite GSC's suggestion that the page is indexed) until a few days later. I am experiencing a similar delay with respect to updates on existing pages (e.g. title tag), whereas before, the updates were nearly instantaneous for me. In short, I'm waiting about a week or so for indexing updates.
 
This could explain why a post I made months back has gained no traction, despite being indexed.
 
Ryuzaki said:
EpG4dCt.png
Click to expand...

Regarding the "stale search results". I know this is from June 2, but I've seen very little movement in the SERPs in the last 2 weeks (monitor around 400 websites). Curious about what @CCarter @eliquid are seeing over at SerpWoo?
 
I saw that John Mueller said the other day that there were no indexing problems and he doubled down on that today, saying that most people are complaining about stuff they don't understand. That we now have more information in Search Console and that's confusing people. Google tells us content they encounter but choose not to index and not all of that should be indexed. I understand most people are moving too fast or are semi-stupid, but we're not all that stupid.
 
Ryuzaki said:
I understand most people are moving too fast or are semi-stupid, but we're not all that stupid.
Click to expand...

Agreed. This level of idiocy lends a ton of credence to your and I think @CCarter's views on marketing, branding and creating a business instead of just relying on SEO.

I'm back to experiencing semi-significant delays. My guess is this is either a bug or an intentional play on Google's part.

Edit: I've experienced this before with a page (one of the ones I was complaining about earlier) that was finally indexed (following a delay) and the page is ranking for two of the main terms based solely on on-page and the domain's overall authority in a competitive niche. I mention this because it tends to negate Google's suggestion that I'm confused and also the folks who tend to think this is only an issue if your site's garbage (although my site could use some work and is undergoing repairs).
 
Last edited:
Ryuzaki said:
Google tells us content they encounter but choose not to index and not all of that should be indexed.
Click to expand...

I dunno, every indexation problem I’ve come investigated always has glaring branding/marketing problems. Like incomplete social profiles, no strong interlinks to the un-index pages, or simple stuff like zero images within a blog post.

Someone is going to say why are social profiled important? Well have you Google any serious brand lately? They have their social profiles in the Knowledge Graph- that means Google is using social activity at some level to measure your brand.

Google SW and you will see a YouTube carousel or Twitter carousel and other elements that show literally we are active in those platforms. When we tweet less Twitter carousels show up less.

You have to have a rounded out brand presence online, it literally is a signal, why else would Google have these specialized carousels??

It’s like one customer that didn’t think their mobile Pagespeed score of 9 out of 100 was the reason they were tanking since the September 2019 update, magically when they improved their mobile to 70/100 the May 2020 update rewarded them.

It’s 2020 folks, WTF.

You can’t build weak affiliate sites with zero presence outside of Google’s index and expect Google to give a shit about your weak-ass brand - BestLiftChairsForSeniors.com will never be a serious brand.

Some people are very lazy with their social, their tweets are automated from a wordpress plugin and have zero interactions, just a title and link to the latest blog post - basically spam. If a surfer can’t tell the difference between your tweets and a spambot then it’s basically spam.

Even if you don’t believe social is a signal, it is a factor at some level of a brand signal, otherwise Google wouldn’t create the carousels nor have the profiles in Knowledge Graphs.

I’ve seen people link to blank Pinterests - why?

I don’t particulate on Facebook, so there is almost zero social signal from there, but Twitter and YouTube I’m all over it. You don’t have to participate everywhere, but you have to be somewhere other than Google’s index for Google to care about your brand.

Based on all the information I’ve gathered and people who have shown me their sites, I believe John Mueller, there isn’t an indexation problem - the sites simply have a branding problem. Hate me or whatever, a lot of you know the truth deep down. Most are barely making an effort outside of SEO, barely.
 
OkoA9dI.png


I think it's safe to say that they're very confident there's no indexation problem. If you're having these issues, as many obviously are, it's probably time to turn your attention to reasons on your own side why this might be happening.

Google may be indexing less content in general as the size of the web keeps exploding. Perhaps they want a page to gain a certain amount of page rank before it's worth indexing, unless it's on a "trusted domain" like a news site. Hard to say.

If you're having these problems, what can you tell us about the domain? Is it newer? Does it have many links? How much content is on there that's indexed? How much older are the indexed ones than the ones that won't index? Etc and so forth.
 
I think I’ve figured out the problem with my site.

When I get content from my writers, I upload it as soon as I get it, with no formatting. I’ll check the headings and grammar, but that’s it. The idea is to let it age in the index (ironic) before I get around to formatting it.

When I first noticed the indexation issues, about 80% of my content was unformatted. Now there are only 20 blog posts that are not formatted. Every other post now has images, videos, tables, and internal links.

As of June 23, there are only 50 pages in the ‘crawled – not currently indexed’ category, out of over 200. I’m waiting for Search Console to update and expect more to be indexed, as I’ve seen an increase in traffic.

I think you may be correct. The web is getting bigger and Google only wants to index quality content. Words on a page with no images, videos, and other visual content is not quality. Perhaps a site with more authority can get away with it, but my site is only 6 months old.

CCarter said:
I dunno, every indexation problem I’ve come investigated always has glaring branding/marketing problems. Like incomplete social profiles, no strong interlinks to the un-index pages, or simple stuff like zero images within a blog post.
Click to expand...

Spot on, as usual.
 
Yes, good to get into a habit of linking internally, with the intent of actually funnelling traffic, not just linking.
 
CCarter said:
I’ve seen people link to blank Pinterests - why?

I don’t particulate on Facebook, so there is almost zero social signal from there, but Twitter and YouTube I’m all over it. You don’t have to participate everywhere, but you have to be somewhere other than Google’s index for Google to care about your brand.
Click to expand...
For those who have this problem, I've been moderately active on Pinterest with one site. Google lists the Pinterest links from more than ten different country Pinterests as part of the backlink profile of that site on GSC. (I realise that the links listed in the Search Console are not reliable or comprehensive or even, traditionally, the most valuable links, but I find it interesting nonetheless.)

On another new-ish site with comparatively poor backlink profile, each fresh post is immediately posted with picture, précis and link to Facebook and promoted to an interest audience for a couple days. (Nowadays, they each tend to get 200+ genuine likes, a few shares, and a few visits.) There has never been a problem with Google indexation with this site.

It would be interesting to know if any of the sites with indexation problems actually had genuine social media activity and interaction. From my own experience I would strongly agree with @CCarter's quote above.
 
Ok, so now I experienced this.

I published a post on the 17th and by yesterday it hadn't been indexed. It was on a "weak" domain, that otherwise does well for its posts. It only has like 5 pages or so, but they're ranking well.

I asked Google to index it yesterday in Search Console and today it is indexed.

If it was really not a bug, why would Google instantly index it, when I added it to Search Console? It already gets traffic too.
 
The saga continues. Google said they did have indexing problems, then claimed they don't, and now they went on record on a podcast saying...

They said that they didn't "have an indexing problem" but they had a "crawling problem" where the crawlers went into hyperdrive and fed way too much data to the indexers, causing those to bog down. Sounds like an indexing problem to me.

Leave it to Gary Illyes, the hero who comes out and tells the full truth when needed, to set the record straight. He also said Google now has monitors set up for Twitter to scan for people complaining about indexing issues (meaning they can't spot it internally?)

All this has done is taken me back to the Matt Cutts days where I believed that I couldn't believe a single word they were saying. Double speak, vague statements, sometimes straight up falsehoods (in my opinion, don't sue me), etc. John Mueller is the new Matt Cutts, for sure.
 
I just noticed this in Google Search Console:

The "Request Indexing" feature of the URL Inspection Tool has been disabled in order to make some technical updates. We expect it to be re-enabled in the coming weeks. In the meanwhile, Google continues to find and index content through our regular methods, as covered here.
 
Yeah, they just had another round of indexing problems and this time admitted that that's what was going on. Had to do with the mobile index. I think it wasn't just getting new pages indexed but existing ones falling out of the index too.

Random note, but I read that the desktop index is going away entirely. So if a site isn't responsive or at least have a crappy m-dot version then it won't be included in Google's index any more. I agree with this move.
 
Lots of weird stuff happening. Last month's page updates that were weeks ago indexed and changed just reverted back to in SERPs.

Be careful about making any serious SEO on-page changes until Google figures itself out.
 
NetZero said:
Lots of weird stuff happening. Last month's page updates that were weeks ago indexed and changed just reverted back to in SERPs.
Click to expand...
I'm seeing older versions of pages jump back into the index. Like the http version of a page when the site has been moved to https for the last number of months. Very odd.
 
Is the indexing bug back or just for me?

I had a couple of pages drop out of the index for no apparent reason. GSC says they are indexed, but they're not.
 
I wasn't effected by any indexing bug until now, GSC is telling me some page are set to noindex (they're not) and it's not indexing them.
 
You must log in or register to reply here.
Back