Site icon SEO Effect

New: better link building based on 25x larger link database

Keesjan Deelstra
link data visualization - SEO Effect

Hura, our link data service provider MOZ has made available a 25x larger link index from now. As a result, the link data that you can use in SEO Effect tools is again competitive with the other big link boys and girls.

We have worked hard again to connect the new api. This has been available from June ’18.

What does this mean for you as an SEO Effect user?

This means that you will see more links in the following reports:

The data transition is marked in the graph so that you later understand where the jump came from.

Unfortunately, for the time being, the spam flag points are no longer available. So you still see the spam score of your site and that of your competitors, but you can no longer see the details of the why. Moz has announced that this data will be available again. Ian, head development at Moz assured me the following:

You are correct about the Spam Score details. We have completely red our Spam Score model, including the flags we are crawling for. For now, we will not have that Spam Score Flag data available, but do plan to add it in when we can. Sorry if this causes any issues I know that was useful information. Our new Spam Score will be accurate and crawled on a more regular cadence, so that metric will become much stronger.

Back to basics; how does such an index work again?

Before I explain the differences and why it is usefull, back to the basics. How does such an index crawl work again and how ‘true’ is it?

As you know, the links – number, quality (on topic and authority) and eg spread over domains – are very important for your rankings. Google indexes a part of the web with its bots. This means that all content is ‘scraped’ with all internal and external links in it.

With this Google builds its own network, a gigantic web of sites that are connected through the links. but Google does not see everything. That is also called the ‘dark web’. (dark web is also used for all those dark trade sites that do not tolerate daylight but originally it just meant the unknown part of the internet.)

The Google link truth

In Google search console you can see which links Google has found on your site. If you got a new link yesterday, Google must first have discovered that content if the bot wants to notice and follow this link.

Only then can it take into account the link score and therefore for your ranking. Google has been crawling more and faster over the years. But also in Google you are not vieweing 100% of the truth.

The MOZ link truth

The same is true, of course, for other scrapers like those of Moz, Majestic or Ahref. They only see part of reality. A part will overlap with what Google knows and part will be unknown to Google and vice versa.

Just a few points on how Mozscape compile there index:

Moz:

“The idea here is that we’re focusing on the highest-quality links we can find, coming from the most prominent pages of authoritative sites. So, while you may not see every link for a site within our index, we’re aiming to report the most valuable ones available!

Most new sites and links will be indexed by our spiders and available in Mozscape and Open Site Explorer within 60 days, but some take even longer for many reasons, including the crawl-ability of sites, the number of inbound links to them, and the depth of pages in subdirectories. This tends to bias our index in favor of newer links. Linking data is only stored in the index for about 180 days, so unless the crawler has a compelling reason to return to your site and the sites linking to yours and rediscover those links, they can fall back out of our index again. That doesn’t mean they’re not out there affecting your SEO, just that our tools don’t see them anymore.

Since Moz focuses on quality of links over quantity, we are always focused on the most relevant links to display to our users. It’s possible that Moz’s index will leave out some of the lower-quality (non-link juice providing) links out of our index because of this. So, that might explain why you may see some discrepancies with what other tools may be showing.”

Read more about the implications of this index mechanism
Less

Thus the explanation of the filling of the Moscape index.

I asked myself the following:

When I fully understand the crawl process starting with the top 10 billion URLs, smaller languages with have by nature lower MozRank will be underperforming in this index?

Is this right?

In The Netherlands, there a re a few sites with Domain authority of 70+, but not a lot, the most are lower then 30…

Tawny from Moz replied:

That seems like a safe assumption, Keesjan.

Sites with fewer established links will have lower authority metrics, and since our index mostly covers English-language sites and sites based in the US, UK, and Canada, it won’t cover every link for sites in places like the Netherlands, unfortunately.

Hopefully the new crawler will span a much broader range of sites!

And it did! the new crawler is 25x bigger then the old one.

Differences with the link data from before May 2018

The differences at a glance:

  1. The index has become 25x bigger
  2. the index is each day fresh in versus a month of offline data.
  3. Because the index is always updated every day, the DA and PA values are up to date and you are no longer looking at the past.
  4. The fresher DA score is therfore a better predictor of your ranking score.
  5. As more sites get a DA score, the average DA score decreases.

Bigger index

Because the index is 25x as large, many more links are found in the Dutch language area. That was sometimes a problem. For example, for our sister site Interneteffect in the previous version 454 links were shown t.ov. 1073 in the new version.

Compared to the indexes of the competition, we see that SEO Effect tools is doing much better now. Better than Ahrefs and the fresh majestic index.

MOZ/SEO Effect Majestic Fresh Majestic Historic Ahrefs
indexgrootte in trillions * 4,7 1 7 3

* One trillion is ten to the 12th power, ie 1,000,000,000,000 or 1012. The old index of Moz was 0.13 trillion in size.

Fresh link index data

Previously, the index was refreshed about once every month. If you viewed a competitor’s link profile in SEO Effect at the end of the month, that could already be a bit behind reality. with this update the values ​​are up to date every day.

Fresh DA and PA values

Because every day the index is updated with fresh links, the Domain Authority (DA) and the page Authority (PA) are also refreshed.

The DA score is a better predictor of rank score

Because the DA is fresher, this value is now an even better indication of your rank score.

Because more sites get a DA score, the average DA score decreases

Moz reported that in the new index the DA of Google went from 97 to 94. Always look at the relative value of your scores, compared to those of the competitors. We also see this in the measurements that we record.

It is logical that this happens. The DA is a value of 0 to 100 and is a relative score. The more websites Mozbot crawls, the more the DA score should be distributed. This was also the case with the old familiar Google PageRank. (RIP)

Some more figures with differences in SEO Effect

From a number of sites we compare the number of backlinks in the index. Since January 2017 you can follow the link profile of your competitors in SEO Effect.

We generally see a factor of 0.5 to 22 more links in individual link profiles. With percentally larger improvements for smaller sites in smaller language areas.

Your action: Discover more link secrets from your competitor

He does not become fatter when weighing a pig

 

Thus a Montana pig farmer. So also get started with link prospecting based on the link profile of a competitor.

We think this increased linkdata index is a big improvement. Do let us know the difference between the previous and current scores of your site via chat, tweet or comment? Gladly!

Exit mobile version