Hura, our link data service provider MOZ has made available a 25x larger link index from now. As a result, the link data that you can use in SEO Effect tools is again competitive with the other big link boys and girls.
We have worked hard again to connect the new api. This has been available from June ’18.
What does this mean for you as an SEO Effect user?
This means that you will see more links in the following reports:
- The report: ‘Own link profile‘:
- Table; the moment you run the first update from now
- The report ‘Link profile competitors’; at this month update of June
- De tool: ‘Find links based on link profile competitors‘
The data transition is marked in the graph so that you later understand where the jump came from.
Unfortunately, for the time being, the spam flag points are no longer available. So you still see the spam score of your site and that of your competitors, but you can no longer see the details of the why. Moz has announced that this data will be available again. Ian, head development at Moz assured me the following:
You are correct about the Spam Score details. We have completely red our Spam Score model, including the flags we are crawling for. For now, we will not have that Spam Score Flag data available, but do plan to add it in when we can. Sorry if this causes any issues I know that was useful information. Our new Spam Score will be accurate and crawled on a more regular cadence, so that metric will become much stronger.
Back to basics; how does such an index work again?
Before I explain the differences and why it is usefull, back to the basics. How does such an index crawl work again and how ‘true’ is it?
As you know, the links – number, quality (on topic and authority) and eg spread over domains – are very important for your rankings. Google indexes a part of the web with its bots. This means that all content is ‘scraped’ with all internal and external links in it.
With this Google builds its own network, a gigantic web of sites that are connected through the links. but Google does not see everything. That is also called the ‘dark web’. (dark web is also used for all those dark trade sites that do not tolerate daylight but originally it just meant the unknown part of the internet.)
The Google link truth
In Google search console you can see which links Google has found on your site. If you got a new link yesterday, Google must first have discovered that content if the bot wants to notice and follow this link.
Only then can it take into account the link score and therefore for your ranking. Google has been crawling more and faster over the years. But also in Google you are not vieweing 100% of the truth.
The MOZ link truth
The same is true, of course, for other scrapers like those of Moz, Majestic or Ahref. They only see part of reality. A part will overlap with what Google knows and part will be unknown to Google and vice versa.
Just a few points on how Mozscape compile there index:
- They grab the most recent index.
- They take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains).
- Then they start crawling from the top down until we’ve crawled ~130 billion URLs
Thus the explanation of the filling of the Moscape index.
I asked myself the following:
When I fully understand the crawl process starting with the top 10 billion URLs, smaller languages with have by nature lower MozRank will be underperforming in this index?
Is this right?
In The Netherlands, there a re a few sites with Domain authority of 70+, but not a lot, the most are lower then 30…
Tawny from Moz replied:
That seems like a safe assumption, Keesjan.
Sites with fewer established links will have lower authority metrics, and since our index mostly covers English-language sites and sites based in the US, UK, and Canada, it won’t cover every link for sites in places like the Netherlands, unfortunately.
Hopefully the new crawler will span a much broader range of sites!
And it did! the new crawler is 25x bigger then the old one.
Differences with the link data from before May 2018
The differences at a glance:
- The index has become 25x bigger
- the index is each day fresh in versus a month of offline data.
- Because the index is always updated every day, the DA and PA values are up to date and you are no longer looking at the past.
- The fresher DA score is therfore a better predictor of your ranking score.
- As more sites get a DA score, the average DA score decreases.
Because the index is 25x as large, many more links are found in the Dutch language area. That was sometimes a problem. For example, for our sister site Interneteffect in the previous version 454 links were shown t.ov. 1073 in the new version.
Compared to the indexes of the competition, we see that SEO Effect tools is doing much better now. Better than Ahrefs and the fresh majestic index.
|MOZ/SEO Effect||Majestic Fresh||Majestic Historic||Ahrefs|
|indexgrootte in trillions *||4,7||1||7||3|
* One trillion is ten to the 12th power, ie 1,000,000,000,000 or 1012. The old index of Moz was 0.13 trillion in size.
Fresh link index data
Previously, the index was refreshed about once every month. If you viewed a competitor’s link profile in SEO Effect at the end of the month, that could already be a bit behind reality. with this update the values are up to date every day.
Fresh DA and PA values
Because every day the index is updated with fresh links, the Domain Authority (DA) and the page Authority (PA) are also refreshed.
The DA score is a better predictor of rank score
Because the DA is fresher, this value is now an even better indication of your rank score.
Because more sites get a DA score, the average DA score decreases
Moz reported that in the new index the DA of Google went from 97 to 94. Always look at the relative value of your scores, compared to those of the competitors. We also see this in the measurements that we record.
It is logical that this happens. The DA is a value of 0 to 100 and is a relative score. The more websites Mozbot crawls, the more the DA score should be distributed. This was also the case with the old familiar Google PageRank. (RIP)
Some more figures with differences in SEO Effect
From a number of sites we compare the number of backlinks in the index. Since January 2017 you can follow the link profile of your competitors in SEO Effect.
- old: 276,727,994
- new: 1,945,730,210
- old: 780,188
- new: 1,055,574
- old: 606
- new: 1,875
- old: 2,035
- new: 44,567
We generally see a factor of 0.5 to 22 more links in individual link profiles. With percentally larger improvements for smaller sites in smaller language areas.
Your action: Discover more link secrets from your competitor
He does not become fatter when weighing a pig
We think this increased linkdata index is a big improvement. Do let us know the difference between the previous and current scores of your site via chat, tweet or comment? Gladly!