It was always available as a more or less hidden feature, but is has been upgraded to a separate report.
When you create a new campaign, our SEO Effect crawler bot jumps into action to search and map your site. Depending on your crawl budget in time and number of links — which in its turn depends on your campaign level — our bot finds a number of web pages.
This results are used to use the internal link structure in individual page scans that you can do in SEO Effect. While crawling, the bot can also encounter pages with errors like ‘404 not found’ or other errors. We combine these errors together with the errors from Google Search Console (previously Google Webmaster Tools) into a nice overview of errors that need solving.
If you fix these errors, your internal link juice distribution will improve. Because you remove dead ends (or dead links) the linkjuice is not wasted but passed on the pages that matter.
How do I re-crawl my site to actualise the crawl report?
- Go to Reports > Crawl Errors
- Check the report en especially its actuality: “Last crawl dd 18-04-2015 in the example above”.
- Start a re-crawl by hitting the button ‘Recrawl domain’.
- Wait until the re-crawl is done and export the the crawl errors.
- Open the export in e.g. Excel. Ad an extra column and not for each error what the correct (or new) target for that URL should be.
- Hand the list over to your web developer and ask them to turn it into a list of 301 redirects for your CMS.
Extra: SEO Effect re-crawl now keeps your page-keyword combinations
Another improvement: If you had made combinations of keywords and URLs for your page scans, than these combinations are now kept during a re-crawl. Provided that the URL is found again of course. This used to be a pain; after a re-crawl al the connections had to be redone.
Now suppose that a URL (matched to a keyword) cannot be found, then you have three options:
- Fox the crawl error (if the page was indeed not working) do another re-crawl and SEO effect will find your page again.
- Match the keyword to another URL.
- If the URL does exist but cannot be found by our crawler for some reason, please add the URL manual and connect the keyword again to that page.
After the the re-crawl your can find the lost pages (if any) via: Site optimization > Connect keywords to URLs:The link above the table on the right.
