Skip to content

Screaming Frog SEO Spider

SEO Spider is a desktop crawler that checks page addresses and records what each one returned. It helps you trace broken links and redirects across a site you manage.

Visit Screaming Frog SEO Spider

What can you learn from a list of your own pages?

A crawl asks for the pages in scope and records the replies. Use List mode when you already know the old addresses to check. See which return a page, which send visitors elsewhere, and which fail. That is evidence about the crawler’s requests. Use a search engine report for its indexing decision.

How do you keep a small check within its intended pages?

Start from a short list of approved pages and confirm the crawl settings before running. List mode normally starts at depth zero and its default differs from a normal crawl because it can ignore robots.txt. Inspect the settings instead of treating that file as a boundary. Keep private addresses and credentials out of shared exports.

What fits in Free and what can you keep?

The free edition limits crawls to 500 URLs and includes common checks for broken links, redirects and titles. The yearly license adds features such as JavaScript rendering and saving and reopening a crawl. Keep a clean export of the list-mode rows you need, plus the intended destination for each old address.

Which old page needs a redirect fixed?

Draw an old pricing address and the new address on paper. The expected result is one direct redirect to a working page.

  1. List two fictional observations
    One old address goes directly to the new page. Another goes through two intermediate addresses before reaching it.
  2. Choose the fix
    Flag the longer route for review. Keep the original, final destination and observed status together.
  3. Write the check to repeat
    If the site owner later runs a crawl, use only the agreed list and compare the final destination with the expected one.