Website crawler

Crawl a whole website and get an SEO issues report, a score out of 100, and a spreadsheet of every page: titles, descriptions, headings, word counts, links, contact details and product prices.

Website address Maximum pages (1 to 10000)

Product directory options

Only this section of the site Full product list, fast – every product address in the shop's sitemap, with names and product codes, in a minute or two. No prices. Visit product pages for prices (name, price, SKU, brand, stock) – about 4 seconds a page

Big shops can have tens of thousands of products. Get the full list first, then visit one section (for example /boilers/) for prices.

Compare with a competitor

Competitor's website address

Both sites are crawled with the same settings and shown side by side.

Start crawl

The crawler follows robots.txt and waits a few seconds between pages so it doesn't overload anyone's site. That's about 4 seconds per page, so large sites take a while. Keep this page open while it runs.