SiteOne Crawler

3 videos, more than a thousand words

Desktop Application

Command-line interface

HTML report

First Steps

Install

Desktop Application

For most users, a desktop application that is easy to install and use will be the most suitable.

Download the latest version of the installer from releases, or here are quick links for most common platforms:

• Windows x64 - MSI (recommended) / EXE / portable EXE
• Linux x64 - AppImage (recommended) / DEB / SNAP
• macOS arm64 (M1/2/3) - DMG, macOS x64 (Intel) - DMG
  
  Note for macOS users: As of 12/12/2023, the macOS app is already signed and officially notarized by Apple. We will also strive to publish the next versions to the Mac App Store for easier installation and updates.

Command-line Interface

Use ready-to-use packages or manual installation for you platform. System requirements are minimal.

Run the first crawl

Start with basic crawler usage, see examples and then explore the rich features and dozens of command-line options.

We believe that even after watching the videos above, you already know that using the desktop application and the command-line interface are really very simple.

---

In version 1.0.7, we also implemented a free online service that allows you to upload the HTML report of your website to a secure unique URL for easy sharing. In the desktop application, you only need to check one checkbox, and in the command-line version, you only need to add --upload. More information on the page Online HTML report (upload).

Discover how it can help you

See how this tool can help website owners and consultants, developers, DevOps or QA engineers.

If you are missing some functionality in the tool and would appreciate it, write to us or write a feature request right away.

Share your experience

If you find this tool useful, share it with your friends or colleagues by buttons below. We have a lot of ideas and plans for further development of this tool and we want it to be useful for developers and other users around the world.

Key Features

Ease of Use

We strive to make the tool to be simple to use even for less technical users and easy to install without complex dependencies. Although the tool is intended primarily for advanced users with command-line access, it can also be used through a desktop application with a graphical interface. For the easiest way to share a report with colleagues, you can use free online HTML report upload feature.

Deep Website Crawling

Finds and analyzes every single page, image, font, CSS, JS or document on your website. It is really heuristic and searches even responsive images in all forms of srcset, looks for images or fonts in CSS, even parses and looks for files in the build manifests of some popular frameworks.

Dev/DevOps Assistant

Finds imperfections and non-optimized parts of the website that you neglected during development, helps you warm up the cache, or test the functionality of the entire website as part of your CI/CD pipelines.

Offline Website Generator

Generates a static offline version of your website that you can store and browse offline without webserver. A great tool for archival purposes or robust HA (high-availability) scenarios, where regularly maintained static versions of the website can be part of DR (disaster recovery) plans.

Analyzer and Reporter

It will perform a lot of different analyzes of your website and provide recommendations and various details in a detailed report. Analyzed are SEO and OpenGraph, headings, redirects and 404, performance, accessibility, security, and many other aspects of your website.

Swiss Army Knife

This tool has a lot of features, and we plan to implement more and more useful functionality into it that community members find useful. We want this tool to be the favorite all-purpose Swiss Army Knife of every good craftsman who builds quality websites. To make it a piece in his puzzle of favorite tools.