Skip to content

Performance Analysis

The Performance Analysis offers detailed insights into the loading times of individual URLs. This is crucial for developers and site administrators to identify performance bottlenecks and optimize web pages.

  • Request Duration: Displays the time taken for each URL to load, with sortable columns to prioritize performance optimization efforts.
  • Top Fastest URLs: Highlights the best-performing pages.
  • Top Slowest URLs: Identifies the slowest-loading pages, indicating areas that may need improvement.

When running the crawler, you’ll see loading times for each URL in the main table:

URL | Status | Type | Time | Size | Cache | Access. | Best pr.
---------------------------------------------------------------------------------------------------------------------------
/ | 200 | HTML | 34 ms | 45 kB | 60 min | 1/1/2 | 1/5
/features/content-type-analysis/ | 200 | HTML | 16 ms | 41 kB | 60 min | 1/3 | 1/5
/favicon.svg | 200 | Image | 17 ms | 680 B | 12 mon | |
/_astro/ec.j8ofn.css | 200 | CSS | 15 ms | 18 kB | 12 mon | |
/robots.txt | 200 | Document | 10 ms | 135 B | 60 min | |
/features/exports-and-reports/ | 200 | HTML | 479 ms | 42 kB | 60 min | 1/3 | 1/5
/siteone-crawler-app-demo.gif | 200 | Image | 956 ms | 3 MB | 12 mon | |

In the HTML report, you’ll find dedicated sections for performance metrics, showing both the fastest and slowest URLs:

URL | Response time | Size | Content type
--------------------------------------------------------------------------
/siteone-crawler-app-demo.gif | 956 ms | 3 MB | Image
/features/exports-and-reports/ | 479 ms | 42 kB | HTML
/siteone-crawler-command-line-demo.* | 471 ms | 8 MB | Image
/siteone-crawler-command-line-demo.* | 347 ms | 4 MB | Image
/siteone-crawler-command-line-demo.* | 70 ms | 1 MB | Image
URL | Response time | Size | Content type
--------------------------------------------------------------------------
/robots.txt | 10 ms | 135 B | Document
/_astro/page.LS5KDvwX.js | 10 ms | 2 kB | JS
/getting-started/quick-start-guide/ | 13 ms | 48 kB | HTML
/features/performance-analysis/ | 13 ms | 41 kB | HTML
/features/heading-analysis/ | 13 ms | 40 kB | HTML

You can configure the limits and thresholds for these lists using the following command-line options:

  • --fastest-urls-top-limit=<int>: Number of URLs in TOP fastest list (default is 20).
  • --fastest-urls-max-time=<val>: Maximum response time for an URL to be considered fast (default is 1).
  • --slowest-urls-top-limit=<int>: Number of URLs in TOP slowest list (default is 20).
  • --slowest-urls-min-time=<val>: Minimum response time threshold for slow URLs (default is 0.01).
  • --slowest-urls-max-time=<val>: Maximum response time for an URL to be evaluated as very slow (default is 3).

By analyzing these metrics, you can determine why certain pages perform better than others and implement strategies to enhance overall site speed and user experience. The results help identify:

  • Resource-heavy pages that need optimization
  • Slow-loading assets that should be compressed or lazy-loaded
  • Fast pages that can serve as performance benchmarks
  • Correlations between content types and loading performance

In the future we want to implement aggregation and statistics of average/median responses, e.g. based on the recognized page type (press release, hotel detail, country detail, etc.). This can provide insight into which page types suffer the most from poor performance.