Motivation
About the main author
My name is Jรกn Regeลก, and in my professional practice at SiteOne I have led the development of dozens of large-scale projects over the course of 19 years. I led teams, wrote analyses, communicated with clients, designed extensive relational databases, optimized our company processes, designed technical frameworks, optimized, secured, programmed web and desktop applications, both in frontend and backend. But mainly the backend. My big hobby was programming code generators or various tools to support debugging between layers of our stacks, class mapping between different technologies, etc.
In addition to leading development, I also managed the infrastructure for many years, assembled and managed Linux servers, designed and implemented various cluster solutions consisting of open-source technologies or managed situations during large-scale DDoS attacks on our clients, designed and implemented better protections. I have never been a proponent of overpriced commercial blackbox solutions, I have always preferred a solution that I understood in detail and could adapt as needed.
My experience, responsibility for production operations and a relatively rare combination of hard skills gave me the ability to understand the consequences of technological decisions, when I can quickly imagine and calculate what a new application feature will mean for the load on HW, disk controllers, what problems can be caused by network latency, transmission speeds between servers, high availability risks, etc. I feel sorry for todayโs IT generation that these overlaps are not built at all, which has a significantly negative impact on the resulting solutions, their management and OPEX costs.
In recent years, however, due to team management, I have been programming very little, and within technology, I have been active mainly in the field of server management.
Even though I only code marginally these days, I still consider myself a good web development craftsman who can help :-)
Motivation to create this project
A few months ago, our first son was born extremely prematurely. He weighed only 750 grams and we visited him in the hospital every single day for 139 long days. It was, and still is, a very challenging period of life for me and my family. Thousands of kilometers in the car, receiving stressful news and hundreds of hours sitting next to the incubator forced me to think about my life, family, work, our meaning in this world, the power but also the fragility of life, the power of self-realization and similar topics.
I realized that I have a lot of experience and knowledge that I would like to share. Iโve contributed to various projects throughout my life, but it didnโt seem like enough. I wanted to create something that would be useful for many people, something that would be free, something that might still be used with pleasure when my son grows up. A bit naive, I know :-)
When developing web projects, we very often needed and used various tools, both paid and free, that helped us check the website for various types of flaws. But none of them did everything we needed, or not in that depth.
Despite a minimum of sleep and free time, I decided that over the course of a few weeks I would code a tool that would be useful for people in the field of web project development and operation. A tool that I, my great colleagues at SiteOne, and those interested in the community will be developing for years to come.
Todayโs initial version is far from perfect as I was making it up as I went along. It will also require continuous refactoring, but we will do everything with sensitivity so as not to change API interface and CLI parameters. Or at least not in the same major version.
I want the use of this tool to be an automatic part of the development and operation of the websites that we at SiteOne create for our clients. At the same time, I want this tool to be used as widely as possible by teams and individuals around the world, so donโt hesitate to send us your feedback or ideas for improvements.
We are happy to implement what we consider beneficial for more people. And as developers, you can actively participate in the development of the project yourself. Although it is not perfect, during development I thought about simple extensibility, and it is quite easy to develop e.g. other analyzers or exporters. At the same time, the crawler can start an internal configurable websocket server and continuously send crawling results to your application.
Note: in other texts I address the authors as โweโ - this is because, although I wrote most of the code myself, some colleagues helped me at the level of feature consultations, preparation of visual parts, logo, etc., and at the same time I want to involve other colleagues more in the future. I thank them for this and look forward to further cooperation ;-)