Skip to main content

21 posts tagged with "technology"

View All Tags

Running web scraping service securely

· 7 min read
Aleh Zasypkin
Creator of Secutils.dev

Hello!

In my previous post, I shared the update regarding the upcoming "Q3 2023 - Jul-Sep" milestone. While I briefly covered how I implemented the notifications subsystem in Secutils.dev, there are a few other important changes I've been working on for this milestone. One of these changes is related to the fact that I’m preparing to allow Secutils.dev users to inject custom JavaScript scripts into the web pages they track resources for (yay 🎉). As a result, I've spent some time hardening the Web Scraper environment's security and wanted to share what you should keep in mind if you’re building a service that needs to scrape arbitrary web pages.

Q3 2023 update - Notifications

· 5 min read
Aleh Zasypkin
Creator of Secutils.dev

Hello!

With just one month remaining in the "Q3 2023 - Jul-Sep" milestone (this is how I structure my roadmap), I wanted to provide a quick progress update. A significant deliverable for this milestone includes adding support for email notifications and other transactional emails.

Notifications, in general, and email notifications, specifically, are integral to any product that involves any monitoring or tracking activities. Secutils.dev already includes, and will continue to expand, features that require the ability to send notifications. Two notable examples include sending notifications for changes detected by the web page resources trackers and changes detected in the tracked content security policies (CSP).

Building a scheduler for a Rust application

· 5 min read
Aleh Zasypkin
Creator of Secutils.dev

Hello!

As you might have learned from the "A Plan for the Q3 2023 Iteration" post, my focus for this iteration is on adding support for automatic scheduled resource checks for the "Web Scraping → Resources trackers" utility in Secutils.dev. This work is already in progress, and in this post, I'd like to share more details about how I'm designing the scheduler for Secutils.dev. If you're building a scheduler for your application, hopefully, you can learn a useful thing or two.

A plan for Q3 2023 iteration

· 4 min read
Aleh Zasypkin
Creator of Secutils.dev

Hello!

In one of my previous posts, I mentioned that I wrapped up the "Q2 2023 Apr-Jun" iteration and moved on to the next one: "Q3 2023 Jul-Sep". In this post, I want to briefly cover what I'm going to work on during this iteration. I'll highlight three main areas: improvements to the certificate templates, scheduled automatic web page resources checks, and shareable content.

Q3 2023 Jul-Sep iteration

Detecting changes in JavaScript and CSS isn't an easy task, Part 3

· 6 min read
Aleh Zasypkin
Creator of Secutils.dev

Hello!

This is the third and final part of a series of posts (part #1, part #2) where I explain why comparing JavaScript and CSS files isn't as simple as it may initially seem. Additionally, I'll share how I tackled this problem for the Resources Tracker utility in Secutils.dev.

In the previous posts, I covered various challenges, including handling both inline and external resources, dealing with dynamically loaded and frequently changing resources, and comparing data and blob URLs. Today, I'd like to discuss the security-related challenges you should be mindful of if you're planning to build a similar tool like the Resources Tracker utility.