Photo Corners headlinesarchivemikepasini.com
A S C R A P B O O K O F S O L U T I O N S F O R T H E P H O T O G R A P H E R
Enhancing the enjoyment of taking pictures with news that matters, features that entertain and images that delight. Published frequently.
Site Tweak: Squashing Another Bot
10 April 2025
At the end of last month we were alarmed to see our daily bandwidth jump as high as 15-GB. With Sonic's bandwidth protection to avoid extra charges for more bandwidth, that would shut us down for a month in 10 days. So we lept into action early this month.
It was tricky.
This particular bot did not identify itself in our User Agent list. It appeared as just another generic group, although it consumed 90 percent of our bandwidth aw an unresolved domain.
Our IP address block has been active for a few days now and it's working.
So we couldn't block it by name. But there is more than one way to protect a site from assault.
We could block it by IP address. So we dug through the log to see which IP addresses was battering the site and found a range of them, each of which was far in excess of what a normal reader of the site would generate.
Our IP address block has been active for a few days now and it's working. It's down to 0.01-GB from that 15.00-GB high.
We'll keep an eye on it, though. We blocked as narrow a range as we saw in the logs but we could be more aggressive if the hits from the rest of the range increase.
Key to that is our local Perl script to read the log and deliver an HTML report each morning to our browser. We came up with that approach when Sonic retired the FTP server we ran our server-based check on that sent us a daily email to monitor bandwidth and didn't realize we were being battered until our bandwidth protection was about to kick in. Sonic generously waved the extra charges for the month while we scrambled to duplicate the monitoring locally.
So we'll know right away now if there's a fire to put out.