The only interesting thing about this day… one of Google’s crawlers tripped one of my automatic blockers (by loading my blog login page). So 66.249.66/24 got blocked. And as I expect (and blogged about before), Google is far and away the leading consumer of my web site. Just their crawlers in 66.249.66/24 account for 9,177 connection attempts per day. Yes, some of the SYNs are retries. But just the same… if your web site is around for a while, it’ll get hammered by Google day in and day out.
I already know their crawlers are not particularly smart. For example, I’ve blocked them from pulling PDF files because they grab the same ones over and over (often in the same week), including PDFs that haven’t changed in over 10 years. So I’m going to leave the block in place for a while to see what happens. I suspect nothing; I don’t think the crawlers are smart enough to stop trying, despite getting nothing in response.