Cyber Attack and Server Upgrade

As those of you who tried to access over the past few days know, this system has been down for maintainance. Now it's time to let you know the why and what for.

A week or so ago there was a security announcement stating that the Apache Web Server was vulnerable to DoS (Denial of Service) attacks on UNIX and complete break-ins on MS platforms. I don't know why anyone would want to DoS this site, so I wasn't very concerned about it. I had been experimenting with Apache 2.0 and Tomcat 4 on another machine, but hadn't quite gotten them to communicate with each other, yet. So I wasn't really ready to upgrade.

My plans went out the window on Monday morning (JST), however. I awoke with a warning from my IDS (Intrusion Detection System) that an attack was taking place. I did some poking around in my log files and found that a user of Mid-Hudson Communications, Inc. in Albany, NY was attacking me as I was looking at the logs! I quickly setup my firewall to block everything from Mid-Hudson's upstream provider, and the attack came to a screaching halt. Based on the logs, the MO of the attack was as follows:

1. Find sites "powered by FreeBSD" from Google.
2. Request a bad page to get an error page that lists the web server name and version.
3. After visually confirming that the site is running an exploitable version of Apache, run a kit that attempts to break in.

After close to an hour, the attacker's tool still hadn't broken into the site, so I got off lucky. However, this looked like the initial warnings about Apache were wrong. So I investigated the matter further.

What I found was that a "Grey Hat" ("White Hats" are hackers who help out by pointing out flaws, "Black Hats" are scum who take over and/or deface computers for personal gain, "Grey Hats" are hackers who are kind of in the middle, warning of problems yet distriuting tool kits that help "script kiddies" - brainless kids, usually teens out of school, who can't code themselves - cause mischief) had released a kit that showed that UNIX was vulnerable to break-ins the previous weekend. And I didn't pay attention to security bulletins over this past weekend!

Nonetheless, the first thing I did was shut down the web site. If there is a security hole, I would be as neglegent as the 4+ CodeRed/Nimda attackers I see daily to allow it to stay open. (And Microsoft is neglegent to continue to sell software off the shelves with multiple security holes out of the box! They should recall their software as auto manufacturers do.) I upgraded Apache to a safe version and put out a notice to let you all know before I started work. After my day job, I worked until 1:00 in the morning trying to get it all going again. And I was so close (in hind sight). The final step I had forgotten to do was to set the owner of the files I had restored to be that of the "user" running the web server. A trivial mistake, but one that cost me a couple of days. (Isn't it always some bone head reason like "Is the power plugged in?")

Anyway, I feel that full disclosure of this sort of thing is best. That's why I'm letting you all know what happened and what I did about the incident. I can find no evidence that the attacker gained access to the system and/or accessed the database. And I promise to take seriously any security bulletins for software that I'm using, even when they're listed as a low risk, as this one was. I will gambare to protect your data.