Those of you who have been on Bullet within the last week or so will know the “fun” I’ve been having with our new firewall. It’s a Cisco ASA5520 with the IDS card installed.
When I first tried bringing the machine into service I found that the machine would crash within 10 minutes. It took quite a while to get a new version of the operating system for it, which seemed to work a lot better.
However, on Thursday it was brought to my attention that ssh connections were no-longer reliable, especially if they were high traffic ones. So, I spent the whole of yesterday investigating the thing with tcpdump running at both ends of the connection and on the IDS (which runs Linux). The packets looked OK within the our network and at the IDS but there was some which were becoming corrupted at the far end, which the Linux kernel wasn’t spotting and was hence getting through to the ssh application. I tried loads of tweeks on the firewall to see if I could change the problem, but it wouldn’t go away.
Anyway, today I decided to test the same thing from home using my Apple iBook. The connection (and rsync, which I was using to test) seemed to work OK, but there were a few pauses. The problem still happened when I tried from my Linux server, however. I then tried the rsync to a different machine in Earth Sciences and it broke.. So, it looks like there are two bugs.. one in the ASA IOS (or there’s duff memory damaging packets) and a bug in the Linux IP stack which stops it from noticing that packets are corrupt. :-/
Anyway, as soon as I get BT SkyNet to move our support contract from them to Cisco (which I originally requested when I purchased the equipment but was given their own support contract) I’ll lodge a bug report with Cisco. Goodness knows how I’ll get a bug report sent to the Linux kernel hackers… maybe I should e-mail Alan.