Help with virus/hackers

Please redirect me if this is not the appropriate place for this post.

I have several Debian/Woody/2.4.19 webserver/firewalls at various locations that seem to have been hacked or victum of a worm or virus. It is hard to articulate exactly the symptoms since it quickly brings the system down, but here is what I know so far:

1) There is no more output to /var/log/syslog. The contents of the file is '0'.
2) 'last' works, but with no unexpected ftp or telnet logins.
3) Windows systems on the inside seem to have been infected with the W23.HLLW.ULTIMAX worm that propagates through Windows networking. Samba was indeed running on the servers.
4) If I telnet into the server and 'ls', I get:
ls: uncrecognized prefix: do
ls: unparsable value for LS_COLORS environment variable

But I can su to root.

5) On some systems I rebooted and got the console errors "can't open /etc/console/boottime.kmap.gz", and it can't seem to mount the the filesystem and complete the boot.

The first machine went down last Friday in San Antonio TX last Friday. Then within a few hours two more went down that was on the same DSL providers's network. Today I experienced the problem on a server in Manchester NH.

Can anyone offer any advice or insight?

Joe Briggs
Briggs Media Systems
105 Burnsen Ave.
Manchester NH 01304 USA
TEL/FAX 603-232-3115 MOBILE 603-493-2386
www.briggsmedia.com

Without knowing what services were publically accessible, and internally accessible (since you dont know if someone used something like a windows worm to attack from the inside) no.

cert have good general faq's on cleaning up after attackers and also track attacks for patterns so do notify them.

[SNIPPED...]

It is unlikely that one of those Windows worms or virii affected your system. It is more likely that you simply have a trashed file-system. To check for an invasion, do the following.

(1) Disconnect the network wire.
(2) Boot with init=/bin/bash
(3) fsck each file-system slice by hand. Look in /etc/fstab to get them all.
(4) Execute mount -a to mount all the slices in the correct order.
(5) Examine /etc/inetd.conf (if one exists). If you see an unusual entry near the end, you have been 'rooted'. Newer systems use xinetd and won't get invaded this way.

(6)  Check /etc/passwd for a strange account.
(7)  Check /bin/login for a new file-date.
(8)  Check /usr/sbin/sendmail for a new file-date.
     Check /usr/sbin/inetd      ""
     Check /usr/sbin/xinetd     ""
     Check /usr/sbin/syslogd    ""
     Check /usr/sbin/klogd      ""
     Check /usr/sbin/in.*       ""

If any of these files have recent writes, tar off all user-data and completely install Linux again (from a distribution) from scratch. Do not use a recent backup. It could have already been invaded.

If none of these have recent writes, just change the password on the root account and be happy. You just has some file-system corruption and you can fix up /etc/DIR_COLORS (for your color-ls problem) and fix /etc/profile or /root/.bashrc, /root/.profile to fix the bad environment variables created by these scripts.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.20 on an i686 machine (797.90 BogoMips). Why is the government concerned about the lunatic fringe? Think about it.

Doesnt help you

Wrong. Old xinetd < 2.3.10 has remote root exploits and real ones circulate

Rootkits patch other stuff

Rootkits know about avoiding this

Never do this. You don't know what else has changed on the system. You should always (barring odd exceptions) do a full reinstall. Also clean user executable files if neccessary (roots .login is often archived and people rerun exploits from it...)

See the cert documents on recovering from an attack

If you really want to examine the remains of a compromise, boot from a CD-based distro or something like that, and mount the partitions read-only.

If you don't want to, or have no idea what you're looking at, as Alan said, recover and verify user data, then reformat and reinstall.

Oh, yes. If you were running tripwire, and being good about keeping the database somewhere on read-only media, you might be able to detect file modifications. Place emphasis on might.

I'm trying to think up one of those odd situations ...

-- John E. Jasen ([email protected])
-- User Error #2361: Please insert coffee and try again.

Also, note that any data stored on that machine is potentially compromised, such as passwords for other boxes, etc. You should really also change all of those. If the box was a firewall, the rulesets also become known, etc.

I've often wondered whether it would be worth connecting a very large serial EEPROM to a serial port interface, and have it effectively appear as a solid state printer, (to that you could cheaply log to an unmodifyable device). Has anybody ever tried this?

John.

I suspect there are better solutions; namely another host running something like passlogd with the xmit wires cut and the hosts sending udp broadcast messages (or plug it into the monitor port of the switch).
Also, since the poster was running Woody, apt-cron'ing security might
also lend itself to usefulness, along with apt-listchanges and maybe a little expect script.

rgds,
Chris

Linux supports console on printer. Its not totally foolproof (there is a famous story of someone who simply reprinted the past two days of logs edited so the admins wouldnt realise when they looked) but it works pretty well. Just use a dot-matrix printer save keeping HP, Lexmark or Xerox in business 8)

!!! You can't be serious :-)

Aren't you concerned with all of the trees that will be cut down to make that paper, though?

I think 1 tree = about 50 reams. Let's say you get through a ream a day, that's a tree every couple of months!

Maybe there is a way to encode the data in the rings of the tree while it's still growing, that would be the ultimate WORM device :-) :-) :-).

John.

Doesnt help you

Wrong. Old xinetd < 2.3.10 has remote root exploits and real ones circulate

Rootkits patch other stuff

Rootkits know about avoiding this

Never do this. You don't know what else has changed on the system. You should always (barring odd exceptions) do a full reinstall. Also clean user executable files if neccessary (roots .login is often archived and people rerun exploits from it...)

See the cert documents on recovering from an attack