Scan 19 Analysis Tyler Hudak pbemfun@yahoo.com Scan of the Month 19 (October) Forward ------- In order to make the log files pulled from the Snort binary capture, I placed a "$" before any of the commands that were run by the intruder to differentiate between the commands run and the responses. The files were not modified in any other way and can therefore be unreadable at times. The tools that I used to analyze the Snort binary capture are: Snort: www.snort.org Ethereal: www.ethereal.com TCPflow: http://www.circlemud.org/~jelson/software/tcpflow/ Analysis -------- The first thing I did was download the scan file and verify its integrity with md5sum. [tyler@localhost honeynet]$ md5sum scan19.tgz 11e0be295d138df14111796a7733a5d2 scan19.tgz The md5sum matches the sum from the webpage, so we are good to go. To get a general impression of what occurred, I ran the newdat3.log log file through Snort to get an idea of what attacks occurred against the honeypot. I also ran the it through tcpflow to get a dump of all the TCP conversations that occurred and loaded the log file into Ethereal to see all of the packets. $ snort -dr newdat3.log -A full -h 192.168.1.0/24 -l ./snortlog $ tcpflow -r newdat3.log The full snort alert file generated can be seen in the snort-logs directory included with the analysis. By looking through the snort alert file we can see that initially an RPC portmap request and RPC statdx exploit was launched against the honeypot from IP address 210.114.220.46. [**] [1:583:1] RPC portmap request rstatd [**] [Classification: Attempted Information Leak] [Priority: 3] 09/15-22:06:06.819252 210.114.220.46:653 -> 192.168.1.102:111 UDP TTL:47 TOS:0x0 ID:41887 IpLen:20 DgmLen:84 Len: 64 [Xref => http://www.whitehats.com/info/IDS10] [**] [1:1282:1] RPC EXPLOIT statdx [**] [Classification: Attempted Administrator Privilege Gain] [Priority: 10] 09/15-22:06:07.719989 210.114.220.46:654 -> 192.168.1.102:919 UDP TTL:47 TOS:0x0 ID:41890 IpLen:20 DgmLen:1104 Len: 1084 [Xref => http://www.whitehats.com/info/IDS442] However, this attack occurred on 9/15 and the honeypot was compromised on 9/16 so this attack was ignored. The next alert generated was a SCAN Proxy attempt, occurring from 206.75.218.84 on 9/16-15:24:25.57. However, by filtering out only traffic to and from 206.75.218.84 within Ethereal, we can see that this IP address was probably only scanning for a few open ports and is not a concern. The next few alerts are bad telnet logins from IP address 217.156.93.166. By running the "Follow TCP Stream" tool within Ethereal for this telnet session, I could see that person attempted to log in to the machine using the nobody user ID twice. The person then started to log in as uucp, but never put in a password as the connection timed out. The entire conversation can be see the file telnet1.txt that is included. Since it was stated that the honeypot had been compromised two times before by the same intruder, this was probably the same intruder trying to reconnect using the login ID's previously put on the machine. By following the packets in Ethereal, we can see that the same IP address then attempts to connect to ports 24 and 6666 with no luck. Less than a minute after attempting to connect to port 6666, an FTP session is opened up from IP address 217.156.93.166. This can be seen in slog2.dat as well as an anonymous user logging into the machine. By running "Follow TCP Stream" within Ethereal for this conversation, which can be seen in ftp1.txt, we can see that the intruder launched an attack on the FTP server. The snort logs, shown below, let us know that this attack is the wu-ftpd SITE EXEC format string attack. [**] [1:338:1] FTP EXPLOIT format string [**] [Classification: Attempted User Privilege Gain] [Priority: 8] 09/16-18:55:52.235847 207.35.251.172:2243 -> 192.168.1.102:21 TCP TTL:48 TOS:0x0 ID:16648 IpLen:20 DgmLen:76 DF ***AP*** Seq: 0xCF7869CC Ack: 0xEBCD7EC0 Win: 0x7D78 TcpLen: 32 TCP Options (3) => NOP NOP TS: 237391678 29673183 [Xref => http://www.whitehats.com/info/IDS453] [**] [1:361:2] FTP site exec [**] [Classification: Potentially Bad Traffic] [Priority: 2] 09/16-18:55:52.552709 207.35.251.172:2243 -> 192.168.1.102:21 TCP TTL:48 TOS:0x0 ID:16651 IpLen:20 DgmLen:468 DF ***AP*** Seq: 0xCF7869E4 Ack: 0xEBCD7EFE Win: 0x7D78 TcpLen: 32 TCP Options (3) => NOP NOP TS: 237391708 29673193 [Xref => http://www.securityfocus.com/bid/2241] [Xref => http://www.whitehats.com/info/IDS317] ... [**] [1:344:1] FTP EXPLOIT wu-ftpd 2.6.0 linux overflow [**] [Classification: Attempted Administrator Privilege Gain] [Priority: 10] 09/16-18:55:59.485710 207.35.251.172:2243 -> 192.168.1.102:21 TCP TTL:48 TOS:0x0 ID:16786 IpLen:20 DgmLen:201 DF ***AP*** Seq: 0xCF78AE1C Ack: 0xEBCE0EB9 Win: 0x7C70 TcpLen: 32 TCP Options (3) => NOP NOP TS: 237392403 29673724 [Xref => http://www.whitehats.com/info/IDS287] This exploit gives the intruder root level shell access. Now that the intruder has gained access, they make sure they are root and see if anyone else is logged in. All of the intruder's commands run can be seen in ftp1.txt. After exploring the file system for a little while, the intruder eventually makes it back to the root level of the file system and deletes the password for user nobody by running "password nobody -d". At this point, the intruder telnets to the honeypot again from 217.156.93.166 and logs in as nobody, probably to verify that they could actually log in. The intruder runs 'w' to see if anyone was logged on and then logged off. This can be seen in telnet2.txt. The intruder then returns to the FTP shell session from 207.35.251.172 and runs the following commands: cd / mkdir -p /etc/X11/applnk/Internet/.etc mkdir -p /etc/X11/applnk/Internet/.etcpasswd touch -acmr /etc/passwd /etc/X11/applnk/Internet/.etcpasswd touch -acmr /etc /etc/X11/applnk/Internet/.etc passwd nobody -d /usr/sbin/adduser dns -d/bin -u 0 -g 0 -s/bin/bash passwd dns -d touch -acmr /etc/X11/applnk/Internet/.etcpasswd /etc/passwd touch -acmr /etc/X11/applnk/Internet/.etc /etc This is where the intruder attempts to hide his edits from the MAC times. He first creates two directories, /etc/X11/applnk/Internet/.etc and /etc/X11/applnk/Internet/.etcpasswd and sets the directory's MAC times to be the same as /etc and /etc/passwd using the 'touch -acmr' command. He is essentially saving the MAC times of /etc and /etc/passwd. The intruder then proceeds to delete the password for nobody again, add a user named dns with root privileges and delete the password for dns. Finally, the intruder copies back /etc and /etc/passwd's original MAC times using the 'touch -acmr' command again. However, the intruder does not succeed in hiding his edits from the MAC times. See the answer to question 3 for further explanation After attempting to hide his edits of the password files, the intruder does various listings of the root directory and /etc and finally looks at the contents of /etc/passwd-. This connection is finally closed 30 minutes later. After looking at /etc/passwd-, the intruder telnets in from 217.156.93.166 and logs in as nobody, with no password. The intruder then runs "su dns" and becomes root. The transcripts of this telnet session, captured using tcpflow, can be seen in telnet3.txt. The intruder runs "w" to make sure he's the only one on the machine and eventually makes it into the /dev/rd directory. An FTP session to teleport.go.ro is then run. The transcript of this session, gotten by using the "Follow TCP Stream" option in Ethereal, is located in . The intruder logs in as user "teleport" with password "gunoierul". He then downloads three files, Zer0.tar.gz, copy.tar.gz and ooty.tar.gz. These three files are rootkits. By using the "Follow TCP Stream" option, I was able to capture the rootkits and are in the files Zer0.tar.gz, copy.tar.gz and ooty.tar.gz which I included with the analysis. By looking at the rootkits, we can see that Zer0.tar.gz is the t0rn rootkit combined with the adore LKM rootkit. copy.tar.gz looks to be a new auto-rooter exploiting one of the popular wu-ftpd 2.6.0 exploits. Running strings on the wu-scan file in the rootkit can see this. It should also be noted that this rootkit contains a smurf program with a list of about 100 IP addresses; possibly machines to attack. ooty.tar.gz is a bunch of local root exploits packaged together. I have not seen these together before, so I am assuming that it is also new (the packaging, not the exploits). After all of the rootkits finished downloading, the intruder logged off teleport.go.ro. The intruder then untarred Zer0.tar.gz and ran "./Go 24". The rootkit is a combination of the t0rn rootkit and the adore LKM rootkit. The intruder runs the Go script to setup the rootkit. The script takes one command line option that is the port to run the trojaned SSH on. The setup script is a modified version of the setup script that comes with the t0rn rootkit. The first thing the script does is kill syslog. After that it checks to see if syslog was logging to any other machines and displays if it was. Next the rootkit creates three directories, /tmp/.dir1, /tmp/.dir2 and /tmp/.dir3. These directories are used to save the MAC times for /bin, /usr/X11R6/bin and /etc/rc.d/rc3.d. Four hidden directories that will be used to store the rootkit's files are then created in these directories. The rootkit then copies /bin/bash to one of the hidden directories, renames it to zsh and chmod's it to 7777. It then unzips the trojaned SSH and copies it to a hidden directory, renamed as nscdx. The rootkit also emails the system information using the shsml script to hatcheryhatched@hotmail.com. By looking in the Snort capture, the email can be seen. The SSH hash and two gzipped files, adr.tgz and tls.tgz, are moved. The files are then ungzipped to /usr/X11R6/bin/,./copy/adr. Next, the config file for the trojaned SSH is set up and the trojaned SSH is executed. The system information is then printed out for the intruder. The first few rules of IPchains are printed off and, if necessary, make and gcc is installed. The adore LKM rootkit is then installed. First the makefile for it is created and run. If it is created successfully, the files are loaded via insmod (in the stad Bourne shell script), a sniffer is started and a number of directories are hidden with the ava program. Finally, the rootkit backdoors the login program and cleans up. While the rootkit is installing, the intruder runs a SYN port scan from 207.35.251.172. This can be seen in the Snort dump and in the Snort alert logs. After the port scan, the rootkit is more or less installed and the intruder now has an SSH session running on port 24. Since the session is encrypted, we cannot see what he types and the responses through the Snort logs. However, since the honeypot was set up to log all commands run to the syslog server, we can look at the messages from the syslog server and see what commands the intruder ran. The following are the commands the intruder run from the bash shell, pulled from the syslog messages. w whoami cd /dev/rd/sdc0 ls rm Zer0.tar.gz ls alias ls='ls --color' ls ls passwd nobody ping www.yahoo.com pico /etc/rc.d/rc3.d/S50inet ls mv copy.tar.gz /usr/X11R6/bin/,./copy/ cd /usr/X11R6/bin/,./copy/ mv copy.tar.gz ../ ls cd .. tar zxvf copy.tar.gz chmod 7777 * ls rm copy.tar.gz cd copy chmod 7777 * ls uname -r pstree The logs end here. Questions -------- 1. What vulnerability did the intruder exploit? The intruder exploited the wu-ftpd SITE EXEC format string vulnerability. This vulnerability is fully documented at http://www.cert.org/advisories/CA- 2000-13.html, http://www.securityfocus.com/bid/1387 and http://cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2000-0573. A good description of the vulnerability, taken from the CERT Advisory, is as follows: The wu-ftpd "site exec" vulnerability is the result of missing character- formatting argument in several function calls that implement the "site exec" command functionality. Normally if "site exec" is enabled, a user logged into an ftp server (including the 'ftp' or 'anonymous' user) may execute a restricted subset of quoted commands on the server itself. However, if a malicious user can pass character format strings consisting of carefully constructed *printf() conversion characters (%f, %p, %n, etc) while executing a "site exec" command, the ftp daemon may be tricked into executing arbitrary code as root. The honeypot was probably exploited using the autorooter, which is found in the copy.tar.gz file that the intruder downloaded. 2. What ways, and in what order, did the intruder use to connect and run commands on the system? First, the intruder ran the wu-ftpd exploit that creates a root shell. The intruder was then able to run commands within this shell. The intruder then connects briefly using telnet and logs in as nobody, whose password they had previously deleted in the FTP shell. After going back to the FTP shell and running some commands, the intruder telnets back to the honeypot, logs in as nobody and su's to dns, which they have given root privileges. The intruder uses this to FTP out to teleport.go.ro to download the rootkits and install them. The rootkit installs a trojaned version of SSH on port 24. The intruder then connects on port 24 using SSH and executes commands until the log completes. 3. How did the intruder try to hide his edits from the MAC times? During the FTP session created by the exploit, the intruder modifies the password files to be able to log in and get root privileges. However, the intruder attempts to hide his edits from the MAC times. He does this by created two hidden directories and copying the MAC times of /etc and /etc/passwd to these directories using the "touch -acmr" command. By doing this, he is essentially saving the MAC times of /etc and /etc/passwd. Using the touch command with these options, the MAC times of the hidden directories are copied to be the same as /etc and /etc/passwd. The intruder then copied back the MAC times after they finished editing the files. These commands can be seen in the file ftp1.txt that I provided. It should be noted that what the intruder is attempting to do, hide the fact that he edited the password files from the MAC times, does not work. This is because the intruder had modified the password files when he first deleted the password for nobody. The MAC times were changed at that point so when he runs the touch command, he is actually saving the MAC times from his change only seconds earlier. If one were to compare the MAC times for /etc, /etc/passwd or /etc/shadow using something like tripwire, they would see that the files had changed. Additionally, when the intruder used the passwd and the adduser command, the file /etc/shadow was modified in addition to /etc and /etc/passwd. However, the intruder did not save the MAC times of /etc/shadow so they are not hidden from the system. 4. The intruder downloaded rootkits, what were they called? Are the new/custom rootkits? The intruder downloaded three rootkits called Zer0.tar.gz, copy.tar.gz and ooty.tar.gz. Zer0.tar.gz is a custom rootkit composed of the t0rn rootkit and the adore LKM rootkit. An excellent analysis of the t0rn rootkit can be found at http://www.sans.org/y2k/t0rn.htm and an excellent analysis of the adore LKM rootkit can be found at http://wwww.sans.org/y2k/practical/Michael_Reiter_GCIH.zip. copy.tar.gz is a custom autorooter that looks for and exploits the wu-ftpd SITE EXEC format string exploit. This is most likely what was used to compromise the honeypot. ooty.tar.gz is a collection of compiled local root exploits. The exploits are not new and can be easily found on the Internet. However, I have never seen them packaged together before so I can only assume this is custom rootkit. 5. Recover (tell how you did it too) the rootkits from the Snort binary capture. Using the "Follow TCP Stream" option in Ethereal, I was easily able to grab the rootkits from the Snort Binary capture. To do this, I followed the logs in Ethereal until I came across the beginning of the FTP data stream. By selecting one of the packets of the stream and running the "Follow TCP Stream" option, I was able to save the resulting data and recover the rootkits. In the Snort capture, there are three separate streams for each rootkit, so this has to be repeated three times. 6. What does the rootkit do to hide the presence of the attacker on the system? The rootkit that was installed came from Zer0.tar.gz and does the following things to hide the presence of the attacker on the system. With the exception of the information contained within adore.h, all of the things done can be seen in the Go script. 1. The rootkit kills syslog to prevent any further logging that may occur. 2. The rootkit saves the MAC times of the directories in which the rootkit files are placed. It will copy them back when the rootkit installation is finished so it does not look like any files were modified or added. 3. Any files that are placed on the honeypot are placed in a hidden directory or a directory not easily noticed. 4. The rootkit hides the presence of /usr/X11R6/bin/,., /usr/info/.torn, /dev/rd/sdc0, /dev/rd/nscd.init, /etc/rc.d/rc3.d/S50inet and /usr/X11R6/lib/X11/.~ using the adore LKM rootkit program ava. 5. The rootkit hides the presence of anything listening on ":ircd", ":24", ":666", ":443" and ":60000" or any processes called "zsh", "nscdx", "vrssnf", "psybnc". This is all done with the adore LKM rootkit and can be seen in the adore.h file, located in Zer0.tar.gz. 6. The rootkit cleans the logs of any mention of the string login, dns or ftp using the vrssb program. 7. What did you learn from this exercise? In this exercise, I learned that in order to fully analyze an intrusion, multiple tools are needed. I also learned that gunoierul, the password the intruder used, is Romanian for scavenger or refuse-collector. 8. How long did this challenge take you? Research and analysis took me about 2 hours and writing the report took about another 4 hours. BONUS: Based on this challenge, write an example letter of notification to the source owner that attacked the system. Include any evidence or logs that you feel important. The letter can be found in letter.txt.