Today have been a bad day, at 3AM in the morning, received an email alert from Alibaba Cloud indicating successful brute force attacks to a few of my ECS servers. It was my mistake for not changing the server SSH default port, given the chance to the hacker to brute force attack my server access.
I discover what the hacker did to my ECS server is creating a cron job that constantly compile and generate 2 executable scripts call Donald and Trump. The scripts spike my server CPU to 100% and affecting my other application process being slow in performance.
How do I find the root if you ask me? Just run the top command and then press C, you will see the Donald script is running and eating the 100% CPU and also its source location. However, I spotted there is an unknown suspicious process actively running as well.
By deleting Donald and Trump executable scripts will not solve the problem. I discover that there is a cron job running to automatically compile and create both Donald and Trump scripts after a while.
So, I run the systemctl status command in order to further trace which and where actually started the process.
As you can see from the above screenshot, the culprits are:
/tmp/Donald
/usr/bin/ujwofa5
Also note that there is a cron job actually execute ujwofa5. Therefore, we need to remove all the executable scripts and cron job as well.
In order to trace the cron job, just run crontab -l command
For deleting the cron job, run crontab -r command
You may want to stop the cron job if you dont need it at all, just run service crond stop
Now, remove the Donald script and ujwofa5 by executing rm command. Then, kill both processes.
Now, you need to reboot the server in order to permanently remove the script to take effect. After server get back online, you can run top command again to monitor and see if the Donald or Trump script appear again. If everything goes normally, then the problem is considered solved.
It was lucky that Alibaba Cloud sent me email notification about suspicious login to my server actually triggered me to check what's wrong. Lesson learned that I should not being lazy even the server is not important, I should change the SSH default port.
For your knowledge how to change SSH default port, just login to the server again. Open up the SSH config file by executing command vi /etc/ssh/sshd_config
Modify the file by removing the # comment at the Port 22 row, then change the value 22 to your desire port number. Once done, save the file and then restart the SSH service by executing service sshd restart
Now, the server is resume back to normal. Let me know if this helpful, and note that the cron job script name is random and different from each server. You cannot assume your cron job script name is ujwofa5.
I discover what the hacker did to my ECS server is creating a cron job that constantly compile and generate 2 executable scripts call Donald and Trump. The scripts spike my server CPU to 100% and affecting my other application process being slow in performance.
How do I find the root if you ask me? Just run the top command and then press C, you will see the Donald script is running and eating the 100% CPU and also its source location. However, I spotted there is an unknown suspicious process actively running as well.
By deleting Donald and Trump executable scripts will not solve the problem. I discover that there is a cron job running to automatically compile and create both Donald and Trump scripts after a while.
So, I run the systemctl status command in order to further trace which and where actually started the process.
systemctl status <PID>
As you can see from the above screenshot, the culprits are:
/tmp/Donald
/usr/bin/ujwofa5
Also note that there is a cron job actually execute ujwofa5. Therefore, we need to remove all the executable scripts and cron job as well.
In order to trace the cron job, just run crontab -l command
For deleting the cron job, run crontab -r command
You may want to stop the cron job if you dont need it at all, just run service crond stop
Now, remove the Donald script and ujwofa5 by executing rm command. Then, kill both processes.
Now, you need to reboot the server in order to permanently remove the script to take effect. After server get back online, you can run top command again to monitor and see if the Donald or Trump script appear again. If everything goes normally, then the problem is considered solved.
It was lucky that Alibaba Cloud sent me email notification about suspicious login to my server actually triggered me to check what's wrong. Lesson learned that I should not being lazy even the server is not important, I should change the SSH default port.
For your knowledge how to change SSH default port, just login to the server again. Open up the SSH config file by executing command vi /etc/ssh/sshd_config
Modify the file by removing the # comment at the Port 22 row, then change the value 22 to your desire port number. Once done, save the file and then restart the SSH service by executing service sshd restart
Now, the server is resume back to normal. Let me know if this helpful, and note that the cron job script name is random and different from each server. You cannot assume your cron job script name is ujwofa5.
