cPanel Account Hacked. Zombie?

1

Hi.
I have a cPanel account that has been using 100% CPU. Htop says the command is /www/vhosts but that path does not exist. Nixstats says netns is using 100% CPU. Clamav doesn’t help at all.

When the account is active, load goes to 200+

I’ve already deleted malicious code found manually.

Any ideas how to find the problem?

2

Hmm. Might be a process masking the command line. Not sure if there’s a way to do this through htop, top, ps, etc. but you can get the PID (number usually shown in the first column in htop) then do ls -l /proc/1234/exe (where 1234 is the PID) and you’ll be able to see the path of the executable.

it’s possible it’s a malicious script

1 Like
3

Go to the cPanel account and then go to CPU and Concurrent Connection Usage - it’ll show you snapshots of processes that are being ran when the account is maxing out on CPU. This will be more accurate than what is shown on top and go down to a specific PHP file (if it’s PHP).

From there you can delete whatever it is and then go into WHM and kill the processes on the account, or have the host do it for you.

1 Like
4

If I were in your situation I would do this:

If it’s a simple html+php website, backup the website, check all files and clean what needs to be cleaned, delete cPanel account, create a new one and upload the clean backup.

In case of wordpress, download the list of plugin/theme names, screenshot all the settings, backup the post and postmeta database tables.
Again, new cPanel account, install a fresh WP installation and recreate the website.

Also make sure to find how in the hell the website was hacked to begin with…
Maybe change to a webhosting that offers Imunify360 like MightWeb, and put the website behind CloudFlare. It’s still hackable but it would throw away any newbies and most likely automated tools too.

1 Like
5
  • Manually deleting files can be hard because malware authors are getting good at hiding things. You’ll want to manually inspect pretty much every file and folder (including hidden files) on the cPanel account. Clamav rarely detects things for me, maybe try CXS scanning the single user. I think you can get a trial if you just want to screen this one account.
  • Check for any crons
  • You can use “Process Manager” in WHM to automatically kill all of that user’s processes without needing to hunt down each individual one
  • If it’s a Wordpress site, use Wordfence to find out of place files.
  • If it’s a Wordpress site, re-extract a clean Wordpress zip to overwrite all core files and restore their integrity (or any other CMS)
  • Like Max said, if you have Cloudlinux and are using snapshots you can sometimes find malware running through the cpu tab in cPanel: https://i.imgur.com/4PH3QdT.png
1 Like
6

I don’t think you can clean account by deleting files you think are hacked. You should run a scan via some tool such as CXS or immunifyAV

1 Like
7

Hi all. Thanks for your answers. This is a VPS running cPanel without Cloudlinux. What could be done manually has already been done. There are multiple Drupal and WordPress sites in the account.

Somewhere there should be a hidden script. Seconds after the account is activated the whole server becomes almost unusable.

And to be lucky it happens when there is not electricity nor internet connection at home… on Sunday.

I will try your suggestions, maybe a free trial of Inmunify or csx

8

I you wish and are comfortable with (this not being any customer’s account), you can send me files and I’ll run a scan with CXS or immunify per your wish.

9

Do you know how cPanel does this? Do they patch PHP to add extra logging? I’d like to get something similar (logging of PHP scripts that use a huge amount of CPU time) on my non-cPanel server.

10

I reckon it’s just periodically snapshotting and parsing ps output

11

PHP doesn’t spin up one process per request though - that’s prohibitively expensive if you receive a large number of concurrent requests. Most people use PHP-FPM now, which uses a pool of workers to handle requests. Not sure if cPanel uses PHP-FPM, or if it’s still using the PHP Apache module.

I wonder if it’s scraping info about which scripts are being executed by the PHP workers, and correlating the PIDs.

12

That entirely depends on the mode in which PHP runs.

Anyways you’re right that this wouldn’t work for PHP-FPM - or anything FastCGI for that matter.

I guess you could take a look at one of the decrypted versions of cPanel floating around the internet.

13

Finally there ir electricity in the city and internet connection.

First try. It worked!

image
image862×670 75.4 KB

1000×600 71.9 KB

1 Like
14

Installed Imunify360 and detected the infected files. Not everything but 90% at least.
The account is active and idling again!
Thank you guys!

5 Likes
15

3 Likes
16

It’s happening again with another account, and it’s sending spam :frowning:

And this is not working this time.
Output:

lrwxrwxrwx 1 thecpaneluser thecpaneluser 0 abr 16 13:58 /proc/11845/exe -> /usr/bin/perl

Also:

# lsof -p 11845
COMMAND     PID         USER   FD   TYPE             DEVICE SIZE/OFF      NODE NAME
/usr/sbin 11845 thecpaneluser  cwd    DIR                7,0    24576         2 /tmp
/usr/sbin 11845 thecpaneluser  rtd    DIR              253,0     4096        64 /
/usr/sbin 11845 thecpaneluser  txt    REG              253,0    11488    409739 /usr/bin/perl
/usr/sbin 11845 thecpaneluser  mem    REG              253,0   163400  34868187 /usr/lib64/ld-2.17.so
/usr/sbin 11845 thecpaneluser  mem    REG              253,0  1647344       370 /usr/lib64/perl5/CORE/libperl.so
/usr/sbin 11845 thecpaneluser  mem    REG              253,0   105824  34871681 /usr/lib64/libresolv-2.17.so
/usr/sbin 11845 thecpaneluser  mem    REG              253,0   115848  34871657 /usr/lib64/libnsl-2.17.so
/usr/sbin 11845 thecpaneluser  mem    REG              253,0    19288  34868269 /usr/lib64/libdl-2.17.so
/usr/sbin 11845 thecpaneluser  mem    REG              253,0  1137024  34868271 /usr/lib64/libm-2.17.so
/usr/sbin 11845 thecpaneluser  mem    REG              253,0    40664  34785617 /usr/lib64/libcrypt-2.17.so
/usr/sbin 11845 thecpaneluser  mem    REG              253,0    14496  34787459 /usr/lib64/libutil-2.17.so
/usr/sbin 11845 thecpaneluser  mem    REG              253,0   142232  33561993 /usr/lib64/libpthread-2.17.so
/usr/sbin 11845 thecpaneluser  mem    REG              253,0  2156160  33561967 /usr/lib64/libc-2.17.so
/usr/sbin 11845 thecpaneluser  mem    REG              253,0    11392  33561679 /usr/lib64/libfreebl3.so
/usr/sbin 11845 thecpaneluser  mem    REG              253,0    19888  33554644 /usr/lib64/perl5/auto/IO/IO.so
/usr/sbin 11845 thecpaneluser  mem    REG              253,0    44520 100960242 /usr/lib64/perl5/vendor_perl/auto/Socket/Socket.so
/usr/sbin 11845 thecpaneluser    0r  FIFO                0,9      0t0 100517919 pipe
/usr/sbin 11845 thecpaneluser    1w  FIFO                0,9      0t0 100516576 pipe
/usr/sbin 11845 thecpaneluser    2w  FIFO                0,9      0t0 100517401 pipe
/usr/sbin 11845 thecpaneluser    3u  IPv4          108518375      0t0       TCP mycpanelserver.example.com:45338->sip.sattrakt.com:http (SYN_SENT)
/usr/sbin 11845 thecpaneluser    4u  unix 0xffff9d2065413800      0t0 100517406 socket

This takes my attention: sip.sattrakt.com

I think I’m going to buy a Cloudlinux license and CXS, but it will be next week.

Meanwhile, any ideas to find the file?

19

\ :monkey_face: / \ :monkey_face: /

Give this a try :

https://documentation.cpanel.net/display/68Docs/cPHulk+Brute+Force+Protection

20

I’m using CSF

21

Its often running from tmp location but als often deleted after process is started. You can try searching the account for perl script though and run scan via clamav.

Also once its compromised, deleting files won’t do anything. You need to thoroughly clean or even rebuild the account.

1 Like
22

Even better, a complete server reinstall…