[SOLVED] Hacking on SSH even though no portforwarding or WAN rules that allow it

[iorx]

Wrong forum-category?

Jan 26 06:34:37 	sshd 	77025 	Failed password for root from 193.201.224.109 port 55489 ssh2
Jan 26 06:34:37 	sshlockout 	26057 	Error adding entry 193.201.224.109 to table sshlockout.
Jan 26 06:34:37 	sshlockout 	26057 	Locking out 193.201.224.109 after 15 invalid attempts
Jan 26 06:34:37 	sshd 	77025 	Failed password for root from 193.201.224.109 port 55489 ssh2
Jan 26 06:34:36 	sshlockout 	26057 	Error adding entry 193.201.224.109 to table sshlockout.
Jan 26 06:34:36 	sshlockout 	26057 	Locking out 193.201.224.109 after 15 invalid attempts
Jan 26 06:34:36 	sshd 	77025 	Failed password for root from 193.201.224.109 port 55489 ssh2
Jan 26 06:34:36 	sshlockout 	26057 	Error adding entry 193.201.224.109 to table sshlockout.
Jan 26 06:34:36 	sshlockout 	26057 	Locking out 193.201.224.109 after 15 invalid attempts
Jan 26 06:34:36 	sshd 	77025 	Failed password for root from 193.201.224.109 port 55489 ssh2
Jan 26 06:34:36 	sshlockout 	26057 	Error adding entry 193.201.224.109 to table sshlockout.
Jan 26 06:34:36 	sshlockout 	26057 	Locking out 193.201.224.109 after 15 invalid attempts
Jan 26 06:34:36 	sshd 	77025 	Failed password for root from 193.201.224.109 port 55489 ssh2
Jan 26 06:34:29 	sshd 	76380 	Disconnecting: Too many authentication failures [preauth]
Jan 26 06:34:29 	sshd 	76380 	error: maximum authentication attempts exceeded for root from 193.201.224.109 port 52582 ssh2 [preauth]
Jan 26 06:34:29 	sshd 	76380 	Failed password for root from 193.201.224.109 port 52582 ssh2 

Obviously to me is that the traffic originates from the outside

How do one go about figuring out how and where this traffic has sneaked its way through?

[johnpoz]

And what are you wan rules?  Lets see them.

[Derelict]

And what port is sshd configured to listen on? (System > Advanced, Secure Shell, SSH Port)

[iorx]

I've changed port now for the ssh. It was present on default 22 before that.

Going through the raw system.log to see if I can grasp what's going on here. Any particular info which should not be in the file if I'm to attach it here?

It looks like the firewall lost power at 05:20 this morning. After that the attacks begun.

Them' rules coming up, attached png.

[KOM]

Nothing there would allow ssh, if that's the complete list.  Anything in your floating rules?  What packages are you running, if any?

[iorx]

Floating rules only have an ICMP * * * * rule, right now.
When pfBlockerNG is active there are some more.

When I encountered the attack I changed the port of ssh and disabled pfblockerng, and started investigating. I'm going to restore that config and document how it looked here.

This error in the system.log eludes me.
Jan 26 05:25:28 fwna php-cgi: rc.bootup: New alert found: There were error(s) loading the rules: /tmp/rules.debug:29: cannot load "/var/db/aliastables/pfB_Firehol.txt": Invalid argument - The line in question reads [29]: table <pfb_firehol>persist file "/var/db/aliastables/pfB_Firehol.txt"

Can that cause rules not to be loaded as they should?

Attached are screen dumps.

</pfb_firehol>

[johnpoz]

"It looks like the firewall lost power at 05:20 this morning. After that the attacks begun."

If your rules do not load – then sure it could be like no firewall at all..

[KOM]

If your rules do not load – then sure it could be like no firewall at all..

That makes sense, and yet it seems like incorrect behaviour for pf.  One glitch in the ruleset and it throws them all out and hangs your naked ass out on the Internet?  Wonderful.

[johnpoz]

Just saying it could happen… Yeah that is with any firewall.. If rules do not load should be block all, etc..  But all depends on the loading of the rules fail..

[Derelict]

As far as I know it is only on reboot. If there is a problem loading the rule set after it is loaded the first time it simply does not apply the changed/bad rule set.

https://redmine.pfsense.org/issues/6028

[KOM]

As far as I know it is only on reboot.

Good to know, thanks.

I suspect we can put this issue to bed.  A glitchy pfBlocker ruleset combined with a forced reboot equals pants totally down.

[iorx]

Yes, that explains the behavior before I disabled the pfblocker. I had no routing from within the LAN, could not ping anything outside the LAN.
Disabling pfBlocker triggered a reload of the rules, and with that it restored a working rule set.

I'm going for a rebuild and replace of this instance to be sure nothing was compromised. And is going to have a long hard look at the UPS procedure which should have taken this system down gracefully. Obviously they don't work as intended.

Although, the OpenVPN connecting this office to the HQ was working all along, even with the rule set not in place. Maybe not so strange as it operates directly on the WAN interface.

The scariest thing about this whole thing is that 05:25:24 the firewall came back from the unplanned power outage. 05:26:33 I see the first attempt for an unauthorized logon attempt, which then escalates into a flood of webadmin and ssh logins. Got a bunch of interesting login names tried anyway :-)
Src of attack: 5.101.40.10, 193.201.224.109, 103.79.143.141, 85.182.38.103
193.201.224.109 was some kind of robot, as it was checking for logon with the username in alphabetic order.

Thanks ladies and germs for helping me come to an explanation and closure!

Brgs,

[johnpoz]

"05:26:33 I see the first attempt for an unauthorized logon attempt,"

Not really strange to see that… There is a shitton of noise on the net..