Reroot exposes SSH, Telnet, Web UI to WAN

NollipfSense

@user1337 What's reroot...a typo? If you didn't mess with the firewall and only have one custom rule, how did you determine those items were exposed to the world?

Gertjan

"Telnet" doesn't exist on pfSense by default. I thought that one followed FTP into a museum.
SSH is not activated by default, but listens on all interface if it is, true.
Like the GUI web server.

The default firewall action for every interface is : block.
Exception : only the LAN interface : by default, the GUI contains a pass all rule.

@NollipfSense :

The reroot 'reboot' is here :

or option 5 SSH/console :

Netgate pfSense Plus will reboot. This may take a few minutes, depending on your hardware.
Do you want to proceed?

    Y/y: Reboot normally
    R/r: Reroot (Stop processes, remount disks, re-run startup sequence)
    S: Reboot into Single User Mode (requires console access!)
    Enter: Abort

Enter an option:

I'll reroot this afternoon, and nmap on the WAN side.

User1337

My mistake, for some reason I thought there was telnet. I do use SSH, however. I determined it by checking my logs and seeing brute force attacks, verified it with my phone's non-wifi connection as well.

Gertjan

@user1337

So, when you reroot, you can use a device like a phone, and via the 3G/4G/4G you 'hammer' (nmap if possible ) your IP WAN, and you can connect to SSH and the pfSense GUI ?

What are your WAN firewall rules ?

User1337

@gertjan Yes, the first thing that pops up when connecting to my public facing IP is the web UI. I even disabled the traffic shaper rule for the sake of troubleshooting and was only left with the default block bogon/private networks rules.on WAN. Again, with full reboot the firewall works as intended and any external attempts time out. nmap after reroot:

Interesting ports on x.x.x.x:
Not shown: 996 filtered ports
PORT    STATE SERVICE
22/tcp  open  ssh
53/tcp  open  domain
80/tcp  open  http
443/tcp open  https

Nmap done: 1 IP address (1 host up) scanned in 16.93 seconds

NOCling

No problem here, can you show your ruleset?

If you setup Rules like this:

You expose your WAN IP to the Guest Users, you need this first:

User1337

@nocling My rules:

NOCling

Your last Rule allows your LAN to 443,80,22 on the pfSense WAN Adresse.

Do not try a Scan on the WAN Adresse from a LAN Client!

Gertjan

@user1337

LAN : ok
WAN : why ? the default 'hidden' block rule will take care of these. "The good WAN is an empty WAN".
Floating : Looks fine to me.

@nocling said in Reroot exposes SSH, Telnet, Web UI to WAN:

Do not try a Scan on the WAN Adresse from a LAN Client!

Because he's scanning the outside from the inside ?
Well, yeah, the test is wrong then.

I was serious :

@gertjan said in Reroot exposes SSH, Telnet, Web UI to WAN:

you can use a device like a phone, and via the 3G/4G/4G you 'hammer' (nmap if possible ) your IP WAN

In short : from the outside.

NOCling

Limiter Rules works best with match and no quick.
Have a look on the documentation:
https://docs.netgate.com/pfsense/en/latest/firewall/floating-rules.html?highlight=match

NollipfSense

@gertjan Learned something new today, thank you!

User1337

@nocling said in Reroot exposes SSH, Telnet, Web UI to WAN:

Your last Rule allows your LAN to 443,80,22 on the pfSense WAN Adresse.

Do not try a Scan on the WAN Adresse from a LAN Client!

Unless I am missing something, It's a default rule created by the setup wizard. Disabling this would lock me out even from LAN. I changed the limiter rule to "Match," but regardless even with the rule disabled my network becomes exposed to WAN after reroot.

@gertjan said in Reroot exposes SSH, Telnet, Web UI to WAN:

In short : from the outside.

The test was done from my phone with LTE. Why do you think I did it from LAN given the premise of the thread?

User1337

Ok, the cause of the issue is from using the TOE function of my NIC (Chelsio T5). With TOE disabled, nmap fails even after reroot. With TOE enabled, nmap can scan ports and port 80 appears open but my web UI is configured to use HTTPS, so fails to connect from WAN. After reroot, everything becomes exposed and accessible from WAN. I thought the ASIC was a self-contained firewall that further passed packets to the regular (system) firewall, but it appears to (partially) bypass it. I guess that's how it achieves line-rate filtering. For now I guess I will leave TOE disabled unless someone can suggest how to use it without bypassing PfSense's firewall.

User1337

Sorry, can't edit old posts, but I found the answer: https://calomel.org/freebsd_chelsio_toe_firewall.html:

The Chelsio Offload Policy (COP) manages when the TCP Offload Engine (TOE) takes affect allowing the card to only offload TCP connections which you want to offload and leave the other connection to the default FreeBSD TCP stack.
...
SECURITY NOTE: The Chelsio TCP Offload Engine (TOE) will completely bypass the FreeBSD TCP stack as well as any Chelsio filter rules. This means that traffic using TOE will NOT be filtered using our Chelsio Rules of Engagement filter rules or the Pf packet filter, nor will Pf log TOE connections. Netstat will show the connections using "netstat -np tcp" though.

SteveITS

@user1337 That’s a feature? I wonder if that’s something Netgate can disable in the driver/config. You might open a Redmine.pfsense.org about it.

Side note:there’s a long thread here somewhere about (some?) Chelsio cards downloading at half speed in 23.01.

stephenw10

Hmm, so this was eliminated by disabling TOE or rather simply not enabling it?

What pfSense version did you see this in?

You only saw it after rerooting?

This was permanent after the reroot? Not just while the reroot was happening?

Steve

User1337

@steveits TOE is disabled by default and can't be enabled by accident so nothing Netgate needs to worry about. My WAN speed matches my expectations when using typical web-based speed tests, but I only have a ~350/10Mbps plan so it's not enough to expose a large bottleneck.

@stephenw10 Yes, I disabled TOE by commenting out these lines in a custom script in /usr/local/etc/rc.d:

#ifconfig cxl0 toe
#ifconfig cxl1 toe

I enabled TOE on LAN manually via SSH ifconfig cxl1 toe and indeed it works, 1.15 vs .23 TIME+ shown in htop for each respective iperf2 server process. No difference with iperf3. Interestingly enough pfblockerng also works, which means the system firewall is taking effect. This all may be due to this detail from the same calomel link:

TCP TIMING: We noticed that short lived connections of less then 0.6 seconds will NOT use the Chelsio TCP Offload Engine (TOE) even if TOE is allowed universally or through Chelsio Offload Policy (COP). Not sure of the reason.

Next I tried enabling TOE on WAN ifconfig cxl0 toe and noticed SSH and web UI don't become exposed to WAN until after reroot. Version CE 2.6.0 (22.01). Yes it's permanently exposed after reroot. At this point it's not a real issue but may be of help to anyone else that goes down this path.

stephenw10

Yes, that's a nice catch. It should definitely be documented.

Did you try both 2.6 and 22.01 or just noted they are closely equivalent?

User1337

@stephenw10 Just 2.6.0.

jimp

Given the description of how TOE works I would have expected it to always bypass pf and not just at reroot. I'll try to work a working against using that into the docs but I'm not sure where it might fit that users would see it. The most likely place would seem to be https://docs.netgate.com/pfsense/en/latest/hardware/tune.html#chelsio-cxgbe-4-cards

In the future, if you believe you have discovered a security vulnerability, please report it privately as described on https://www.netgate.com/security and not on a public forum post. That way we can investigate it and work on a fix before it is widely known to the public.