SSH connection stalls when going through pfSense

pfuzr

Hi pfSense experts,
I have been having issues with SSH connections stalling on me when I connect to a Raspberry Pi W0.
My network setup:
SG1000(pfSense 2.4.5 dev branch) has LAN connection with 192.168.34.0/24 - Hosts on this network are assigned static IPs
wRouter1 192.168.34.2 is WAN of wireless router with internal network as 192.168.37.0/24 - Mainly used for wireless clients + 1 Raspberry Pi 2 (running pi-hole)

Connecting to the rpi0w when on wireless connection or via LAN connected to wRouter1 works fine and I don't loose the connection. pfSense does not come in this connection. But when I connect via SSH when I'm on the 192.158.34.0/24 network, the connection just stalls after a few seconds (random 5 - 15 seconds).

uzr@gurh 09:24:22 {av} $> ssh pi@pi0w
pi@pi0w password: 
Linux pi0cam 4.19.57+ #1244 Thu Jul 4 18:42:50 BST 2019 armv6l
pi@pi0w:~ $ uptime
 09:25:01 up 4 days, 10:07,  1 user,  load average: 0.00, 0.00, 0.00
pi@pi0w:~ $ clear client_loop: send disconnect: Broken pipe
uzr@desk1 09:41:09 {av}

(above connection stalled at command $clear and then broke about 15 minutes later). SSH option of -vvv does not indicate anything wrong with the connection. A second connection on SSH shows that user is logged on till the connection breaks.

Changing SSH settings as suggested by Rpi users for such stalls has not helped. Also, earlier I thought that this was due to the virtualized setup of pfSense (it was a VM in xcp-ng running stable release 2.4.x) with similar networking setup. I tried to rule out xcp-ng by just using the SG1000 and I'm still facing the stalls.

Any suggestions?
All help appreciated.

johnpoz

I am trying to understand your network here..

So you have this..

Is your wlan router doing nat? Or is it setup as downstream router in pfsense? If nat your connecting 192.168.34.2 and being forwarded in by your wlan router?

If its not doing nat, you have an asymmetrical setup.. Unless your host routing on your pc on the 34 network on how to get to the 37 network.

If your client on your 34 network tries to go to 192.168.37 it goes to pfsense, who sends it to .2 but your answer will not come back through pfsense..

So you have few choices to fix this - if understanding your setup correctly.

Just use that wlan router as AP and let pfsense be directly connect to this 37 network.
Nat at this wireless router and use port forwarding to get to stuff behind it.
Create a transit network between your downstream wireless router and pfsense so you are not asymmetrical
Host route on your devices on 34 network and tell them hey to get to the .37 use 192.168.34.2

pfuzr

Yes, you are right that it is indeed a asymmetrical setup! Didn't dawn on me earlier.

Using the router as AP would be my first choice. However, I'm not much experienced about how to set that up i.e. providing .37 addresses to clients from the AP. Any pointer will be helpful.

johnpoz

So use any wifi router as just as an AP..

Connect it to the network via one of its lan ports. Turn off its dhcp server and set it up dhcp on pfsense. Setup wifi router lan IP to be on the network.

If your wifi router does allow for setting a gateway on the routers lan IP, many soho firmware doesn't allow for this sort of setup.. You have few of options.

Just access its web gui via that network or wireless.
Put on 3rd party firmware that allows for a gateway setup so you can get to it from your other networks. dd-wrt for example
Source nat on pfsense, so that when you need to manage the wireless settings on the wifi router looks like its coming from pfsense IP in that network.

Do you have another interface to use on pfsense? Does your switch support vlans?

You need to have another interface on pfsense that will be say 192.168.37.1/24 with your wifi router lan IP being say 192.168.37.2, and dhcp scope on pfsense to be set for say 192.168.37.10-250, this leaves a few ips on each end of the range for statics. Adjust that as you see fit.

Now since both of these networks are directly connected to pfsense, no need for any routes on pfsense to be created. Pfsense will already know how to get to any of its local attached networks. This gives you much finer control of what devices on this 37 network can do or not do now as well.

pfuzr

@johnpoz Thanks for the reply. The part for setting up the router as AP I understand well enough. It was the PF sense part which I want help in.
Regarding the pfsense interface for .37 network, I have the SG1000 with WAN and LAN as two physical interfaces. Unfortunately, my switch doesn't support VLANs (unmanaged switch). What are my options with this hardware?

PS: another option would be to change back the SG1000 with the xcp-ng based VM as that box has 4 network ports on it. But setting up SG1000 was to simplify my Internet connection in case of issues (power failure, etc), so that it can be just plugged in and started by anyone at home.

johnpoz

You need to get a vlan capable switch. Or can your wifi router run 3rd party - you can do the prob do the vlan stuff dd-wrt. Have not seen any native firmware that actually support vlans on wifi routers.

Then you could connect your wifi router into sg1000, and any other dumb switches you have into the wifi routers other lan ports to put them say on your 34 network.

But you can pick up a 8 port smart switch that can do vlans for like 40$.. They are not all that expensive..

Or you could upgrade to a sg1100 that has switch ports to use on your lan side that would allow for 1 being wlan with your ap and other lan.

pfuzr

No VLAN capable managed switch yet. Thinking of a Unifi switch though.

For now, I will try to use my Asus AC87U (~~n66u~~) router with merlin firmware as it has VLAN capabilities.

johnpoz

There are much better switches than unifi ;) For better price points to be honest.

If you were already heavy into their stuff, AP, Camera's, There router, etc. Then sure ok.. But since you have nothing better switches out there for sure.

How many ports?

pfuzr

@johnpoz said in SSH connection stalls when going through pfSense:

There are much better switches than unifi ;) For better price points to be honest.

If you were already heavy into their stuff, AP, Camera's, There router, etc. Then sure ok.. But since you have nothing better switches out there for sure.

I don't ~~won~~ own a Unifi device yet. I had heard good things about them about being "config and forget" with a modern/easy GUI. Also, a colleague who uses a 24 port unifi switch, said that they hardly make any noise. Completely open to suggestions though.

How many ports?

16 might be idea. 8 would work also.

johnpoz

Any of the $40 vlan switches would quiet - they are all fanless. And are set them and forget them with a gui.. Most switches in that port density range would be fanless

It would all come down to features to be honest. And what kind of money you were looking to spend.

They have 2 line of switches, if you don't want to use the controller so it integrate with your other unifi access points for example you would want their edge switches. Which do have a gui, their other switches not the edge line only have cli unless you use the controller.

What kind of other features would you be looking for other than vlans? I am not aware of any of there lower cost switches doing L3.. If you would ever want to do that, not even sure if their higher end models do that - I think they are all only L2.

So their ES-10X runs like 100$ and show the sg350-10 for 147.. But the 350 has way more features, and they have great gui for sure..I have 2 different sg300-28 and sg300-10

You would have to compare their full feature set to determine if they support everything you would want, or possible in the future. The 350 line added rspan for example which my sg300 are lacking - which would be pretty slick ;)

But if all you are looking for is the basics - pretty much any 40$ smart switch would work for you.. Other than the tplink ones - stay away from them, they have bad history of just complete an utter lack of understanding of point of vlans ;) There older models for sure wouldn't allow you to remove vlan 1 ;) They have suppose to have fix that in latest models v3, v4 but they didn't go back and fix v2 or v1 models..

one thing that you might want to do some research on as well - and might have some issues with the 40$ line of smart switches is what tls versions they support.. 1.2 is becoming min, and many a browser are going to bitch at you if the gui connection is not atleast 1.2.. I just did a quick google and seems the edge switches only to 1.0? And don't see anything about upgrading.. Just something to think about that could get overlooked.

I would say you could just do http since you only be managing local - but browser are starting to bitch at you about this as well.. I would love if my sg300 would do 1.3, but doubt they will add it.. But atleast they do 1.2.

pfuzr

Thanks for the informative post. I prefer not to just go and pick up a cheap $40 managed switch because usually the price-cuts are mostly due to lesser features and lower grade hardware. I do need to research about this a bit if I want a decent switch.

You reminded of another important thing: support. It is better to pay a little more to get a recent switch which gets updates on time than something which might be EoL in 2-3 years. Your TLS concerns highlight the same thing. Also, promptness of the manufacturer in releasing updates will keep the device safe from vulnerabilities.

For now the features that I have in mind (more will be added after some research): jumbo frame support, 802.1Q VLAN support, port trunking (IEEE 802.3ad with LACP for upcoming NAS), low noise (I see some CISCO switches run like a jet engine for several minutes at startup!), ACLs, PoE (nice to have, but not a requirement).

johnpoz

@pfuzr said in SSH connection stalls when going through pfSense:

I see some CISCO switches run like a jet engine for several minutes at startup!

In the smb line unless your talking higher port density >28, they are normally all fanless
https://www.cisco.com/c/en/us/products/collateral/switches/small-business-smart-switches/data-sheet-c78-737359.html

So you don't have any fans until you hit the 28 port poe model in the sg350 line for example.

As to those features, for example the $40 dgs-1100-08 has all of that other than the ACLs, and there is a poe model. And if you look at their firmware release history, they do update it..

Just trying to point out that you don't have to drop $200 bucks to get a switch that can do what you want.. And that you can get instant gratification for a few bucks ;) vs dicking with work arounds. Especially if high port density is not a requirement currently. Keep in mind that lacp is not going to get you much with your nas unless you have lots of clients, and or your devices that are going to be moving traffic to and from it also have multiple interfaces.. Or what your worried about is failure mitigation on loss of port on your switch or nas.

Since you stated 8 port would work, doesn't seem like you have enough clients to worry about setting up lacp ;)