Issue to establish SSH connection between two different network interfaces
-
@fabiolanza Perhaps clean up the rules and start with having a "default" allow "VLAN" to any rule. There is already a default deny rule, that you can't see. So with just an empty list of rules, you can do nothing from a VLAN.
A simple standard setup could be something like this for an IoTVLAN from which you might only allow access to internet, and nothing local.
Since rules 1 and 2 block any access to LAN and GuestVLAN, the only thing left is the internet in this case.
If I wanted something on IoT_VLAN to be able to access a specific device/server on my default LAN (192.168.1.10 for example). I would add an allow rule at the top, with Source - IOT_VLAN Subnet and Destination 192.168.1.10. I could add port 22 as well, to make it more specific. Or I could say Source (and specify an IP) so that only one specific device could access that server on LAN...
But there shouldn't be a need to do special Outboud NAT rules for any of this to work. pfsense does not masquerade or change ports for inter VLAN communication.
However, I think you have now set it up to NAT... which is the opposite of what you wanted. Since it shows that "cross over" symbol. Which is why you see masquerading happening in your pcap data.
If you want static port, you need to specify that in the rule, and it will change to this symbol. But, like I said, no need for any special NAT rules for inter VLAN communication to work, just set it to Auto.
-
@Gblenn thank you so much for taking some time from your Sunday to answer my post.
I added allow all rules in both interfaces and placed them at the top of the rule set. I also tried disabling the outbound NAT which was there to enforce NO NAT, as for a while I suspected that there could be happening some weird NAT between the interfaces. However, I still obtain the same issue. SSH works when the connection is happening in the same VLAN, but then it has to traverse the other VLAN interface in the firewall, it does not work. If I check the blocked traffic, there's nothing related, as expected since the firewall rules are in place.
-
One thing came to my mind... my Netgate device is the https://docs.netgate.com/pfsense/en/latest/solutions/sg-5100/io-ports.html. If I look at the specs (below), there are two considerations:
- It says that port IGB0 is for WAN and IGB1 for LAN. In m case, I am using both for WAN purposes, as I have a dual WAN setup;
- There are some other notes considerations on limitation related to speed negotiation.
Could these considerations have to do with the issue that I am experiencing?
-
@fabiolanza Ok, so if you place them in the same VLAN, it works, but when you move one of them over to another VLAN it stops working, correct? With a rule similar to my allow VLAN to any at the top, there should not be anything stopping it from working.
Did you try setting outbound NAT to Auto?And have you checked the DHCP Server settings on the respective VLAN's. Could it be something there that is out of the ordinary (using KEA?).
-
@fabiolanza said in Issue to establish SSH connection between two different network interfaces:
One thing came to my mind... my Netgate device is the https://docs.netgate.com/pfsense/en/latest/solutions/sg-5100/io-ports.html. If I look at the specs (below), there are two considerations:
- It says that port IGB0 is for WAN and IGB1 for LAN. In m case, I am using both for WAN purposes, as I have a dual WAN setup;
- There are some other notes considerations on limitation related to speed negotiation.
Could these considerations have to do with the issue that I am experiencing?
I doubt it, but someone else may be able to verify. I think those notes are mainly about the physical connection...
-
If it was either a NAT or policy routing issue I'd expect to see the connection fail completely. But here the initial TCP handshake completes correctly. There is two way traffic.
Also you would see NAT in the states that are opened and you don't.
Try pcaps in pfSense on each interface and see if the initial large packet from the server makes it to either.
-
@Gblenn I did try outbound NAT to automatic, it did not work the same way... DHCP works correctly on both interfaces, I can't think how it could be the issue, but thanks!
-
@fabiolanza Since it works if both are on the same subnet, but not when they are in different. Perhaps you should check for some settings on the server. Like if there are limitations in /etc/ssh/sshd_config that may prevents access from different subnets etc. Look for allowed users, IP ranges and things that may limit access.
Perhaps you can find something by comparing the logs and pcap output from when they are in the same subnet where it's working... -
@Gblenn SSH is not the only service not working. I tried to iperf3 between both hosts across the interfaces and that fails too... this is indication that it's something to do with the network. I have added firewall rules for iperf3 and I see the hits in the allow rule... it is possible to buy a single support ticket from Netgate without having to pay the yearly subscription?
-
@fabiolanza Ok, that's interesting... so what does your network look like, pfsense and switches?
Does pinging between the hosts work?
If you have a rule as mentioned, allow all to any, adding one for iperf3 will do nothing to help.
What do your rules look like right now, at both VLAN's? -
Do you see the same failure between any two hosts on those subnets?
-
@stephenw10 and @Gblenn let me add more information...
If I place my VMs on the same VLAN, the communication between them works without issues, including the SSH connection between them and other services.
When I try to connect to the VMs from a different VLAN, services like SSH and IPERF3 will not work. Looking at the pfSense firewall logs, I see that TCP/S passes as expected, but then TCP/RA will get blocked. I added a floating rule to pass TCP/RA but it did not change anything.
Please see an image of my network diagram for more details.
Thanks
-
Is the layer3 switch routing between those VLANs? If is that could create asymmetric traffic in pfSense.
But run pcaps in pfSense so you can whats actually going through both the interfaces. The earlier pcap show the server sends the Key Exchange Init packet but the client never receives it. So does that arrive at pfSense at all?
