Unable to telnet to another LAN subnet, until after a successful ping/traceroute
-
I have a strange situation, I am unable to telnet/http/ssh etc to hosts on another subnet, until after I do a ping or traceroute, after which the telnet/http/ssh is successful straightaway.
My network setup is like this:
internet---pfsense----L3 switch-----default subnet (VLAN 1) 10.10.0.0/16 \ -------2nd subnet (VLAN 20) 10.20.0.0/16the L3 switch is doing the VLAN routing, and has an interface on each VLAN as follows:
VLAN 1: 10.10.0.100/16
VLAN 2: 10.20.0.1/16all hosts on the default subnet can get to anywhere (i.e. internet, VLAN 1, VLAN 20). They are using the pfsense firewall as the default gateway.
the pfsense firewall has default gateway the WAN IP
and has a route to the L3 switch for VLAN routing;
i.e. network 10.20.0.0/16 default gw 10.10.0.100/16all hosts on the 2nd subnet (VLAN 20) have default gateway the L3 interface on VLAN 20
they can telnet to the L3 switch interfaces, either 10.10.0.100 or 10.20.0.1, but they cannot telnet to any other hosts on VLAN 1.
(NB: I'm using telnet as a test tool; this applies to ssh, http etc)however, if I then ping a host on VLAN 1, then I can magically telnet/ssh/http etc to that same host for the next few minutes (until some period of time, the inactivity resets something which an ICMP packet magically solved…)
The system logs show that the firewall return route is being blocked with TCP:SA or TCP:R. However, my network diagram is pretty clear:
(1) vlan20 host -> L3 switch VLAN20 interface -> L3 switch VLAN 1 interface -> vlan1 host (2) vlan20 host <- L3 switch VLAN20 interface <- L3 switch VLAN 1 interface <- pfsense gatway <- vlan1 hostand despite adding explicit rules to allow the block, it is still blocked, so obviously this is not a firewall issue per se.
In order to resolve this (I'm not sure if this is just a bandaid or considered a proper fix), I had to enable system -> advanced -> firewall and nat, static route filtering (Bypass firewall rules for traffic on the same interface ).
Does anybody have any suggestions as to why this is happening?
-
Don't you enabled routing between vlans on switch?
O looks like packages are crossing vlans without passing through pfsense.
Are all vlans isolated foram each other?
-
Hi marcelloc.
Yes, the switch is doing the inter-vlan routing. But if the vlan's need to go to the internet, then they go through the pfsense firewall.
And yes, the vlan's are isolated from each (tagged), without the switch doing any routing, there will be no traffic seen on the other vlans if you do some packet sniffing.
So it seems strange that (1) on the one hand, it doesn't seem to be a firewall issue, since I can add plenty of rules to allow all traffic; (2) yet when I disable the firewall rules application between subnet, then it can start working!
-
Yes, the switch is doing the inter-vlan routing. But if the vlan's need to go to the internet, then they go through the pfsense firewall.
If switch route between vlans, the traffic must not reach firewall interface.
Check switch to see what is wrong.And yes, the vlan's are isolated from each (tagged), without the switch doing any routing, there will be no traffic seen on the other vlans if you do some packet sniffing.
Are you routing or not? First you said That you are routing and then you said not routing ???
-
I have a strange situation, I am unable to telnet/http/ssh etc to hosts on another subnet, until after I do a ping or traceroute, after which the telnet/http/ssh is successful straightaway.
My network setup is like this:
internet---pfsense----L3 switch-----default subnet (VLAN 1) 10.10.0.0/16 \ -------2nd subnet (VLAN 20) 10.20.0.0/16the L3 switch is doing the VLAN routing, and has an interface on each VLAN as follows:
VLAN 1: 10.10.0.100/16
VLAN 2: 10.20.0.1/16all hosts on the default subnet can get to anywhere (i.e. internet, VLAN 1, VLAN 20). They are using the pfsense firewall as the default gateway.
the pfsense firewall has default gateway the WAN IP
and has a route to the L3 switch for VLAN routing;
i.e. network 10.20.0.0/16 default gw 10.10.0.100/16all hosts on the 2nd subnet (VLAN 20) have default gateway the L3 interface on VLAN 20
they can telnet to the L3 switch interfaces, either 10.10.0.100 or 10.20.0.1, but they cannot telnet to any other hosts on VLAN 1.
(NB: I'm using telnet as a test tool; this applies to ssh, http etc)however, if I then ping a host on VLAN 1, then I can magically telnet/ssh/http etc to that same host for the next few minutes (until some period of time, the inactivity resets something which an ICMP packet magically solved…)
The system logs show that the firewall return route is being blocked with TCP:SA or TCP:R. However, my network diagram is pretty clear:
(1) vlan20 host -> L3 switch VLAN20 interface -> L3 switch VLAN 1 interface -> vlan1 host (2) vlan20 host <- L3 switch VLAN20 interface <- L3 switch VLAN 1 interface <- pfsense gatway <- vlan1 hostand despite adding explicit rules to allow the block, it is still blocked, so obviously this is not a firewall issue per se.
In order to resolve this (I'm not sure if this is just a bandaid or considered a proper fix), I had to enable system -> advanced -> firewall and nat, static route filtering (Bypass firewall rules for traffic on the same interface ).
Does anybody have any suggestions as to why this is happening?
Personally, i think you've got this set up wrong.
The client machines should have the switches routing interfaces as their default gateways - IE, on vlan 1, the switch has an interface with an IP address. That should be the default gateway for clients on vlan 1. On vlan 20, same thing. Then put a default route on the switch such that any other traffic gets routed to the pfsense box IP. This keeps traffic off of your firewall for inter-vlan routing, and it only has to deal with traffic meant for the internet. Unless you want to have a firewall between your vlans - which you might. -
marcello: the L3 switch is doing routing between the vlans. pfsense is not involved between vlans.
However, for the default vlan, pfsense is the default gateway, hence any destination to the vlan subnets, will be routed from the pfsense box to the L3 switch and onwards.
althornin: yes, it seems strange, I actually tried setting the default gateway for the default vlan to the L3 switch (as you point out, the interface that the L3 switch has on this subnet). However, I found that ntop would report traffic as one big chunk, from the MAC address of the L3 switch.
Prior to putting in the pfense box, this was the exact configuration I had with an old fedora box as firewall, i.e. the L3 switch only doing VLAN routing, and the default vlan using the firewall as the default gateway.
old setup:
default vlan: 10.10.0.0/24
L3 switch: 10.10.0.100/24
default gateway: 10.10.0.254/24 (the old firewall)second vlan: 10.20.0.0/24
L3 switch: 10.20.0.1/24
default gateway: 10.20.0.1/24 (i.e. the L3 switch interface in this subnet)new setup:
default vlan: 10.10.0.0/24
L3 switch: 10.10.0.100/24
default gateway: 10.10.0.1/24 (the new pfsense firewall)second vlan: 10.20.0.0/24
L3 switch: 10.20.0.1/24
default gateway: 10.20.0.1/24 (i.e. the L3 switch interface in this subnet)so apart from the IP address change, everything else is the same.
The reason I say it is strange, is that to resolve this issue, I turn off that pfense advanced rule to not check rules between subnets on the same interface. Which, although I don't have this situation now, in future I may need to block say vlan2/subnet2 from vlan3/subnet3. So it's just a stopgap whilst I figure this out.
-
The reason I say it is strange, is that to resolve this issue, I turn off that pfense advanced rule to not check rules between subnets on the same interface. Which, although I don't have this situation now, in future I may need to block say vlan2/subnet2 from vlan3/subnet3. So it's just a stopgap whilst I figure this out.
When you need to filter between vlans, tag second vlan to pfSense firewall and configure everybody gateway to pfSense respective vlan ip.
