Routing with L3 switch
-
Hi,
I am using pfsense 2.3.4, and my L3 switch Brocade ICX 7250. I configured VLAN in L3 switch. I can ping the 8.8.8.8 & google.com from L3 switch but not pinging from PC, Ping details are below,Firewall IP details
–-------------------
WAN Ip: 117.83.134.238/30
gateway ip: 117.83.134.237Lan: 192.168.2.1/24
L3 Switch
Ip route: 0.0.0.0 0.0.0.0 192.168.2.1
dns: 192.168.2.1 8.8.8.8eth 1/1/1 ip: 192.168.2.10/24
vlan 100
tagged eth 1/1/2
router-interface ve2 ip address 192.168.100.1/24vlan 101
tagged eth 1/1/2
router-interface ve3 ip address 192.168.101.1/24L2 Switch
vlan 100
tagged eth 1/1/1
untagged eth 1/1/2vlan 100
tagged eth 1/1/1
untagged eth 1/1/3PC
192.168.100.3/24
gateway: 192.168.100.1
P dns: 192.168.2.1
S DNs: 8.8.8.8Ping from firewall to 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=59 time=65.019 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=59 time=58.046 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=59 time=59.222 msPing from firewall to L3
PING 192.168.2.10 (192.168.2.10): 56 data bytes
64 bytes from 192.168.2.10: icmp_seq=0 ttl=64 time=0.507 ms
64 bytes from 192.168.2.10: icmp_seq=1 ttl=64 time=0.456 ms
64 bytes from 192.168.2.10: icmp_seq=2 ttl=64 time=0.479 ms--- 192.168.2.10 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.456/0.481/0.507/0.021 msPing from firewall to VLAN100
PING 192.168.100.1 (192.168.100.1): from 192.168.2.1: 56 data bytes
--- 192.168.100.1 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet lossPing from firewall to VLAN100
PING 192.168.100.3 (192.168.100.3) from 192.168.2.1: 56 data bytes
--- 192.168.100.3 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet lossPing from L3 to firewall
ICX7250-24 Router#ping 192.168.2.1
Sending 1, 16-byte ICMP Echo to 192.168.2.1, timeout 5000 msec, TTL 64
Type Control-c to abort
Reply from 192.168.2.1 : bytes=16 time<1ms TTL=64
Success rate is 100 percent (1/1), round-trip min/avg/max=0/0/0 ms.Ping from L3 to 4.2.2.2
ICX7250-24 Router#ping 4.2.2.2
Sending 1, 16-byte ICMP Echo to 4.2.2.2, timeout 5000 msec, TTL 64
Type Control-c to abort
Reply from 4.2.2.2 : bytes=16 time=159ms TTL=55
Success rate is 100 percent (1/1), round-trip min/avg/max=159/159/159 ms.Ping from L3 to vlan100 pc
ICX7250-24 Router#ping 192.168.100.3
Sending 1, 16-byte ICMP Echo to 192.168.100.3, timeout 5000 msec, TTL 64
Type Control-c to abort
Reply from 192.168.100.3 : bytes=16 time=1ms TTL=64
Success rate is 100 percent (1/1), round-trip min/avg/max=1/1/1 ms.Ping from L3 vlan100 to firewall & 4.2.2.2
ICX7250-24 Router#ping 192.168.2.1 source 192.168.100.1
Sending 1, 16-byte ICMP Echo to 192.168.2.1, timeout 5000 msec, TTL 64
Type Control-c to abort
Request timed out.
No reply from remote host.ICX7250-24 Router#ping 4.2.2.2 source 192.168.100.1
Sending 1, 16-byte ICMP Echo to 4.2.2.2, timeout 5000 msec, TTL 64
Type Control-c to abort
Request timed out.
No reply from remote host.Ping from pc
C:\Users\lenovo>ping 192.168.100.1
Pinging 192.168.100.1 with 32 bytes of data:
Reply from 192.168.100.1: bytes=32 time<1ms TTL=64
Reply from 192.168.100.1: bytes=32 time<1ms TTL=64
Reply from 192.168.100.1: bytes=32 time<1ms TTL=64
Reply from 192.168.100.1: bytes=32 time<1ms TTL=64Ping statistics for 192.168.100.1:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0msC:\Users\lenovo>ping 192.168.2.10
Pinging 192.168.2.10 with 32 bytes of data:
Reply from 192.168.2.10: bytes=32 time=25ms TTL=64
Reply from 192.168.2.10: bytes=32 time=2ms TTL=64
Reply from 192.168.2.10: bytes=32 time=44ms TTL=64
Reply from 192.168.2.10: bytes=32 time=2ms TTL=64Ping statistics for 192.168.2.10:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 2ms, Maximum = 44ms, Average = 18msC:\Users\lenovo>ping 192.168.2.1
Pinging 192.168.2.1 with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.Ping statistics for 192.168.2.1:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),C:\Users\lenovo>ping 8.8.8.8
Pinging 8.8.8.8 with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.Ping statistics for 8.8.8.8:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),I think it's reverse routing issue in pfsense firewall, i tried to add the same in System -> RoutingStatic -> Routes
but it's not working. can some one help me on this issue.
-
Yes, it is most likely a routing issue.
Make sure you add your L3 switch (192.168.2.10) as a gateway on the LAN interface
Next add a static routes to 192.168.100.0/24 and select the LAN gateway you just added from the drop-down list.
You might also want to repeat that for 192.168.101.0/24 so that both VLAN 100 and 101 are reachable.
Lastly, make sure you have firewall rules to allow these subnets out. -
I don't see a router-interface on the transit network to pfSense. Pretty sure you need that. What you have looks like it's for management only.
vlan X
untagged eth 1/1/1
router-interface ve1 ip address 192.168.2.10/241/1/1 connected to pfSense . Note that I would tag this on principle and assign transit to vlan X on pfSense. That way you can tag other VLANs to your switch if necessary without mixing tagged and untagged traffic (see brocade dual-mode ports for that).
I would not put the Layer 3 switch on LAN with a bunch of hosts. There should be two things on 192.168.2.0/24 - pfSense and the Switch. No other hosts or you will have asymmetric routing issues which are unnecessary and bad.
-
Derelict,
Nice diagram, very clear! -
There are only 4 important things which you need to get right using a layer 3 switch.
1. Make sure the layer 3 switch uses the LAN interface on pfsense for it's default gateway.
2. With static routes pfsense needs routing statements for all networks on the layer 3 switch pointing to the gateway IP address on the layer 3 switch
3. You need firewall statements on pfsense to allow all the networks on the layer 3 switch out through the firewall on pfsense
4. The PCs on the layer 3 switch need to have the layer 3 switch's network as their default gatewayI think these are the important steps which make a layer 3 switch work with pfsense or any router.
-
5. Don't put any other hosts on LAN with the switch or you will have asymmetric routing issues.
-
5. Don't put any other hosts on LAN with the switch or you will have asymmetric routing issues.
I have heard that before but it works if you let the layer 3 switch handle the local routing. And if you don't believe me try it.
-
It is still asymmetric and it is still bad design. Even if it works for you now it will likely bite you in the ass later.
-
It may be bad design but it happens even in production environments.
-
Bad design is bad design, regardless of scale.
-
I think both agree on bad design.
