Need Help Resolving ?Asymmetric Routing? Issue in a Network with pfSense and Netgear Managed Switch (GS724Tv4)
-
@johnpoz
AND it was exactly what you have pointed out. One of my proxmox instances in the LAN had 2 ip addresses assigned to the host setup, one in VLAN10 and one in VLAN100 which basically let to your described behavior.Thanks a lot for taking so much time and reading through all of these messages and screenshot. I have learned a lot and I am so thankful that people like you exist. Please keep up the great work you are doing.
-
@oliverus000 glad you got sorted, and I could help you figure it out.
edit
If you need/want to put vms on multiple networks, that is fine.. And sure ok the Host is actually connected to multiple networks. But the host itself doesn't really need an IP in these multiple networks.. Just the VMs do.. I do same sort of thing.. I have multiple vlans trunked to my VM host via an interface.. So I can put a VM on network A or B or C even.. But to manage the host, to talk to the host to do HOST things, the host only has an IP in network A for example..All of these networks are isolated at layer 2.. An no VM actually has interfaces in more than one network. So there is no concern that when talking to a VM that is on network B from a device in network A, would the vm be able to answer back via network A.
Now could this be a security issue.. Yeah it could be, if the HOST itself was compromised.. It does have interface that could be used in multiple networks.. But in day to day operation this is not a issue. And this host is not exposed to the public internet or anything.. The only way to talk to this host management IP is via being on the management network.
-
J johnpoz referenced this topic on
-
@johnpoz
I only have 1 LAN port setup on pfsense with no DHCP. All VLANs are defined in my Cisco L3 switch and use DHCP off the Cisco L3 switch. The gateway for the shared VLAN connecting to my Pfsense is defined as a VLAN for my Cisco L3 switch and defined only as a LAN port, no VLANs on Pfsense. I use the gateway on the shared VLAN pointing for the clients as the L3 switch and the gateway for Pfsense, no VLAN defined, pointing to my WAN port. The default routing on the L3 switch is pointing to the Pfsense LAN port so unknown IPs are routed to Pfsense. This works fine.It is easier to setup. On my last setup I had a point-to-point network to transfer from my L3 switch to Pfsense. At home I see no difference.
-
@coxhaus yeah that is a transit network, if you have not hosts on it.. That has zero anything to do with this thread.
Why not just put everything in 1 vlans to be honest if your not firewalling between your vlans.. Other than different broadcast domain there is little advantage to that sort of setup.. Do you have any acls setup between these vlans? if your not filtering traffic between the vlans.. Why not just put them all on 1.. Your making your life more complicated for no actual benefit of being able to easy firewall between your segments.
-
@johnpoz
I do, I use ACLs for my guest VLAN on my L3 switch. I have my printer defined on my guest VLAN and share a 29-bit mask so my guests can print and my main LAN. I also run3 Cisco wireless APs since I have a large home with 12-foot ceiling. I have separate VLANs with separate SSID setup for roaming for guest and my main LAN. Cisco 150ax wireless APs setup as 1 virtual wireless AP all grouped together.Using the firewall on my Pfsense router would be slow compared to my line speed Cisco L3 switch. Plus, that would mean Pfsense is doing the routing not my switch. If you use a trunk port you are not doing L3 switching, you are doing L2 switching.
PS
I had a server rack on a separate VLAN, but when I retired I turned off my rack. -
@coxhaus again - has nothing to do with the user problem.. I would be happy to discuss routing on a switch vs a router, and or using a trunk port over 1 physical interface drawbacks and benefits, etc..
We already fixed the OP issue, which he had multihomed device which was answering via the interface it had in the client devices network.
Yes there are times it make sense to route at the switch.. and other times it makes sense to route at pfsense, which is never going to be your switching infrastructure unless you have one of the appliances that had a built in switch..
None of which had anything to do with the users problem.. Now if they were multihoming on a switch with asymetrical flow, they prob would of never seen or ran into the issue since they wouldn't of been flowing through a stateful firewall.
Firewall rules are much easier to create and manage at pfsense vs an ACL on a switch that is for sure ;)
-
@johnpoz I just fed your reasons why I would not want to use 1 VLAN as you requested.
I route at the switch to pfsense. If you don't route at the switch you are not doing layer 3 switching on the LAN side. -
@coxhaus You seem to keep bringing up this layer 3 switching.. This isn't a thing ;)
You can route at your layer 3 switch, sure.. But your not "switching" at layer 3.
-
@johnpoz
I think you are within the switch when you go from one network to another but probably not to Pfsense. If you are using Pfsense for layer 3 then no you are not L3 switching. The different network packets need to go to Pfsense to figure out where to go. Using L3 switching they are switched between networks on the switch at line speed from port to port, no Pfsense is involved. Yes, they are layer 3 so you could call it line speed routing but not if you use Pfsense.I use Pfsense for my internet connection, my front door, mainly firewall, DNS, etc.. All my local routing and data is handled by my Cisco devices. Traffic destined for the internet passes from my Cisco devices to Pfsense and vice versa. DHCP and all VLANs are defined in my Cisco devices. I only have 1 LAN port defined on Pfsense which I route to, no VLANs.
-
@coxhaus yeah I get what your doing, its not uncommon in larger networks.. And sure if you need 10ge for example between segments.
But I can route at pfsense, I have no "speed" issues - its more than capable of routing between my networks at full gig speed. Many users of pfsense will just route and firewall with it, because it makes it easier to do the "firewall" part between their segments. Creating firewall rules are far more complex at a L3 switch doing routing than on pfsense..
And the dhcp server on pfsense is more than likely way more robust and features than the dhcp server you have on some L3 switch. If I want to sniff traffic between my networks it really easy to do because all the traffic between networks passes through pfsense.
You have a simple transit/connector network between your edge and core if you will.. Its just not called L3 switching was my point ;)
Here is example of setup
You are doing the setup on the left, vs the one on the right. You could be doing a combination of both, etc.. My switch can do routing, just not currently leveraging it for example. Because I have no need.. And its much easier to control intervlan/network traffic with the firewall rules on pfsense than on my switch..
The OP problem would prob not presented itself if they were doing routing on their switch vs pfsense, because the switch wouldn't of been stateful. That is not necessarily a "good" thing if you will.
Also many users might not even have a switch capable of doing the routing between networks.. It might be a simple L2 switch that while can do multiple vlans, doesn't have the routing or ACL capabilities.
-
@johnpoz Yes, I managed a larger Cisco network many years ago when I worked. I run the same setup basically.
We all called it L3 switching back then. You are moving traffic from port to port in a switch. It just happens to be network to network. Routing does not fit. We used eigrp routing from routers and L3 switches. L3 switching seems to fit vs just routing because it is happening at line speed not like regular slow routing.What would you call it to signify line speed routing? I think L3 switching works.
Yes, you get a lot of users thinking they are doing L3 but really, they are doing L2 switching. Most people don't understand L3 switching. You need to be more a Cisco person. And now with 10gig and greater NICs needing L3 switching is getting farther out than it was 20 years ago. But when you push the edge a router and a L2 switch will break down compared to using a L3 switch.
Oh, about your drawing mine is a little different my connecting L3 switch network is a VLAN port 192.168.0.2, not real IP address, but close which plugs into a NIC in Pfsense 192.168.0.1, VLAN not defined. So, both ports are defined in the same network 192.168.0.0/24. All network ports are defined as a VLAN but not the NIC which is not defined as a VLAN just regular IP address. Gateway for NIC IP address 192.168.0.1 is WAN. This I think is the default gateway on Pfsense but I am not sure as we are into the GUI and I am no expert on Pfsense.
Gateway for VLAN on switch is the VLAN gateway on the L3 switch. All local traffic is routed by my L3 switch. Unknown traffic is routed to Pfsense NIC 192.168.0.1 my defualt route on my L3 switch. There are Pfsense firewall rules that allow networks on my L3 switch to pass.
I don't use a network to pass traffic on. I did on an older Pfsense setup but I decided it was not need for me and my home. My way is easier to setup. You just add the L3 switch to the network and what ever DHCP IP address is assigned is used. So, when I got a new Cisco L3 switch I went this way. It is easier to change switches for me.The best DHCP server I have used is Microsoft's DHCP server. We used it at work and I used it at home before I turned off my rack. I tought it worked well.