What am I doing wrong (pfSense behind edge router)
-
Well yeah that setup is clearly asymmetrical... If you want to put a router downstream from your edge, then that should be on a transit network (no hosts on this network). Your using 10.0.0/11 as transit (path to other router) - which has hosts on it..
The simple correct fix would be to setup a transit from your edge router to your downstream router.
At a loss to how you came up with a /11 mask? That just makes no sense at all.. You can not possibly have anything close to 2.1 million devices on those segments..
Also a bit confused as to what your doing with default gateway in your different segments of .1, and then calling out .2 as the vlan?
You need to look at the paths between devices, and what that device uses as its gateway to get to other networks, you run into issues with the return traffic and hairpinning... If your going to put hosts on a transit network, then you either need to host route on each device, or you need to nat the downstream, etc. The correct solution is actual transit network to the downstream router (no hosts on it this network)... Be cause talking to or from these devices to your downstream networks is where you run into asymmetrical traffic.
You have 2 transit networks actually both with hosts on them.. your 10.32 is transit to get to your 10.64 through your cisco router..
Yeah that is a mess ;) You could fix it with transit off your edge router, and then your 2 downstreams connected to that transit. I have to run - but can draw it up for you after I get to work.
-
Ok, so you're saying instead of having the edge router LAN side be 10.0.0.1 and the FW WAN be 10.0.0.3 in the same network with all the wireless devices, create like a /29 network that only has routers on it and no hosts? Or a /30 between edge router and FW WAN and another /30 between FW LAN and the cisco router (currently 10.32.0.3)?
-
If it were I, I would need the pfSense at the edge.
-
@NollipfSense Yeah... the thing about that is my wife doesn't necessarily see the value of a dedicated firewall running on a laptop in our living room (where the only cable connection is) at all times. So I'm trying to keep as much stuff in the guest room/office as possible to appease her.
-
Hmm, we don't know what ports those things are connected to but I would assume the three VLANs are completely separated there in the switch?
In which case you do have, effectively, just a very large transit subnet between pfSense and the Cisco router, 10.32.0.0/11. It certainly doesn't need to be that big but shouldn't be a problem either.
I can't really see where you would have an asymmetric route assuming it actually is connected as shown in the diagram and the VLANs are separated entirely in the switch.Can we see the actual blocked traffic logs you're seeing?
Is it possible that desktop has some other path, via wifi maybe?
Steve
-
@duhduhdee My recipe for the perfect device to use in this situation is an Apple Mini...very small form factor, robust hardware, and aesthetically pleasing to look at. That's what I did...picked up an Apple Mini 2011 server on eBay for $200 and maxed out the memory (16GB).
-
@stephenw10 Sure I'll post some logs when I get home.
My desktop is definitely Gi0/20. The others, I'm pretty sure (will confirm when home) are:
Gi0/1 - 10.0.0.1 (Netgear edge router)
Gi0/3 - 10.0.0.3 pfSense Wan int
Gi0/6 - Pretty sure this was my work laptop
Gi0/7 - 10.0.0.4 Asus wireless router in AP mode
Gi0/9 - 10.32.0.1 pfSense LAN int
Gi0/11 - 10.32.0.3 Cisco router
Gi0/19 - 10.64.0.1 Cisco router
Gi0/20 - 10.64.0.3 Desktopre: the desktop having another path, i don't think so, it doesn't have a wireless NIC except for a USB one I plug in sometimes, and it's not connected now. I thought maybe the Asus router in AP mode was confusing things, so I disconnected it but it didn't seem to have much effect.
-
@NollipfSense That's a pretty good idea, she loves Apple stuff. Thanks
-
Mmm, well that looks correct from what I can see.
Gonna have to see the blocked traffic logs I think.
Steve
-
The devices were all connected to the switchports that I thought they were.
Idk what format would be the most helpful, so here's a screenshot.
Here also is part of my states summary page by IP pair:
I seem to be connected enough to make this post from the ip address (10.64.0.3) and VLAN that I want to be on. But frustratingly, I can go to the youtube page below and see all the thumbnails load and get a bunch of the page formatting, but the video itself just loads and loads forever.
As I write this, I seem to also be blocking a bunch of 239.255.255.250:1900 UDP, which is Simple Service Discovery Protocol. But I don't know why I need that to communicate with most sites on the internet.
Thanks for your help.
-
That network is clearly asymmetrical - you can NOT put hosts on a transit network without issues, unless your going to host route or nat.. I have gone over this so many times its becoming sad!!!
I got tied up in meetings all day at work so unable to draw up how it would be done correctly... And now going to watch some TV with the wife, just got home... But will for sure draw up how you would do this correctly with out being asymmetrical..
Again you have to follow the traffic when devices are talking to each other from the different segments and where their default gateway is to see the asymmetrical flow.. If your just talking traffic to and from the outside internet then its not a problem.. The problem is when you have devices talking to each other over such a setup.
-
@johnpoz I appreciate your help.
-
So here is why its asymmetrical
Here is how you fix that - with a transit network.
If you want to put hosts on a transit network, then you need a route on every host saying hey to get to the downstream network go to X vs your default gateway. Or you have to nat at the downstream network.
-
@johnpoz I really appreciate the diagram.
I think I understand you, but I don't think I have the hardware to accomplish it. I have two Cisco routers, and each only has 2 ports. I also have the edge router, a Netgear R7800, which has one WAN port and then 4 switch ports.
The edge router in your diagram has at least 3 ports. I don't think I have any device capable of routing in more than 2 directions.
So I guess I either need to buy another router or figure out how to NAT what I have.
-
do it with vlans then.
-
@duhduhdee said in What am I doing wrong (pfSense behind edge router):
So I guess I either need to buy another router or figure out how to NAT what I have.
Or you could get a cheap managed switch (avoid TP-Link) and use VLANs.
-
Yeah he already has a cisco switch there that clearly can do vlans... So he should be good to go as long as his edge router can do vlans...
-
Unless I'm misreading this the connection seeing issues is from 10.64.0.3 to external IPs, which should not be asymmetric.
Yes you would certainly have asymmetry problems connecting from, say, 10.64.0.3 to 10.2.0.4. Assuming the Cisco is not NATing, which I assume it isn't since there appears to be a static route to 10.64.0.0/11 on pfSense.
None of that blocked traffic you've shown looks like the result of asymmetry either. It would all be TCP flagged if so.
Steve
-
Yeah, I'm reading in the manual about the edge router's VLAN options. It talks about "Setting up a bridge for a VLAN Tag group". Which I guess bridge = trunk, in netgear's terminology?
If that works, that would be ideal. If all else fails I'll take the laptop pfSense is running on, connect it to the modem, make it the edge router, put the netgear router in AP mode, get rid of the other routers and subnets and just let the cisco switch separate everything by VLAN.
Maybe my wife won't mind if I try to arrange things around the laptop in a pleasing fashion.
Thanks everyone.
-
To external IPs it shouldn't be a problem with asymmetrical true... But the overall design is flawed... Even if he has no need to talk to other devices at this time... The design is not correct to ever allow internal hosts to talk to each other at some future point in time.
Hosts should never be on a transit. If you have something that forces you do to it, then you have to host routing or natting.. Neither of which are desired things to have to do.
