Unifi Dream Machine and PFSense
-
Something is up with my UDM and PFSense. If I access SSH from a different VLAN, the connection breaks in 30 seconds to a minute. With HTTPS, the Network management page often fails to load (Bad Request) but occasionally loads (but not Protect or the main Settings page). On the same VLAN, it works perfectly. SSH/HTTPS works great for any other service across VLANs.
Any ideas on how I could have misconfigured PFSense? Nothing in the logs regarding this IP being blocked. Rules are as such, in order, for each VLAN:
- Allow rules for specific protocols (to route between VLANs)
- Deny all traffic from private subnets
- Allow WAN access (for Internet)
-
Clearing the states table for the rule (having port 443 and 22) made HTTPS working again. SSH still times out. PFSense was rebooted about a day ago.
-
@tyler-montney-0 just trying to figure out ssh and https failson pfsense or on wifi ap?
-
How are those things connected? A diagram might help here.
Is it all SSH sessions that fail or just to pfSense? Or to the UDM?
That sort of timeout period says asymmetric routing to me.
Steve
-
@stephenw10 said in Unifi Dream Machine and PFSense:
asymmetric routing to me.
Yeah - lets see a diagram... Dream Machine is a wifi router, and also doing vlans in the network... Love to see how you have this all connected. And using it with pfsense.
If you have networks hanging off pfsense, and the dream machine.. Are pfsense and dream machine connected via a transit?
-
My bad.
- VLAN A is to appease the UDM as I don't want to use it for its network capabilities. It won't work without a WAN connection. It's used for nothing else.
- VLAN B is a main VLAN which other servers share.
- VLAN C is my management network, which I manage other VLANs from. This is where I have connection issues, from C to B.
It's SSH and often HTTPS from the management PC to the UDM. I assume PFSense is involved because when I connect from the same subnet, there is no issue (so PFSense is not involved).
What do you mean by "Are pfsense and dream machine connected via a transit"?
@mr-rosh Fails on wired, from the management VLAN to the server VLAN.
-
@tyler-montney-0 that doesn't actually provide any useful info.
Where are these vlans being routed.. Is that connection in pfsense to your switch over what vlan?
If vlan A is your UDM wan, and pfsense lan - is anything else in this vlan A?
-
Still looks like asymmetry.
Assuming the switch is layer 2 then pfSense is routing between the VLANs. When ypou connect to the UDM IP in VLAN B from the host in VLAN C pfSense routes that. Assuming the UDM doesn't have an interface in VLAN C (if it does you could just connect to that) it has to reply using whatever route it has to reach VLAN C and that might be it's default route via the WAN. Asymmetry.
Add a route to the UDM or an interface in VLAN C.
Or hack a workaround by adding a very specfic outbound NAT rule in pfSense on VLAN B.Steve
-
@stephenw10 said in Unifi Dream Machine and PFSense:
Still looks like asymmetry.
Yeah, without some specifics that clearly say not.. Then for sure is, like UDM wan (vlan a) is pfsense lan (and no other devices on this network - so its just a transit vlan).. But still don't know where vlan C sits? Not shown as being routed anywhere? Those edge switches depending on model/firmware might be able to do some really basic routing?? I do recall some sort of announcement from unifi on that.
He says vlan C is the problem.. Ok - where is it routed at? On the switch? If so then B becomes transit and yeah could be asymmetrical for any clients on B talking to C, or C talking to B..
Need more details..
-
"Where are these vlans being routed"
Do you mean what's tagging the VLANs? If so, the switch is.
"If vlan A is your UDM wan, and pfsense lan - is anything else in this vlan A?
Nothing on VLAN A except to provide it "WAN" access. Added for completion, since the UDM is a very finicky piece of hardware.
"He says vlan C is the problem.. Ok - where is it routed at?"
It's the the native VLAN (1). I would assume, PFSense. But as Stephen says, if the UDM has no interface for this network, it would cause the issue I'm seeing.
"Assuming the switch is layer 2"
It is indeed layer 2.
"Assuming the UDM doesn't have an interface in VLAN C"
I don't believe it does. How does it return any data in asymmetric routing?
-
Routing has zero to do with tagging.. No that is not what I mean ;)
What is the gateway of vlan C?
Vlan C can not talk to anything other than other things in vlan C unless its routed..
-
Right, what I figured.
VLAN A: 192.168.0.1
VLAN B: 192.168.1.1
VLAN C: 192.168.2.1I've simply created VLANs for each interface, then created interfaces with those IPv4 values. Finally, created rules to filter traffic between them.
-
Where - On pfsense? That makes no sense at all, since you have the UDM there showing its got a wan and lan that implies routing and nat. Unless you specifically called out that it was not natting... But then why would it have interfaces you call wan and lan if not doing routing, etc..
If not doing nat, and just a downstream router from pfsense via vlan A (transit) then pfsense wouldn't also have an interface in B..
If you were just using UDM as AP with 2 different wifi ssid off it, then you woudn't call them wan and lan..
Like I said your drawing no info, and if anything just confuses the matter because you have not info on it.. Your connection from pfsense to the switch doesn't show what vlan that is, or if its a trunking with all three vlans on it, etc.
-
Ah, there's the disconnect. I apologize, I didn't realize I did that. I'm so used to, in my environment, not treating the UDM as a router (despite it performing those functions). I truly wish the UDM was designed differently.
To clearly state it: I am attempting to not use the UDM as a router wherever possible. However, it is indeed a router and cannot get rid of that aspect.
Here is an updated diagram (wasn't allowed to edit my last one).
I apologize for not providing the details you need; however, I seem to be at a disadvantage here. I'm trying my best not to be the typical forum user who demands help in all caps and provides no details other than "it's not working".
It's probably as @stephenw10 said. I'll inspect my interfaces on the UDM and add a VLAN 1 interface if none is present.
-
@tyler-montney-0 said in Unifi Dream Machine and PFSense:
"Assuming the UDM doesn't have an interface in VLAN C"
I don't believe it does. How does it return any data in asymmetric routing?Yes, that's a good question since this would be blocked entirely if the traffic flow were as I described.
What you are seeing if typical of having both routers in the same subnet but a host connected to only one of them. The target responds to the wrong router and is sent an ICMP redirect allowing it to reply directly. That expires after 30s or so and subsequent traffic is blocked.
That's so much like what you're seeing that's probably what it is. That implies we are missing something from the diagram or have made an incorrect assumption. An interface or route on the UDM perhaps.But, yeah, if you can add a VLAN C interface in the UDM you can just access it directly using that avoiding any possibility of asymmetry.
Steve
-
@tyler-montney-0 said in Unifi Dream Machine and PFSense:
However, it is indeed a router and cannot get rid of that aspect.
Sure you can.. Just don't use its routing features.. It runs its own controller.. Setup whatever networks you want to use for wifi.. Tag them.. Now all that is being done is bridge the wifi the wired L2.. Connect it to your wired network via either multiple ports as uplinks on the respective vlans. Or just trunk 1 and carry all your wireless vlans on it, and whatever management vlan your using on the UDM.
Just like you would do with any other soho wifi router that you just want to use as AP.. Connect it to network via one of its lan ports, turn off its dhcp server = AP..
Just because something can route - doesn't mean you have to use it as that or it needs to be used to function. Its a smart 4 port switch, and 4x4 AP that does vlans.. There is nothing saying you have to let it route..
Taken to basics all an AP Is bridge between wifi and wired.. Sure there is some authing to connect to the wifi side. But after that happens it just bridges traffic from the wifi to the wire.. And either puts tag on it, or doesn't when it puts it on the wire.
If you google to use UDM as AP you run into you can't because it runs its own controller and can not be adopted by another controller as just an AP.. But there is nothing saying you can not just use it as AP in your own network.. When it comes down to it - its a fancy soho wifi router. It has some routing features, it has a 4 port switch and AP.. Since it runs the controller software on itself. You don't need to adopt it by anything. Just setup wifi networks how you want, turn off its dhcp server and connect it to your network via one of its lan ports, or multiple if you want to use the different ports as uplinks for specific vlans, vs doing trunking..
The controller software is nothing more than the web gui you see on any other wifi router..
edit:
Only problem you might have is if wan is not connected, the UDM might not be able to download updates when you want to update the controller software or firmware on the AP? But I would hope there would be other ways to put the firmware on the thing without having to pull it from the internet. I can put firmware on my AP by just pointing it to URL to grab it from other than the unifi urls..But you could always just create a new vlan on pfsense, and use that for the UDMs wan network - and just not do anything with it other than letting the UDM talk to the internet to grab updates.
If I had one to play with - be happy to walk you through a setup.. But was never a fan of such a device.. AP should mounted correctly in the area to provide wifi coverage for the area your trying to provide wifi for.. Why would I want that in the same box that is doing my routing/firewalling.. Other option is sell the thing you really have no use for other than AP, and just get actual AP ;) And mount it in the area that best provides wifi coverage for what you want to cover in wifi, or get multiple APs, etc.
edit2: I just looked if I could find a used one on ebay for low enough price to play with it.. They seem to be going for retail price or higher... Sell it and just get an AP or 2 or 3 of them ;)
-
UDM does actually have an interface on the management VLAN.
@johnpoz I'll review your comment tomorrow.
-
Ah, well you should be able to connect to the UDM using it's IP in the management VLAN from that host in the same VLAN without issue. That traffic would not go through pfSense at all.
Steve
-
"Or just trunk 1 and carry all your wireless vlans on it, and whatever management vlan your using on the UDM."
I could've sworn this didn't work, just based how wacky Ubiquiti is. No harm in trying again.
"Only problem you might have is if wan is not connected, the UDM might not be able to download updates when you want to update the controller software or firmware"
WAN is required at set up and to download updates.
"and just not do anything with it other than letting the UDM talk to the internet to grab updates."
Yep, whitelisting works wonders here.
"If I had one to play with - be happy to walk you through a setup.. But was never a fan of such a device."
Same here, but unfortunately nothing rivals it (that I know of). You're forced to use it if you want to use access control or the NVR. I would've gone with Axis, but it's like triple the cost. It's a shame there's such a disconnect between development and real-world. There's not even a way to automatically back up the recorded video (not officially).
-
So running off just one interface, on the UDM, won't work (just tested). You have to define a "Network" (aka an Interface) to tell it which VLAN/subnet it's on. Otherwise, it won't know what to do with the tagged traffic.
