UDP broadcasts to WAN
-
Hi,
I'm not sure that this is a NAT problem, but it seems to have some connection to it, so..
The problem is that UDP broadcasts are being forwarded from LAN to WAN. I've seen three different kinds:
- from local subnet to port 17500, these are NATted on the way
- from local subnet to port 137 and 138, these go as-is
- from unknown subnet to port 138, no NAT here either
The system is Pcengines Alix and pfSense 2.2. LAN side has a bridge for LAN and WLAN. WAN side is multi-WAN with 3G as backup. But the UDP problem has been seen on the ethernet WAN. The bridge might have a connection to the problem.
Risto
-
Ok, my mistake..
The problem was caused by multi-WAN. I didn't realize that the Gateway setting in a firewall rule overrides the routing table. I'd see it more logical to take the local routes into account.
The remaining question is: how did the kind 2 packets get through NAT as is?
Risto
-
"The bridge might have a connection to the problem."
Yeah think ;) Broadcast traffic is not sent out to other networks no matter what routing you have in place.
-
Well, that's what I thought, but I was obviously wrong.
The bridge is there to allow one DHCP to serve both LAN interfaces, WAN is not part of it. And the strange packets look like routed: the source MAC is that of the WAN.
Risto
-
Again packets that are broadcast would not go out to another network. Unless they were bridged, or maybe IGMP proxy - did you have that setup?
Can you post a sniff of this traffic you were seeing going out the wan?
-
The bridge is there to allow one DHCP to serve both LAN interfaces
Which is totally unneeded. That one DHCP can serve two (or really any number of) different subnets just fine and the firewall will route packets between those just fine as well.
-
Again packets that are broadcast would not go out to another network. Unless they were bridged, or maybe IGMP proxy - did you have that setup?
No.
@johnpoz:Can you post a sniff of this traffic you were seeing going out the wan?
23:54:18.884766 00:0d:b9:17:cb:28 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 92: (tos 0x0, ttl 128, id 1685, offset 0, flags [none], proto UDP (17), length 78)
192.168.1.31.137 > 192.168.1.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
where 00:0d:b9:17:cb:28 is WAN (vr0) MAC,
LAN (bridge0) IP is 192.168.1.7,
WAN (vr0) IP is 80.x.x.x (DHCP).
@doktornotor:Which is totally unneeded. That one DHCP can serve two (or really any number of) different subnets just fine and the firewall will route packets between those just fine as well.
I agree. But my next goal would be to serve 20 VLANs.
Risto
-
So your going to bridge 20 vlans together?? What does that have to do with bridging interfaces for dhcp?
You must have something F'd up in your bridge if your sending out broadcast out your wan.. Or you installed something on pfsense to send it out, or is this a VM running on virtualbox and you didn't remove the windows bindings from the interface and that is windows sending that out. Is pfsense vm?? OR is this hardware? Ah from mac looks like PC Engines..
Again lets say this one more time so you understand it BROADCAST traffic is NOT routed.. Pfsense would not get a FF:FF packet on its lan and say oh lets send this out my wan. It wouldn't happen! Unless you have some sort of bridge, or some kind of broadcast proxy, etc.
Did you install say samba on pfsense? So sniff on your lan at same time your sniffing wan, where is the broadcast coming from if you say pfsense is forwarding it.
-
So your going to bridge 20 vlans together?? What does that have to do with bridging interfaces for dhcp?
Yes. I want to avoid configuring 20 subnets. Perhaps I could try one subnet for the vlans and one for the wlan.
@johnpoz:You must have something F'd up in your bridge if your sending out broadcast out your wan.. Or you installed something on pfsense to send it out, or is this a VM running on virtualbox and you didn't remove the windows bindings from the interface and that is windows sending that out. Is pfsense vm?? OR is this hardware? Ah from mac looks like PC Engines..
Yes, it definitely seems to be caused by the bridge: I took it away, and the symptoms ceased. It is a PC Engines Alix 6b2.
@johnpoz:Again lets say this one more time so you understand it BROADCAST traffic is NOT routed.. Pfsense would not get a FF:FF packet on its lan and say oh lets send this out my wan. It wouldn't happen! Unless you have some sort of bridge, or some kind of broadcast proxy, etc.
Unless there's a bug in it. Well, I don't have any such daemons, just the bridge. But bridged traffic should have the original sender's MAC address. I've changed these two system tunables from their defaults: net.link.bridge.pfil_member=0, net.link.bridge.pfil_bridge=1. And I've defined all bridge ports as edge and private: I don't want them to communicate to each other.
@johnpoz:Did you install say samba on pfsense? So sniff on your lan at same time your sniffing wan, where is the broadcast coming from if you say pfsense is forwarding it.
No Samba. I've seen it coming from a windows box on the lan.
Thanks for your comments so far.
Risto
-
Which is totally unneeded. That one DHCP can serve two (or really any number of) different subnets just fine and the firewall will route packets between those just fine as well.
I agree. But my next goal would be to serve 20 VLANs.
So, you are going to bridge 20 VLANs? Why bother in the first place? Just get a dumb switch and put all on a single LAN. WTF. This must be some disease with the bridging. Lets bridge VLANs, lets bridge WLAN with wired LAN, lets bridge 10 ports to waste hardware trying to turn a router into a dumb switch.
??? :o
-
So, you are going to bridge 20 VLANs? Why bother in the first place? Just get a dumb switch and put all on a single LAN. WTF. This must be some disease with the bridging. Lets bridge VLANs, lets bridge WLAN with wired LAN, lets bridge 10 ports to waste hardware trying to turn a router into a dumb switch.
??? :o
There is a reason for everything. In this case I want to separate the vlans (private bridge ports) so they don't see each other. There is no excuse for pfsense's bridge not working as it should.
Risto
-
Yes. I want to avoid configuring 20 subnets.
Another one who thinks administering a network isn't going to be work. It boggles the mind.
-
There is a reason for everything. In this case I want to separate the vlans (private bridge ports) so they don't see each other. There is no excuse for pfsense's bridge not working as it should.
What you're doing is nonsensical. I don't think you quite understand what a "bridge" does.
There are other, better ways to do what you're trying to do. One being a separate VLAN interface for each VLAN like any sane person would do. Another would be private VLAN edge (protected) ports on a switch with everyone on the same VLAN. Cisco 2950s do this 10/100 and are essentially free. Most "cheap" web-managed switches (trendnet, d-link, TP-link, etc) can fake this with "asymmetric VLANs".
-
"I want to separate the vlans (private bridge ports) so they don't see each other"
Huh - so you don't want clients seeing each other but you want to connect the vlans together with a bridge? Lets taking bridging to its simplest level. Packets come in one interface, they get sent out all bridge members interfaces. So if you want devices on different vlans not to see each other - why would you bridge them?
Sure you can setup firewall rules between a bridge and isolate them that way - but if you don't want them to see each other or only have a few exceptions - why make it a bridge in the first place?
If you have 20 vlans, then setup 20 vlans and 20 network segments and 20 dhcp servers. That is the way you would do it if you ask me.. Or if you don't want to setup dhcp servers, then send them to a dhcp server that supports different scopes, etc. I don't think pfsense allows for serving up multiple scopes, you can have multiple pools but don't think it can serve up multiple segments off one server instance?
Im with Derelict here, we all like shortcuts to min admin - but running a network takes work, if it didn't than any user could do it ;)
-
What you're doing is nonsensical. I don't think you quite understand what a "bridge" does.
I think I do. I'm only trying to use bridge config here as a means to achieve my goal, which I've hopefully explained already. As the bridge ports are all private, it's no usual bridge.
@Derelict:There are other, better ways to do what you're trying to do. One being a separate VLAN interface for each VLAN like any sane person would do. Another would be private VLAN edge (protected) ports on a switch with everyone on the same VLAN. Cisco 2950s do this 10/100 and are essentially free. Most "cheap" web-managed switches (trendnet, d-link, TP-link, etc) can fake this with "asymmetric VLANs".
There are always different ways of doing things. I was disappointed, when I understood, that my switch doesn't support that.
@johnpoz:"I want to separate the vlans (private bridge ports) so they don't see each other"
Huh - so you don't want clients seeing each other but you want to connect the vlans together with a bridge? Lets taking bridging to its simplest level. Packets come in one interface, they get sent out all bridge members interfaces. So if you want devices on different vlans not to see each other - why would you bridge them?
The bridge in mostly for dhcp, but it should shorten the nat table, too. And, as always, there are historical reasons…
@johnpoz:Sure you can setup firewall rules between a bridge and isolate them that way - but if you don't want them to see each other or only have a few exceptions - why make it a bridge in the first place?
There is a simple setting in pfSense when creating the bridge: private port.
@johnpoz:If you have 20 vlans, then setup 20 vlans and 20 network segments and 20 dhcp servers. That is the way you would do it if you ask me.. Or if you don't want to setup dhcp servers, then send them to a dhcp server that supports different scopes, etc. I don't think pfsense allows for serving up multiple scopes, you can have multiple pools but don't think it can serve up multiple segments off one server instance?
As long as it looks like I can make this work with quite short, and thus readable, config, I try to do it my way, then probably yours.
@johnpoz:Im with Derelict here, we all like shortcuts to min admin - but running a network takes work, if it didn't than any user could do it ;)
Me too :)
Risto
-
I was disappointed, when I understood, that my switch doesn't support that.
Get one that does? Like I said, Cisco 2950s are essentially free.
There is a simple setting in pfSense when creating the bridge: private port.
Yes, there is.
So you have created a bridge containing:
eth0_vlan10
eth0_vlan11
eth0_vlan12
…
eth0_vlan29All those interfaces are marked as "private" in the bridge config
You assigned your pfSense LAN interface to BRIDGE0
You created a single subnet on LAN and a single DHCP server on LAN
You have pass any any rules and good outbound NAT on LAN and for LAN's subnet
You have eth0 connected to a switch port with tagged VLANs 10-29
You have stations connected to untagged ports, one each, VLANs 10-29 (20 untagged ports)
And what exactly is not working?
What additional steps or config changes did you do?
I have never tried that private member setting in 2.2. I'll try it tonight.
-
"I was disappointed, when I understood, that my switch doesn't support that."
Get a better switch ;) Its not like you need a 250K nexus dual core setup do something as basic as private vlans. My <$200 cisco sg300 does it for sure.
-
Get one that does? Like I said, Cisco 2950s are essentially free.
I'll keep it in mind.
@Derelict:So you have created a bridge containing:
eth0_vlan10
eth0_vlan11
eth0_vlan12
…
eth0_vlan29All those interfaces are marked as "private" in the bridge config
You assigned your pfSense LAN interface to BRIDGE0
You created a single subnet on LAN and a single DHCP server on LAN
You have pass any any rules and good outbound NAT on LAN and for LAN's subnet
You have eth0 connected to a switch port with tagged VLANs 10-29
You have stations connected to untagged ports, one each, VLANs 10-29 (20 untagged ports)
And what exactly is not working?
UDP broadcasts from lan (LOCAL, see below) subnet are getting through out of WAN. (I am able to stop them by saying LOCAL to !LOCAL instead of any to any.)
@Derelict:What additional steps or config changes did you do?
Essentially everything is as you say. With two more things. BRIDGE0 is actually opt2 named LOCAL. LAN is left separate for switch admin access. And there is a second wan (3G, tier 2), wan group, consisting of these two, and it is given as Gateway in the any to any rule.
@Derelict:I have never tried that private member setting in 2.2. I'll try it tonight.
Thanks for your interest.
Risto
-
Again packets that are broadcast would not go out to another network. Unless they were bridged, or maybe IGMP proxy - did you have that setup?
No.
@johnpoz:Can you post a sniff of this traffic you were seeing going out the wan?
23:54:18.884766 00:0d:b9:17:cb:28 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 92: (tos 0x0, ttl 128, id 1685, offset 0, flags [none], proto UDP (17), length 78)
192.168.1.31.137 > 192.168.1.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
where 00:0d:b9:17:cb:28 is WAN (vr0) MAC,
LAN (bridge0) IP is 192.168.1.7,
WAN (vr0) IP is 80.x.x.x (DHCP).
@doktornotor:Which is totally unneeded. That one DHCP can serve two (or really any number of) different subnets just fine and the firewall will route packets between those just fine as well.
I agree. But my next goal would be to serve 20 VLANs.
Risto
You need to double check all the facts you assert in this post. Don't just gloss over this request and say "yeah, it's just like that" really go back and look again at everything.
What interface was that capture taken on?
Please provide a few more, captured on the WAN interface. Preferably some generic broadcasts like ARP, DHCP, etc. I don't have any windows CIFS hosts to test with - at least not readily.
-
And, real quick, be sure one of your LOCAL VLANs isn't mistakenly created on vr0 instead of your tagged LOCAL interface. It's easy to do.
