UDP broadcasts to WAN



  • Hi,

    I'm not sure that this is a NAT problem, but it seems to have some connection to it, so..

    The problem is that UDP broadcasts are being forwarded from LAN to WAN. I've seen three different kinds:

    1. from local subnet to port 17500, these are NATted on the way
    2. from local subnet to port 137 and 138, these go as-is
    3. from unknown subnet to port 138, no NAT here either

    The system is Pcengines Alix and pfSense 2.2. LAN side has a bridge for LAN and WLAN. WAN side is multi-WAN with 3G as backup. But the UDP problem has been seen on the ethernet WAN. The bridge might have a connection to the problem.

    Risto



  • Ok, my mistake..

    The problem was caused by multi-WAN. I didn't realize that the Gateway setting in a firewall rule overrides the routing table. I'd see it more logical to take the local routes into account.

    The remaining question is: how did the kind 2 packets get through NAT as is?

    Risto


  • LAYER 8 Global Moderator

    "The bridge might have a connection to the problem."

    Yeah think ;)  Broadcast traffic is not sent out to other networks no matter what routing you have in place.



  • Well, that's what I thought, but I was obviously wrong.

    The bridge is there to allow one DHCP to serve both LAN interfaces, WAN is not part of it. And the strange packets look like routed: the source MAC is that of the WAN.

    Risto


  • LAYER 8 Global Moderator

    Again packets that are broadcast would not go out to another network.  Unless they were bridged, or maybe IGMP proxy - did you have that setup?

    Can you post a sniff of this traffic you were seeing going out the wan?


  • Banned

    @ristosu:

    The bridge is there to allow one DHCP to serve both LAN interfaces

    Which is totally unneeded. That one DHCP can serve two (or really any number of) different subnets just fine and the firewall will route packets between those just fine as well.



  • @johnpoz:

    Again packets that are broadcast would not go out to another network.  Unless they were bridged, or maybe IGMP proxy - did you have that setup?

    No.
    @johnpoz:

    Can you post a sniff of this traffic you were seeing going out the wan?

    23:54:18.884766 00:0d:b9:17:cb:28 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 92: (tos 0x0, ttl 128, id 1685, offset 0, flags [none], proto UDP (17), length 78)
        192.168.1.31.137 > 192.168.1.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
    where 00:0d:b9:17:cb:28 is WAN (vr0) MAC,
    LAN (bridge0) IP is 192.168.1.7,
    WAN (vr0) IP is 80.x.x.x (DHCP).
    @doktornotor:

    Which is totally unneeded. That one DHCP can serve two (or really any number of) different subnets just fine and the firewall will route packets between those just fine as well.

    I agree. But my next goal would be to serve 20 VLANs.

    Risto


  • LAYER 8 Global Moderator

    So your going to bridge 20 vlans together??  What does that have to do with bridging interfaces for dhcp?

    You must have something F'd up in your bridge if your sending out broadcast out your wan..  Or you installed something on pfsense to send it out, or is this a VM running on virtualbox and you didn't remove the windows bindings from the interface and that is windows sending that out.  Is pfsense vm??  OR is this hardware?  Ah from mac looks like PC Engines..

    Again lets say this one more time so you understand it BROADCAST traffic is NOT routed.. Pfsense would not get a FF:FF packet on its lan and say oh lets send this out my wan.  It wouldn't happen!  Unless you have some sort of bridge, or some kind of broadcast proxy, etc.

    Did you install say samba on pfsense?  So sniff on your lan at same time your sniffing wan, where is the broadcast coming from if you say pfsense is forwarding it.



  • @johnpoz:

    So your going to bridge 20 vlans together??  What does that have to do with bridging interfaces for dhcp?

    Yes. I want to avoid configuring 20 subnets. Perhaps I could try one subnet for the vlans and one for the wlan.
    @johnpoz:

    You must have something F'd up in your bridge if your sending out broadcast out your wan..  Or you installed something on pfsense to send it out, or is this a VM running on virtualbox and you didn't remove the windows bindings from the interface and that is windows sending that out.  Is pfsense vm??  OR is this hardware?  Ah from mac looks like PC Engines..

    Yes, it definitely seems to be caused by the bridge: I took it away, and the symptoms ceased. It is a PC Engines Alix 6b2.
    @johnpoz:

    Again lets say this one more time so you understand it BROADCAST traffic is NOT routed.. Pfsense would not get a FF:FF packet on its lan and say oh lets send this out my wan.  It wouldn't happen!  Unless you have some sort of bridge, or some kind of broadcast proxy, etc.

    Unless there's a bug in it. Well, I don't have any such daemons, just the bridge. But bridged traffic should have the original sender's MAC address. I've changed these two system tunables from their defaults: net.link.bridge.pfil_member=0, net.link.bridge.pfil_bridge=1. And I've defined all bridge ports as edge and private: I don't want them to communicate to each other.
    @johnpoz:

    Did you install say samba on pfsense?  So sniff on your lan at same time your sniffing wan, where is the broadcast coming from if you say pfsense is forwarding it.

    No Samba. I've seen it coming from a windows box on the lan.

    Thanks for your comments so far.

    Risto


  • Banned

    @ristosu:

    @doktornotor:

    Which is totally unneeded. That one DHCP can serve two (or really any number of) different subnets just fine and the firewall will route packets between those just fine as well.

    I agree. But my next goal would be to serve 20 VLANs.

    So, you are going to bridge 20 VLANs? Why bother in the first place? Just get a dumb switch and put all on a single LAN. WTF. This must be some disease with the bridging. Lets bridge VLANs, lets bridge WLAN with wired LAN, lets bridge 10 ports to waste hardware trying to turn a router into a dumb switch.

    ??? :o



  • @doktornotor:

    So, you are going to bridge 20 VLANs? Why bother in the first place? Just get a dumb switch and put all on a single LAN. WTF. This must be some disease with the bridging. Lets bridge VLANs, lets bridge WLAN with wired LAN, lets bridge 10 ports to waste hardware trying to turn a router into a dumb switch.

    ??? :o

    There is a reason for everything. In this case I want to separate the vlans (private bridge ports) so they don't see each other. There is no excuse for pfsense's bridge not working as it should.

    Risto


  • LAYER 8 Netgate

    Yes. I want to avoid configuring 20 subnets.

    Another one who thinks administering a network isn't going to be work.  It boggles the mind.


  • LAYER 8 Netgate

    There is a reason for everything. In this case I want to separate the vlans (private bridge ports) so they don't see each other. There is no excuse for pfsense's bridge not working as it should.

    What you're doing is nonsensical.  I don't think you quite understand what a "bridge" does.

    There are other, better ways to do what you're trying to do.  One being a separate VLAN interface for each VLAN like any sane person would do.  Another would be private VLAN edge (protected) ports on a switch with everyone on the same VLAN.  Cisco 2950s do this 10/100 and are essentially free.  Most "cheap" web-managed switches (trendnet, d-link, TP-link, etc) can fake this with "asymmetric VLANs".


  • LAYER 8 Global Moderator

    "I want to separate the vlans (private bridge ports) so they don't see each other"

    Huh - so you don't want clients seeing each other but you want to connect the vlans together with a bridge?  Lets taking bridging to its simplest level.  Packets come in one interface, they get sent out all bridge members interfaces.  So if you want devices on different vlans not to see each other - why would you bridge them?

    Sure you can setup firewall rules between a bridge and isolate them that way - but if you don't want them to see each other or only have a few exceptions - why make it a bridge in the first place?

    If you have 20 vlans, then setup 20 vlans and 20 network segments and 20 dhcp servers.  That is the way you would do it if you ask me.. Or if you don't want to setup dhcp servers, then send them to a dhcp server that supports different scopes, etc.  I don't think pfsense allows for serving up multiple scopes, you can have multiple pools but don't think it can serve up multiple segments off one server instance?

    Im with Derelict here, we all like shortcuts to min admin - but running a network takes work, if it didn't than any user could do it ;)



  • @Derelict:

    What you're doing is nonsensical.  I don't think you quite understand what a "bridge" does.

    I think I do. I'm only trying to use bridge config here as a means to achieve my goal, which I've hopefully explained already. As the bridge ports are all private, it's no usual bridge.
    @Derelict:

    There are other, better ways to do what you're trying to do.  One being a separate VLAN interface for each VLAN like any sane person would do.  Another would be private VLAN edge (protected) ports on a switch with everyone on the same VLAN.  Cisco 2950s do this 10/100 and are essentially free.  Most "cheap" web-managed switches (trendnet, d-link, TP-link, etc) can fake this with "asymmetric VLANs".

    There are always different ways of doing things. I was disappointed, when I understood, that my switch doesn't support that.
    @johnpoz:

    "I want to separate the vlans (private bridge ports) so they don't see each other"

    Huh - so you don't want clients seeing each other but you want to connect the vlans together with a bridge?  Lets taking bridging to its simplest level.  Packets come in one interface, they get sent out all bridge members interfaces.  So if you want devices on different vlans not to see each other - why would you bridge them?

    The bridge in mostly for dhcp, but it should shorten the nat table, too. And, as always, there are historical reasons…
    @johnpoz:

    Sure you can setup firewall rules between a bridge and isolate them that way - but if you don't want them to see each other or only have a few exceptions - why make it a bridge in the first place?

    There is a simple setting in pfSense when creating the bridge: private port.
    @johnpoz:

    If you have 20 vlans, then setup 20 vlans and 20 network segments and 20 dhcp servers.  That is the way you would do it if you ask me.. Or if you don't want to setup dhcp servers, then send them to a dhcp server that supports different scopes, etc.  I don't think pfsense allows for serving up multiple scopes, you can have multiple pools but don't think it can serve up multiple segments off one server instance?

    As long as it looks like I can make this work with quite short, and thus readable, config, I try to do it my way, then probably yours.
    @johnpoz:

    Im with Derelict here, we all like shortcuts to min admin - but running a network takes work, if it didn't than any user could do it ;)

    Me too :)

    Risto


  • LAYER 8 Netgate

    I was disappointed, when I understood, that my switch doesn't support that.

    Get one that does?  Like I said, Cisco 2950s are essentially free.

    There is a simple setting in pfSense when creating the bridge: private port.

    Yes, there is.

    So you have created a bridge containing:
    eth0_vlan10
    eth0_vlan11
    eth0_vlan12

    eth0_vlan29

    All those interfaces are marked as "private" in the bridge config

    You assigned your pfSense LAN interface to BRIDGE0

    You created a single subnet on LAN and a single DHCP server on LAN

    You have pass any any rules and good outbound NAT on LAN and for LAN's subnet

    You have eth0 connected to a switch port with tagged VLANs 10-29

    You have stations connected to untagged ports, one each, VLANs 10-29 (20 untagged ports)

    And what exactly is not working?

    What additional steps or config changes did you do?

    I have never tried that private member setting in 2.2.  I'll try it tonight.


  • LAYER 8 Global Moderator

    "I was disappointed, when I understood, that my switch doesn't support that."

    Get a better switch ;)  Its not like you need a 250K nexus dual core setup do something as basic as private vlans.  My <$200 cisco sg300 does it for sure.



  • @Derelict:

    Get one that does?  Like I said, Cisco 2950s are essentially free.

    I'll keep it in mind.
    @Derelict:

    So you have created a bridge containing:
    eth0_vlan10
    eth0_vlan11
    eth0_vlan12

    eth0_vlan29

    All those interfaces are marked as "private" in the bridge config

    You assigned your pfSense LAN interface to BRIDGE0

    You created a single subnet on LAN and a single DHCP server on LAN

    You have pass any any rules and good outbound NAT on LAN and for LAN's subnet

    You have eth0 connected to a switch port with tagged VLANs 10-29

    You have stations connected to untagged ports, one each, VLANs 10-29 (20 untagged ports)

    And what exactly is not working?

    UDP broadcasts from lan (LOCAL, see below) subnet are getting through out of WAN. (I am able to stop them by saying LOCAL to !LOCAL instead of any to any.)
    @Derelict:

    What additional steps or config changes did you do?

    Essentially everything is as you say. With two more things. BRIDGE0 is actually opt2 named LOCAL. LAN is left separate for switch admin access. And there is a second wan (3G, tier 2), wan group, consisting of these two, and it is given as Gateway in the any to any rule.
    @Derelict:

    I have never tried that private member setting in 2.2.  I'll try it tonight.

    Thanks for your interest.

    Risto


  • LAYER 8 Netgate

    @ristosu:

    @johnpoz:

    Again packets that are broadcast would not go out to another network.  Unless they were bridged, or maybe IGMP proxy - did you have that setup?

    No.
    @johnpoz:

    Can you post a sniff of this traffic you were seeing going out the wan?

    23:54:18.884766 00:0d:b9:17:cb:28 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 92: (tos 0x0, ttl 128, id 1685, offset 0, flags [none], proto UDP (17), length 78)
        192.168.1.31.137 > 192.168.1.255.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
    where 00:0d:b9:17:cb:28 is WAN (vr0) MAC,
    LAN (bridge0) IP is 192.168.1.7,
    WAN (vr0) IP is 80.x.x.x (DHCP).
    @doktornotor:

    Which is totally unneeded. That one DHCP can serve two (or really any number of) different subnets just fine and the firewall will route packets between those just fine as well.

    I agree. But my next goal would be to serve 20 VLANs.

    Risto

    You need to double check all the facts you assert in this post.  Don't just gloss over this request and say "yeah, it's just like that" really go back and look again at everything.

    What interface was that capture taken on?

    Please provide a few more, captured on the WAN interface.  Preferably some generic broadcasts like ARP, DHCP, etc.  I don't have any windows CIFS hosts to test with - at least not readily.


  • LAYER 8 Netgate

    And, real quick, be sure one of your LOCAL VLANs isn't mistakenly created on vr0 instead of your tagged LOCAL interface.  It's easy to do.



  • @Derelict:

    And, real quick, be sure one of your LOCAL VLANs isn't mistakenly created on vr0 instead of your tagged LOCAL interface.  It's easy to do.

    [2.2-RELEASE][admin@pfSense.localdomain]/root: ifconfig | grep vr0
    vr0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            inet6 fe80::20d:b9ff:fe17:cb28%vr0 prefixlen 64 scopeid 0x1 
    [2.2-RELEASE][admin@pfSense.localdomain]/root:</up,broadcast,running,simplex,multicast> 
    

    No, it isn't. Good point, though.

    Risto



  • @Derelict:

    What interface was that capture taken on?

    Actually on another host on wan side. But the sender's MAC is pfSense's WAN.
    @Derelict:

    Please provide a few more, captured on the WAN interface.  Preferably some generic broadcasts like ARP, DHCP, etc.  I don't have any windows CIFS hosts to test with - at least not readily.

    I'll try tomorrow.

    Risto


  • LAYER 8 Netgate

    Well, I just built this with three VLANs.  PRIVATE seems to work:

    bridge0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            ether 02:ba:93:e5:a4:00
            inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
            inet6 fe80::1:1%bridge0 prefixlen 64 scopeid 0xb
            nd6 options=1 <performnud>id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
            maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
            root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
            member: re2_vlan30 flags=943<learning,discover,<strong>PRIVATE,AUTOEDGE,AUTOPTP>
                    ifmaxaddr 0 port 10 priority 128 path cost 55
            member: re2_vlan20 flags=943<learning,discover,<strong>PRIVATE,AUTOEDGE,AUTOPTP>
                    ifmaxaddr 0 port 9 priority 128 path cost 55
            member: re2_vlan10 flags=943<learning,discover,<strong>PRIVATE,AUTOEDGE,AUTOPTP>
                    ifmaxaddr 0 port 8 priority 128 path cost 55

    interface FastEthernet0/4
    switchport access vlan 10
    switchport mode access
    !
    interface FastEthernet0/5
    switchport access vlan 20
    switchport mode access
    !
    interface FastEthernet0/6
    switchport access vlan 30
    switchport mode access
    !
    interface FastEthernet0/7
    switchport trunk allowed vlan 10,20,30
    switchport mode trunk
    !</learning,discover,<strong></learning,discover,<strong></learning,discover,<strong></performnud></up,broadcast,running,simplex,multicast>

    Host on fa0/4 gets DHCP and can surf but is isolated from host on fa0/5, that also gets DHCP in the same subnet. Not so much as ARP.  New tool for the toolbox.  Might be useful in a pinch if I'm ever on a desert island without newegg and need pvlanedge.

    But I am not seeing any broadcasts on the BRIDGE0 network leaking out WAN, which will come as a surprise to absolutely nobody except, perhaps, you. ;)

    If I were you, I would just forget that you've found some defect in pfSense and, instead, look at what you've done in your configuration or testing process to see what you think you're seeing.

    If you can tell me exactly how to duplicate it, I'd be happy to try.  With some effort I can get a windows VM into this test environment.

    Even on my lowly Mac I was able to generate some netbios name lookups.  The only thing on WAN when I do the same thing is apinger.

    ![Screen Shot 2015-03-03 at 8.21.25 PM.png](/public/imported_attachments/1/Screen Shot 2015-03-03 at 8.21.25 PM.png)
    ![Screen Shot 2015-03-03 at 8.21.25 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2015-03-03 at 8.21.25 PM.png_thumb)



  • Now the problem seems to be policy based routing. Maybe in combination with the system tunable settings I mentioned before (but forgot to mention yesterday, sorry): net.link.bridge.pfil_member=0, net.link.bridge.pfil_bridge=1.

    You would need a second gateway, create a gateway group, put the gateways on different tiers (the lower number will receive the traffic), and select the group as gateway in the firewall rule, instead of default.

    The packet in your attachment should qualify.

    Risto


  • LAYER 8 Netgate

    @ristosu:

    Now the problem seems to be policy based routing. Maybe in combination with the system tunable settings I mentioned before (but forgot to mention yesterday, sorry): net.link.bridge.pfil_member=0, net.link.bridge.pfil_bridge=1.

    I set mine the same for the previous tests.

    You would need a second gateway, create a gateway group, put the gateways on different tiers (the lower number will receive the traffic), and select the group as gateway in the firewall rule, instead of default.

    The packet in your attachment should qualify.

    Not sure that I'm willing to set that up since I have no reason to believe the results will be any different.  You need to take a GOOD look at what you've done in your environment.  What you're describing is basically impossible.  You screwed something up somewhere.  Probably at layer 2.



  • I'm trying to say that this one config change, from default to gateway group, changes the behaviour.

    You could simply use any, even imaginary, host on wan side as your second gateway.

    ARP or DHCP are not leaking.

    Risto


  • LAYER 8 Global Moderator

    How would "routing" anything have to do with it.. What part do you just not understand that BROADCAST traffic is NOT routed..  what you posted is not even a directed broadcast - its full FF.. Why would pfsense send that anywhere, not going to forward it, not going to route it..


  • Banned

    @ristosu:

    a reason for everything. In this case I want to separate the vlans (private bridge ports) so they don't see each other. There is no excuse for pfsense's bridge not working as it should.

    Dude, you are totally lost. When you want separate private VLANs, then for goddamn sake do NOT bridge them. Plus, the DHCP is the most BS reason to create a bridge, ever. All of this - overengineered, error prone nonsense with multiple additional layers of complexity that may (and clearly do) cause issues - just because you are lazy to get up DHCP server properly.



  • @johnpoz:

    How would "routing" anything have to do with it.. What part do you just not understand that BROADCAST traffic is NOT routed..  what you posted is not even a directed broadcast - its full FF.. Why would pfsense send that anywhere, not going to forward it, not going to route it..

    How should a directed broadcast look like?

    Risto



  • @doktornotor:

    Dude, you are totally lost. When you want separate private VLANs, then for goddamn sake do NOT bridge them. Plus, the DHCP is the most BS reason to create a bridge, ever. All of this - overengineered, error prone nonsense with multiple additional layers of complexity that may (and clearly do) cause issues - just because you are lazy to get up DHCP server properly.

    Thanks for telling me, but actually we've gone through all this before during this thread, so I don't care to explain it anymore, unless you insist. And it's working with a simple firewall setting. I'm more worried about pfsense and possibly the underlying freebsd.

    Risto


  • LAYER 8 Global Moderator

    So I looked at what you posted again..

    192.168.1.31.137 > 192.168.1.255.137

    That is a directed broadcast…. And what IP address is 1.31?  Some on your lan side.. In what world would that ever be routed anywhere??  The only way that would go out some interface that was not in that network is if there was a bridge!

    Or you have a mask wrong somewhere where that .255 would be a host IP.. like 192.168.1.0/23  But if pfsense was going to route that as a host address, why would it not be natted if going on your wan?  What network is your wan on?

    What are the networks on your pfsense with masks?  What network is the wan in?



  • @johnpoz:

    So I looked at what you posted again..

    192.168.1.31.137 > 192.168.1.255.137

    That is a directed broadcast…. And what IP address is 1.31?  Some on your lan side.. In what world would that ever be routed anywhere??  The only way that would go out some interface that was not in that network is if there was a bridge!

    Yes, 1.31 is a windows box on the lan side. Looks to me that somehow this policy based routing overrides the routing table and ignores the local routes. I think it's wrong.
    @johnpoz:

    Or you have a mask wrong somewhere where that .255 would be a host IP.. like 192.168.1.0/23  But if pfsense was going to route that as a host address, why would it not be natted if going on your wan?  What network is your wan on?

    I use /24 masks for simplicity. The wans are the only exceptions (ethernet dhcp and ppp).
    @johnpoz:

    What are the networks on your pfsense with masks?  What network is the wan in?

    Wan in vr0. Here ifconfig of all interfaces with ip:

    vr0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=8280b <rxcsum,txcsum,vlan_mtu,wol_ucast,wol_magic,linkstate>ether 00:0d:b9:17:cb:28
            inet6 fe80::20d:b9ff:fe17:cb28%vr0 prefixlen 64 scopeid 0x1 
            inet 80.220.71.201 netmask 0xffffe000 broadcast 80.220.95.255 
            nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (100baseTX <full-duplex>)
            status: active
    vr1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
            options=8280b <rxcsum,txcsum,vlan_mtu,wol_ucast,wol_magic,linkstate>ether 00:0d:b9:17:cb:29
            inet6 fe80::20d:b9ff:fe17:cb29%vr1 prefixlen 64 scopeid 0x2 
            inet 192.168.2.7 netmask 0xffffff00 broadcast 192.168.2.255 
            inet 192.168.0.7 netmask 0xffffff00 broadcast 192.168.0.255 
            nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (100baseTX <full-duplex>)
            status: active
    lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
            options=600003 <rxcsum,txcsum,rxcsum_ipv6,txcsum_ipv6>inet 127.0.0.1 netmask 0xff000000 
            inet6 ::1 prefixlen 128 
            inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 
            nd6 options=21 <performnud,auto_linklocal>ural0_wlan0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            ether 00:17:31:c7:8f:6d
            inet6 fe80::217:31ff:fec7:8f6d%ural0_wlan0 prefixlen 64 scopeid 0x8 
            inet 192.168.3.7 netmask 0xffffff00 broadcast 192.168.3.255 
            nd6 options=21 <performnud,auto_linklocal>media: IEEE 802.11 Wireless Ethernet autoselect mode 11g <hostap>
            status: running
            ssid pfSense2 channel 1 (2412 MHz 11g) bssid 00:17:31:c7:8f:6d
            regdomain ETSI country FI authmode WPA privacy MIXED deftxkey 2
            TKIP 2:128-bit TKIP 3:128-bit txpower 30 scanvalid 60 protmode OFF
            dtimperiod 1 -dfs
    ppp1: flags=88d1 <up,pointopoint,running,noarp,simplex,multicast>metric 0 mtu 1492
            inet6 fe80::20d:b9ff:fe17:cb28%ppp1 prefixlen 64 scopeid 0x1e 
            inet 10.233.110.117 --> 10.64.64.1 netmask 0xffffffff 
            nd6 options=21 <performnud,auto_linklocal>bridge0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            ether 02:8f:df:55:b9:00
            inet 192.168.1.7 netmask 0xffffff00 broadcast 192.168.1.255 
            nd6 options=1 <performnud>id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
            maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
            root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
            member: vr1_vlan120 flags=b63 <learning,discover,private,edge,autoedge,autoptp>ifmaxaddr 0 port 28 priority 128 path cost 200000
    ...
            member: vr1_vlan101 flags=b63 <learning,discover,private,edge,autoedge,autoptp>ifmaxaddr 0 port 9 priority 128 path cost 200000</learning,discover,private,edge,autoedge,autoptp></learning,discover,private,edge,autoedge,autoptp></performnud></up,broadcast,running,simplex,multicast></performnud,auto_linklocal></up,pointopoint,running,noarp,simplex,multicast></hostap></performnud,auto_linklocal></up,broadcast,running,simplex,multicast></performnud,auto_linklocal></rxcsum,txcsum,rxcsum_ipv6,txcsum_ipv6></up,loopback,running,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,wol_ucast,wol_magic,linkstate></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,wol_ucast,wol_magic,linkstate></up,broadcast,running,simplex,multicast>
    

    Risto


  • LAYER 8 Netgate

    And the switch config?  And a physical diagram of how you have it all connected?


  • LAYER 8 Global Moderator

    "Looks to me that somehow this policy based routing overrides the routing table"

    That might be something if it wasn't broadcast traffic you don't route broadcast traffic.. So unless you have a mask where that looks like a host IP and not a broadcast address it would not be routed.  No matter if policy based or not.



  • @Derelict:

    And the switch config?  And a physical diagram of how you have it all connected?

    I haven't succeeded in getting a shell interface to the switch (Dell PowerConnect 2724), so I just have to describe it here. (The web-management has been problematic, too.)

    • ports 1 to 20 are untagged with vlans 101 to 120 respectively, these go to apartments
    • ports 21 to 23 are without vlans
    • port 24 is tagged with vlans 101 to 120, this is connected to pfsense's lan (vr1)
    • pfsense's lan has vlans 101 to 120 that comprise bridge0
    • pfsense's lan has also two ip addresses for raw access towards the switch
    • pfsense's wan (vr0) is connected to operator's line through a switch (another one)
    • pfsense's wlan (ural0_wlan0) has an ip address and is used as an alternative access to lan side
    • pfsense's 3G stick (ppp1) is used as backup wan

    Risto


  • LAYER 8 Netgate

    pfsense's wan (vr0) is connected to operator's line through a switch (another one)

    What else is connected to that switch?


  • LAYER 8 Netgate

    I shouldn't have to create a gateway group to test this.  All I'll have to do is policy route to the existing gateway instead of the default routing table.

    Even though I know that won't satisfy you so I'll make a group anyway.  Not sure how that will satisfy you, either.



  • @Derelict:

    What else is connected to that switch?

    Two more routers.
    @Derelict:

    I shouldn't have to create a gateway group to test this.  All I'll have to do is policy route to the existing gateway instead of the default routing table.

    Even though I know that won't satisfy you so I'll make a group anyway.  Not sure how that will satisfy you, either.

    You are probably right. One route is enough. I was thinking too complicated. Sorry for that.

    Risto



  • I made a simplified setup with a virtual host in qemu. Two interfaces of type "em" (intel gigabit). Lan ip 192.168.2.1, bridge ip 192.168.0.1 (vlans 101, 102), wan dhcp. I was able to demonstrate the problem by sending a udp packet with nmap. I'll attach the config. It is so simple that it should be easy to spot the error.

    config-virtual.localdomain-20150304230122.xml.txt


  • LAYER 8 Netgate

    How do you guys like your crow?  I'll take mine with sriracha and a nice zinfandel.


Log in to reply