2.3.2 uses dpinger, not apinger.

What does System > Logs, Gateways have to say?

Sounds like you need to check "Bypass firewall rules for traffic on the same interface" under System>Adv, Firewall/NAT.

@heper:

are you able to ping from those interfaces towards the web ? (you can use diagnostics–>ping to select them individual interfaces)

No, im not able to ping anything else than i wrote above unfortunately.
But i can ping 172.16.0.1, and that should not be possible. But i can reject access to RFC 1918 networks on the interfaces. Then that problem is solved.

I thank you for trying to help, but you're missing the point of the topic and I'm not sure theres anything else you can add to be helpful in this instance.  There's nothing 'mangled' about it.  It's simply assigning a VLANs to members of a Bridged interface as required.  I know how to fix it with a managed switch, that is not the issue.

My point is that VLAN's on Bridged interfaces don't appear to work as intended, or, I have something wrong in the configuration.  If I have something wrong on the configuration side, I'd like to troubleshoot to know what it is.  If there is some bug that affects VLAN's on Bridge interfaces, then I'd like to hear others input and see if they can replicate the issue so a bug report can be submitted.

Hi,

Thx for the reply but I did not understand exactly what you mean. Should i remove any of my rules then add another rule to every sub-net? I dont want anything leaking, all internet connection outside the local networks should go trough the VPN. Can you please explain so I can understand it? I am new to all this.

@heper:

you can change the default gateway at system>>routing

I change default gateway , but pfsense choose always DHCP WAN

Hello,

so I enabled the Forwarding mode and Default gateway switching, but neither worked. The Forwarding mode steered me towards DNS Forwarder service instead of DNS Resolver service (not quite sure what's the difference). Using the DNS Forwarder with Default gateway switching disabled seems to be working.

Thanks for Your help!

Thanks I will definitely take a look.

@heper:

https://www.reddit.com/r/Chromecast/comments/454fsi/chromecast_across_subnetsvlans_pfsense/

Awesome!  This worked perfectly thanks!

Take a look at:

https://doc.pfsense.org/index.php/Multi-WAN

https://doc.pfsense.org/index.php/Gateway_Settings

@johnpoz:

If traffic comes in wan1 it going to go back out wan 1 for the answer.

I do not know why not work, on both WAN interfaces is configured the corresponding gateway(L3). If that were working well for us, we would be very happy… maybe is something wrong configured but I can´t find it.

@johnpoz:

If your using GLBP and connection from l3-1 to pfsense fails, then no traffic would come in that interface so why would pfsense send a respond out an interface that did not see the traffic?

Each L3 can ping the correspondig WAN interface in pfsense and also can ping a server inside server networks.

@johnpoz:

Why do you need to set any routes?? is not l3-1 and l3-2 different gateways?  Why are you needing to create routes to this user network at all?

Yes, L3-1 and L3-2 are different gateways. I used Packet Capture in WAN1 interface when a host in User Network ping to a server in Server Network and I see the  "ICMP echo request" packets, but don´t see the corresponding "ICMP echo reply". The only way we've found to make work it is to set a static route to reach network user through one of the L3 (but is not a real solution for us because for access to networks Servers would not have fail-over L3s)

@johnpoz:

Only place you can go is out l3-1 or l3-2.. Is there some part of this network that you did not show that requires you to create routes that are out your wan interfaces?  Where you can only go to either l3-1 or l3-2, those devices might have to make a routing decision on how to send the traffic to the user network, but why would pfsense have to?

All traffic to reach networks that are not directly connected to pfsense (ie. other networks than WAN1 [L3-1], WAN2 [L3-2], LAN [network server]) we want pfsense send it to "any" of the L3s (as you say early:  "traffic comes in wan1 it going to go back out wan 1 for the answer", That would be fine!!). Both L3s know how to route traffic to the User networks or other networks.

Thank you very much for your time. We want to use pfsense and will make every effort to try to configure it properly to our needs.

PD: Packet Capture in LAN interface when a host in User Network ping to a server in Server Network show the "ICMP echo request" packets and corresponding "ICMP echo reply" packets.

It's not that the filter reload is causing a failover, the filter reload occurs because you're having a WAN failure. Likely that you are having loss, but why on two diff gateways at the same time isn't clear from that limited context. Check your gateway logs.

There has to be some way to match the traffic will firewall rules to identify traffic to put on specific WANs.

If your gaming is all done from a console, put the console IP address(es) in an alias, then match those and send them out your "gaming" WAN.

If you game and browse on the same PC, that's much different and a more difficult problem to solve. You might be able to just send TCP ports 80 and 443 out the "browsing" WAN but undoubtedly there will be other non-gaming traffic on other ports (e-mail, FTP, torrents, etc).

@heper:

so:

-all vlan_clients have access to the internet & can access the pfsense webgui

That's correct.

-you have 'allow all rules' on all vlan_interfaces (with proto=any)?

Yes, just like on the default LAN interface. Allow any type of traffic from abc net to any destination. I will eventually build rest of my rules on top of these.

are you sure the clients are accepting connections from each-other? have you tried to turn off windoze firewall ?

Yes I can reach the clients from pfSense but not from a different subnet/vlan. Most of my traffic is ssh/slp anyway so beloved Windows FW doesn't play a role here.

Or can i use squid for it?

"- Only the first packet is kept, which means that delayed packets are discarded. "

What exactly is going to remove the dupe packets?

That is exactly the part that I'm unsure about and will test. My logic behind it is that nobody would make the broadcast mode for no reason and the only reason I can see is to improve stability, but that will only happen if duplicate packages are discarded.

Reordering is an issue especially for TCP, however this is an issue of the internet in general caused by jitter which among other things are caused by multiple paths to the same destination. If jitter is reduced, then reordering should also be reduced.
Retrans is only happening if the packet is lost, or so delayed that TCP gives up waiting for it, thus this should also improve.

However, if broadcast mode is not discarding duplicates then some other mechanism has to be used to achieve this. OpenVPN comes to mind as it can use UDP in transport layer and discards duplicates out of the box.

What makes me daut the idea is what you say: "if this was a good way to help with voip traffic it would be recommended all over the internet as a way to deal with crappy connections. "
So, yes I'm slightly too humble to think that this will be the holy grail of solutions, but I will try any way :)

Maybe, what's stopped the "internet" from jumping on this solution is that it does require a server that we can control with a GOOD connection on the internet somewhere to use as the destination for the LAGG

Anyway, the idea is being tested by bonding 2 OpenVPN connections using Debian with bonding mode = 3 (broadcast). If bonding does not discard the duplicates then we will try with one more OpenVPN tunnel through the bonding interface. (as OpenVPN can discard dublicates)

My first goal is proof of concept… if it works, then a refined solution has to be worked out :)