Or can i use squid for it?

"- Only the first packet is kept, which means that delayed packets are discarded. "

What exactly is going to remove the dupe packets?

That is exactly the part that I'm unsure about and will test. My logic behind it is that nobody would make the broadcast mode for no reason and the only reason I can see is to improve stability, but that will only happen if duplicate packages are discarded.

Reordering is an issue especially for TCP, however this is an issue of the internet in general caused by jitter which among other things are caused by multiple paths to the same destination. If jitter is reduced, then reordering should also be reduced.
Retrans is only happening if the packet is lost, or so delayed that TCP gives up waiting for it, thus this should also improve.

However, if broadcast mode is not discarding duplicates then some other mechanism has to be used to achieve this. OpenVPN comes to mind as it can use UDP in transport layer and discards duplicates out of the box.

What makes me daut the idea is what you say: "if this was a good way to help with voip traffic it would be recommended all over the internet as a way to deal with crappy connections. "
So, yes I'm slightly too humble to think that this will be the holy grail of solutions, but I will try any way :)

Maybe, what's stopped the "internet" from jumping on this solution is that it does require a server that we can control with a GOOD connection on the internet somewhere to use as the destination for the LAGG

Anyway, the idea is being tested by bonding 2 OpenVPN connections using Debian with bonding mode = 3 (broadcast). If bonding does not discard the duplicates then we will try with one more OpenVPN tunnel through the bonding interface. (as OpenVPN can discard dublicates)

My first goal is proof of concept… if it works, then a refined solution has to be worked out :)

No. One gateway per NIC, and one gateway per IP subnet. Doing what you describe is a really bad idea.

Sorry it was the "Weight" in system_gateways_edit.php but that should have only affected gateway groups so might just have been a co-incidence

"so this is what i can do"

How is that..  If you know the network is subpar, why not fix it the right way.  Just redo the setup..  What is the roadblock to correcting the flaws in the network?

You can get switches that support vlans on the lowest of lowest budgets..  What switches are you using now?

@mafiosa:

Can I use them together as multi WAN?

Not without an intermediate NAT device on one of them. A given subnet and IP can only exist on one interface.

I was looking into it and that should work fine, actually. It will take a bit of work but not a big deal.

Thanks again.

Hi,

Problem Solved.

Problem is at our ISP they have blocked port 25, they have been told that unfortunately some servers have been improperly shutdown (due to power problem) , from then onwards our emails are not working, when I raise a complaint they have opened port 25. Now its working. Our emails are going from WAN interface and for internet we are using OPT1 interface.

Thanks,
Ilesh

You should be able to translate the Cisco config into OpenBGPd without too much trouble. 'router bgp 11111' is your ASN, 'network 10.10.10.0' is your network,  'ebgp-multihop' x is multihop x, etc…

You don't show your firewall rules.  So while you have 2 networks if your rules are any only thing you would be blocking is broadcast traffic.

You really need to include the pfsense instructions or that little guide you put together is pretty useless.  And you need to be clear what port your connecting to pfsense and why your tagging it.

Configure what?  So you don't want it to firewall or nat, just route?  Then turn off firewall or just make any any rules, disable nat.  There you go just routing..

Issue resolved. I simply restored the PFsense to a saved configuration, then rebuilt my Cisco Router and it all came back.
Thanks for the help!

I finally figured out my problem after re-reading the Multi-Wan section of the Wiki.  Specifically this section

Policy Route Negation
When a firewall rule directs traffic into the gateway, it bypasses the routing table on the firewall. Policy route negation is just a rule that passes traffic to other local or VPN-connected networks that does not have a gateway set. By not setting a gateway on that rule it will bypass the gateway group and use the routing table on the firewall. These rules should be at the top of the list – or at least above any rules using gateways.

We had a rule in the LAN section to allow IPv4 traffic everywhere, but we had set the gateway to our WAN failover group bypassing the routing table.  We added another rule above that to use the default gateway and all is well.  Thanks for the help.

Typing to myself this far…

I’ve manage to do a work around with two static routes. As the issue seems to only be with resolving the hostname in OpenVPN Client, and I have two Domain overrides.
Why not just put them as separate static routes to each WAN?

Static routes (System > Routing > Static Routes)

OpenVPN_ns1 > WAN1
OpenVPN_ns2 > WAN2

This actually works, tunnel brings up on WAN2 and I can confirm traffic flow but after a couple of minutes when simulating member down (WAN1 unplugged)… Then the tunnel brakes with a flood of new message in the log.

OpenVPN log

write UDPv4: No buffer space available (code=55)

Getting same message in the console of pfSense trying to ping something.

[2.3.1-RELEASE][admin@-]/root: ping x.x.x.x PING x.x.x.x (x.x.x.x): 56 data bytes ping: sendto: No buffer space available ping: sendto: No buffer space available

Can someone explain why that is happening?
As soon I bring up WAN1 again everything is working normally.

Thanks
compfreak

Fixit it by myself. Reason was a wrong mtu value

IIRC that's a limitation of pf. It can't use anything other than round-robin or round-robin+sticky when specifying multiple addresses in that way. To use hashing it would have to use a network in that context, which doesn't make sense for gateways. If you want to see something like that, you'll have to advocate to pf directly (OpenBSD) or perhaps FreeBSD since the pf in FreeBSD has diverged from that of OpenBSD.

Thank you  ;)

@cmb:

That's the expected behavior. You need source NAT to access the system with backup status from a VPN via the system with master status.

Hi CMB,

Thank you for the clarification, I can understand why that might be the case, a bit unfortunate as the Source NAT feels like a bit of a hack but I'll try it out and continue with that :)

Thank you!

Edit: Just tested it and it works like a dream, anything to get rid of crappy static routes. Fantastic, thank you again!

Yes, I have traffic that matches Steam (UDP Destination Port 27000:27030 in this case) that goes out the default gateway. I can see these states under the Ficus interface in the Diagnostic > States viewer with matching destination ports.

I would like to see outputs of pfctl -vvsr and pfctl -vvss when the firewall is in this mode.

That will show exactly which rule is passing the traffic in question. It would be especially helpful if you could clear all states, generate the traffic in question, then take these samples.

I realize it might be kind of large.

There's probably a simple explanation for what you're seeing. Just don't know what it is yet.