I'm afraid I won't be of too much use here, but it did occur to me to ask whether you really mean 50MB (as in 50 megabytes per second, or 400 megabits per second)? I'm assuming that's the case, because 50 megabits per second certainly shouldn't be stressing anything. Additionally, is the connection symmetric (50 upstream too)?

Also I don't know whether this is really an option, but if possible I might suggest backing up your configuration and performing a fresh installation. Then you can see whether the problem exists even when you're starting from zero with no packages installed.

IPsec is ... faster.

Hello,

thanks to fill my great ignorance; with your help I resolved the issue.
Now I have this in the rule for LAN interface:

0_1535450259748_pfsense_rule_1.png

and on the outbound NAT I set the correct interface:

0_1535450301470_pfsense_rule_2.png

But now please you can explain something about that?
The first thing is how I can go out via the 88.45.191.140 path even if I am on the WAN interface; or better, when I do traceroute I see that correctly I go out through the "desired" path and not that it is of default.
The second question if about the starting path, i.e.: with the configuration that I have done initially I've seen that the flow is:

192.168.0.3 (swi1) 192.168.0.31 (pfs1) network desired hop

while now with the correct gateway setup on lan->net 7 rule I see only

192.168.7.7 (swi1 address hsrp for net 7)
*network desired hop

so it seems that the pfsense is not engaged.

Thanks.

Using failover, you do have to call out the group in the firewall rule. So yes if you want to allow traffic to your local vlans and not go out the specific gateway this is how its done.

Its gone over here
https://www.netgate.com/docs/pfsense/routing/multi-wan.html

And in the book with more detail - everyone now has access to the book... I would suggest you take a look ;)
https://www.netgate.com/docs/pfsense/book/multiwan/index.html

According to my testing sticky connection is not working as you described. When opening several connections from one machine both wan gateways are being used. And there are persistent connections still active AND all connections are established within sticky connection timeout. If it is supposed to work client based it is not doing that in practice. And that causes issues when a single software opens multiple connections and those are routed through different wan gateways.
One test I made was pretty clear: opening www.whatismyipaddress.com in two browsers -> different wans.

A.

Hello @tejas

you NAT outbound rules?

"OpenVPN-GW" is handled as a gateway group including all OpenVPN instances (servers and clients) on pfSesne. So if you running multiple OpenVPN instances on L assign an interface to the concerned one and use the gateway of it for policy routing. On pfSense R add an outbound NAT rule to the WAN interface for the source network opt1, translating source addresses to the WAN address.

So your internet connection is what exactly 10ge? Multiple gig over a 10ge interface.. Multiple smartjacks... How exactly is this isp connection with multiiple IPs presented to you? Is it a 802.3bz into a switch and you want to run multiple gig interfaces into the same switch on the same L2 to be able to leverage the higher than gig connection?

Unless your bandwidth is higher than what your interface can handle at the physical layer - there is zero reason not to use just a vip or a vlan, etc.

I have 100mbps internet with /24 for ips - why would I need multiple physical interfaces to use all those IPs if I have gig interface?

What does UBNT mean when they say "VLAN aware" mode.

Tag the VLANs on a port on the XG-7100 switch.

Tag the VLANs on a port on the UBNT switch.

Cross-connect them.

@bnelsonjax i'm kind of new to pfsense but you setup a gateway group for your failover right? wan1 tier1 wan2 tier2 then you made a firewall rule for all traffic to that gateway group right?

If so you just make another gateway group but this one is wan1 tier2 wan2 tier2 then when you make the firewall rule you specify the source as the voip device and assign it to the new gateway group you added.

Edit: oh and you put the firewall rule above the old one of course on the list.

To answer my own question, I had to create a Virtual IP (IP Alias) with the single static IP address that the DNS record points to. Then, under VPN -> IPsec -> Mobile Clients -> Edit Phase 1, under 'Interface' the Virtual IP created is given as an option.

I also changed the way the pfSense firewall/router obtains its IP address. The WAN interface now has a static private IP address (192.168.2.1) which is seen by my ISP's gateway device, along with the Virtual IP. (The gateway device is, of course, set properly so that traffic to pfSense isn't filtered or blocked).

So now my IPsec VPN works with one of the static IPs, and traffic from the computers behind pfSense is seen as coming from the DHCP address assigned by my ISP, as I need it to.

I think i solved it .. from what you were saying and the helping and the how the rules go
and then you mentioned thats normal goes to wan and also the vpn that got me thinking i need to block it
it seems to be working i have VPN for my computer and bypass for the xbox and its open.. ill test more tommorow and get back to you but this is what i did seems to do the trick
0_1534551239191_PP2.JPG

@Derelict @jimp thanks for that feedback. I'll will try as you suggest and report back.

I had another conversation with a friend last night and came up with 3 other possible solutions as well.

Ask the ISP for addresses in the same block with the same gateway(preferably in our original address space). I asked this yesterday day and waiting for a response. This is as @Derelict said.

If the above isn't possible, can they tag the new gateway and I could at a vlan sub interface on the wan. Not sure this is possible in pfsense as I haven't had time to investigate.

Add a dumb switch in front of my firewall and split their connection into 2 connections and use another interface on my box for the new gateway and ip's.

While senerio 1 is the most desirable, anyone see problems with 2 or 3?

We've had our public IP for over 10 years and while I could just get a block of them all together we would like to keep our existing.

That being said since our existing is a 173 in a 24 block and the new ones are 208 in a 24 block is oblivious that our ISP is trying to conserve IP's by using 24's and not splitting the blocks up into smaller 28,29 or 30's. Why make 30'and limit the customers they can handle to 64 instead of 254... So I'm thinking or primary IP block is probably full which makes me think I'll be looking to solution 2, 3 or the above as jimp stated.

And the no particular reason we would like to keep our existing IP, other than we've had it a long time...

Do you mean a HA setup with primary/secondary firewalls, or just a dual WAN configuration?
If you mean a dual WAN, your question has two parts-

You could add a rule to policy route 2.2.2.0/24 via the Failover connection gateway. If you have the primary on tier 1, and the secondary on tier 2, it will only use the secondary when the primary is down.
Or I may have misunderstood. Please add more details. Maybe a diagram.

Me and my family are around 6 members with heavy usage. With all their devices connected one of the wans shows 0-3KB/s constantly while only one wan is being used to full. Maybe it's the same gateway issue, I'll try and repost here. As for MLPPP I think they do. Also The videos on youtube show that you can get combined speeds on speedtest and some people I asked say you can and some say you can't, it's conflicted opinions. As for things like steam and IDM that use multi threaded downloads to same server you should get combined speed.