No offense taken whatsoever. Thank you for taking the time to reply Eddie.

After a bit of reading and understanding I've managed to get Dual WAN working correctly for load balancing and failover (working both ways for possible LAN duties - VLAN maybe). All good so far.

pfSense LAN -> Netgear GS728TP -> all my stuff
pfSense WAN -> Vigor 130 (bridge mode)
pfSense OPT1 (WAN 2) -> Asus 4G-N12 LAN

However the Asus 4G-N12 warns that it is in a multi-NAT environment and consequently I cannot set-up DDNS which I need to do because EE will not supply me a static IP.

Is anyone in the UK using any of the known working 3G 4G modems with EE and does the device have a bridge mode / passthrough mode or DMZ that allows me to use DDNS?

Thanks

you can reduce to 1 firewall..  Not sure why you think you need another firewall for more vlans?

"But for some servers we have 2-3 firewalls in 1 VLAN"

That just seems crazy!!!

For your different vlans you can either just use interfaces in the 1 firewall, or just use vlans on top of an existing physical/virtual interface.

Well yeah that would be the normal way to do it ;)  I have no idea what you were attempting to do other than create a train wreck ;)

Glad you got it working, KISS is your friend when setting up networks…

here you go

with the current config everything is working as expected. if i change wan 2 ip to an unused private ip. gateway monitoring and ping from wan 2 to internet stops working.

network.JPG
network.JPG_thumb

Sorry for hijacking but I have the same problem. 2 internet connections with pppoe. 2 subnets and each subnet should use one pppoe connection, so no balancing and failover. I have 4 interfaces. The 2 LAN interfaces work fine, but I can only get one WAN interface to work. If i deactivate one WAN interface the other WAN interface gets an internet connection. If both are active only one is working.

Any ideas?

Where is your controller running?  On your lan, and your AP are on 2 different networks wifi and wifiguest?  Why do you not just put the AP on your lan and use vlans for wifi and wifi_guest?  So then you controller can see your AP on layer 2.

If you want your AP on different layer 2 network than your controller then you need to use layer 3 adoption and management.

https://help.ubnt.com/hc/en-us/articles/204909754-UniFi-Layer-3-methods-for-UAP-adoption-and-management

Please give more information on your current setup…

What modems are you using?

Are they static or DHCP?

How is everything connected...

What pfSense hardware are you using?

in my opinion best thing to do is take screen shots of all your configuration and post them on here like this
http://imgur.com/a/CI7nl

with all that information then someone can help

you guys are awesome thanks!

Thanks alot for your explanation, makes things way more clear for me :)

What does your pfsense box have 2 wans?  And why do you not just connect your freenas to a switch port on your lan??  Or for that matter just put it on an different network segment lets call in your nas segment and then just route/firewall between your lan/nas segments?

Bridges do not turn interfaces into switch ports.  If you need more switch ports on a specific segment, get another switch or a bigger switch, etc. etc..

@Derelict:

pfSense doesn't care what APs you use, unless they're somehow broken.

Alright, gotcha. Thanks a lot for the help!
Going to start ordering it all now  ::)

So again lets ask what is routing these networks?  And how exactly do you have this host connected to both?  What is its routing table.. Is it windows machine - post up

route print

If linux

netstat -r

"If I have a host that needs to be in two networks to separate the traffic that needs to be separate"

Why??? does a host need to be in 2 networks at the same time… If he is in 99, and needs to talk to something in the 66, why can you not route this traffic??

Yes you can have machine connected to more than 1 network at a time, but you wouldn't use both of these networks to route to a default.. So for example you could have the attached pic.  Where machines have an interface in 192.168.0/24 this would be their default gateway.

They then could have another interface in 192.168.1/24 and they could use this interface to talk to each other - lets say for backup or something..  But you would need to make sure you use the 192.168.1 IP address of the other machine, or a fqdn that resolves to that 192.168.1 IP.  And 192.168.1 interface would not have any gateways set on it, etc etc..

multihomed.png
multihomed.png_thumb

Your problem description is unclear. Load balancing doesn't set bandwidth.

However:

You can adjust the gateway weights in a load balancing setup.

You have a combined upload of 17Mbps with 88% on one circuit and about 12% on the other. Dividing each by 4 brings you to 22 and 3 respectively.

Try setting those values on each gateway in the advanced settings as the Weight.

This is not perfect as the system has no way of knowing how much bandwidth a given connection is going to use when it is established and the route is chosen. It also has no way of knowing whether the connection will be used to primarily send or receive data. This algorithm will establish 22 states on one circuit and three on the other for every 25 states created. Sticky connections should also override this.

Should help at least little.

Note that your download is split 60/40 so this might result in under utilization of the 10M download. You might want to skew them toward something between 88/12 and 60/40. Like maybe 70/30 (weights of 7 and 3).

See Also: https://portal.pfsense.org/docs/book/multiwan/policy-routing-load-balancing-and-failover-strategies.html#multiwan-unequal-cost

If you have known traffic that generates uploads you can make another gateway group that fails over from the 15/15 to the 10/2 instead of load balancing the two. Policy route that specific traffic to that gateway group then policy route everything else to the load balancer group. If the 15/15 happens to go down, that traffic will use the other circuit.

@boulate:

My idea was : If "4g Modem (192.168.0.1)" can respond to "Pfsense client (on 192.168.0.100)", and if "Pfsense client (on 10.0.2.1 and 10.0.200.2)" can respond to my "Poste de travail local (10.0.1.1)", the it must be a rooting problem only on the "Pfsense client" no ?

It works similar to this.

If your PC in 10.0.1.1/24 sends a packet to the 4G modem, the packet has the source address 10.0.1.1 and the destination IP 192.168.0.1 when it arrives at the modem. Since you have a site to site VPN, the VPN tunnel network itself is irrelevant here. The packets are just routed over that subnet.
So the 4G modem will send its response to 10.0.1.1, but since this address doesn't fit to any of the subnets on its interfaces and it has no special route for this host, it will send the packet to its upstream gateway. Presumable that's the internet provider.
However the modem will response correct to the pfSense clients WAN 192.168.0.100, because this is a subnet connected to its own interface.

So let's do NAT to get it work:
Go to Firewall > NAT > Outbound, if the rule configuration type is set to Automatic set it to Hybrid or manual and hit save.
Then add a new rule with
interface = WAN
source = 10.0.1.0/24
destination = 192.168.0.0/24 (or any if you also want to access internet hosts over the VPN)
translation = interface address
Save it.

This NAT rule will translate the source address in packets coming from 10.0.1.0/24 to the clients WAN address. So your modem will send responses to that back to the client pfSense and this one will send it back over the VPN to the PC 10.0.1.1.

Thanks Derelict for the response. Yes I think go with first option. I'll enable  pass any rule in the WAN interface.

I think I'll not opt for second option as I'll be using captive portal for LAN A users and later I am planning to introduce Traffic Shaping to prioritize VOIP.

Thanks
Ashima

Can u post screenshots on your configuration?

I want to do the same but something is not working.

My mail server sits on an internal lan with a address scheme of 192.168.10.0/24. I have a NAT rule that associates one of my public IP's to the mail servers IP. As an example the NAT rule looks like this 62.62.62.62 -> 192.168.10.62. I then have rules on the WAN interface to open up the FW for the ports i need for my mail server (25, 465, 143, 993, 443) these rules all have a destination of 192.168.10.62.

The static IP's for the mail server comes from the WAN connection which is a static IP connection. DNS is setup to point the domain name for my mail server to the public address of 62.62.62.62

My second WAN connection (WAN2) is a DHCP connection which is load balanced with the first WAN connection. The load balance setup works and I'm able to search the internet fine and speed test result in the results i expect. The only issue I have is i cannot connect to my mail server via the internet. On the internal network it is fine.

Please let me know if you need additional information.

Thanks,
Judd

NOPE

No solution to this yet.

In fact i feel that while PFsense has matured from a codebase standpoint, it has seriously regressed from a reporting and user management standpoint.

I have felt that the PFsense team has never really focused on the user facing reporting, monitoring and telemetry aspects of this platform.

Even in the latest release this has continued in that they have even gone so far as to remove the ability to email RRD graphs to administrators.

I am forced to look elsewhere for a firewall solution so that I can properly do my job as manager of getway services for my clients.