In fact, it should be NATed, this is why I am confused But as I told in the past, I want move everyone in 10.x.y.z networks to have no problem (as I know also the problem using Public IP for internal use…) I just wanted to make the move to the new network smooth But as I discussed with my boss, he finally agreed with me to move directly to the new network, so it will not be an issue anymore for us. thank you PM

;D Thank you both for your assistance, I've managed to achieve what I set out to do, although it cost me a sleepless night. Turns out there was a static route problem(there was a router on the OPT6 network, which was the gateway for all the machines. I setup static routing on said router for the LAN subnet through the OPT6 address, and now she works like a charm!) Once again, thanks a stack…once I was able to rule out my PFsense box as the problem, I knew where to look. Couldn't have done it without you guys!

You can also make an alias for things like "WAN2 PCs" and add a firewall rule to direct traffic coming from those PCs out WAN2. Then you can just edit the alias to include whatever system you want to route the other way. Bear in mind that will only work for new connections, existing states won't be cut off.

@martap: In my setup I use the DNS forwarder as the resolver for all the clients so even though failover works great using the multiple gateways option internet access does break down because of DNS forwarder not able to forward its requests to the internet dns resolvers. Pitty… You need a static route for one of your DNS servers for 1.2.3 (read the docs), or in 2.0, just pick your other WAN by one of the DNS servers in the drop down box on the general setup page.

Perfect, thanks for that Perry, i'll give beta 2.0 a go then, see if I can figure it out  :) Jon

You have to setup NAT before you can ping or do anything else with a Proxy ARP VIP. Use CARP if you must have a pingable VIP without NAT.

As I said, not something a lot of people need.  ;D My last comment in #8 explains why it won't be done for 2.0. You can use sticky connections to avoid the breakage, and alternatively you can easily modify the source to kill all states on every status change which sounds like probably what you want. Or put any command in there you want, kill only states for specific IPs, lots of possibilities.

It started working again since my last reboot, so rebooted again just now to get it to the problomatic state. This is what I get in the system logs regarding apinger: Sep 3 06:22:42 apinger: ALARM: 8.8.8.8(8.8.8.8) *** down *** Sep 3 06:22:42 apinger: ALARM: 8.8.4.4(8.8.4.4) *** down *** Sep 3 06:22:53 apinger: Error while starting command. Sep 3 06:22:53 apinger: command (/usr/bin/touch /tmp/filter_dirty) exited with status: 1

Cool.

@anthony0975: What is the best way around this?  Can PFsense somehow group both WAN and WAN2 into one Zone so I can just select the zone instead of WAN or WAN2?  I dont think bridging will accomplish this?  Or do I just need to double up on every single rule and have one applied to the WAN interface and the other to WAN2 On 1.2.3 you'd have to double up the rules. On 2.0 you can setup an interface group and manage them together.

ahhhhh so if these modems have routing functionality built in, could I disable dhcp on both of them, assign them unique statics on different subnets, i.e modem1: 192.168.80.1 and modem2:192.168.81.1 and then plug them both into a dumb switch and plug it into my wan interface? If not, i'm going to have to slap another nic card in this box… can I slap another nic into pfsense without re-installing? -m

Submicron, Thanks for that information, it shed some light on the subject… Cheers..  ;D

I posted another topic on this new "side-effect" I'm experiencing. Twice now I've started getting this message: kernel: arplookup x.x.x.41 failed: host is not on local network And my second subnet becomes inaccessable via the rules I've provided directly to the public IP's on the servers. However NAT rules still work. Any thoughts on this? The first time I did it, I fiddled with the DMZ stuff, unbridged, rebridged, rebooted, and it worked, this time it won't start working at all.

FreeBSD doesn't support routing metrics in that way, it would be nice if it did. You might also want to look into running a dynamic routing protocol like OSPF to exchange routes. That does support metrics.

@jimp: 1: GRE would have the lowest overhead, but wouldn't be encrypted. OpenVPN would offer the best of both worlds, but would have some overhead. 2: Simple policy routing will do. Have a rule that passes out from your LAN to * on http/https with no gateway set, and a rule underneath that catches the rest and passes out anything else to the gateway for the VPN (If you assign an OpenVPN instance as an interface you can create a gateway for the other end and use it in policy routing, GRE may allow the same) Two more things: do L2TP and PPTP have the same capabilities as OpenVPN (i.e. can be assigned as an interface, be used with policy routing, etc.), because should I not just use GRE but a VPN, I'd rather use L2TP and PPTP because Mac OS X supports these out of the box, and I try to use as few different modules as possible, because the more there are, the more potentially unexpected interactions and side effects. How stable is 2.0 in the mean time? In particular, what I'm interested in is if I should be able to upgrade REMOTELY from whatever beta (4?) we have now to the final release, because I can't ship the unit back and forth to a colocation provider somewhere half across the US just to do a software upgrade. So if the upgrade mechanism is stable and robust enough, and the configuration is forward compatible, I can start working with pfSense. Otherwise I'll have to wait until whenever these two conditions are met.

I use OSPF to handle redundant OpenVPN connections over multi-wan to ensure I can connect to some locations. It works really well. I wouldn't let the beta tag on that package fool you.