Having the same issue.  Although this between the OpenVPN server and the client.  What happens is when the PfSense is rebooted and a client connects to the vpn none of the routes are pushed to the client, only after I go in to the OpenVPN configuration and click SAVE will it start working again even though the routes are still there.

I think it could be the OpenVPN .conf file is overwritten after reboot and anything in the bottom box where you'd put you custom routes are discarded.

2.3.2-RELEASE (amd64)
built on Tue Jul 19 12:44:43 CDT 2016
FreeBSD 10.3-RELEASE-p5

openvpn-2.3.11                Secure IP/Ethernet tunnel daemon
openvpn-client-export-2.4.2_1  OpenVPN Client Export

pfSense Updates follow the routing table, not policy routing.

Hello,

Not IPSec, so I don't know if it fits your needs, but OpenVPN, here: https://doc.pfsense.org/index.php/Multi-WAN_OpenVPN

Best regards

Kostas

I just installed another host which is connected to the first pfsense and this host cannot ping 172.16.20.1 either, so this is not an OpenVPN issue but a routing issue.

On the OpenVPN Interface (does also apply to the other server interface):

18:17:04.562619 IP 192.168.68.2 > 172.16.20.1: ICMP echo request, id 53527, seq 0, length 64 18:17:05.551177 IP 192.168.68.2 > 172.16.20.1: ICMP echo request, id 53527, seq 1, length 64 18:17:06.595303 IP 192.168.68.2 > 172.16.20.1: ICMP echo request, id 53527, seq 2, length 64 18:17:07.598748 IP 192.168.68.2 > 172.16.20.1: ICMP echo request, id 53527, seq 3, length 64

On the interface which connects both firewalls:

18:18:15.316407 IP 172.16.58.250 > xxx.xxx.xxx.193: ICMP echo request, id 21153, seq 5564, length 8 18:18:15.316952 IP 1xxx.xxx.xxx.193 > 172.16.58.250: ICMP echo reply, id 21153, seq 5564, length 8 18:18:15.321373 IP 172.16.58.250 > 172.16.58.254: ICMP echo request, id 21835, seq 5592, length 8 18:18:15.321385 IP 172.16.58.254 > 172.16.58.250: ICMP echo reply, id 21835, seq 5592, length 8

xxx.xxx.xxx.193 is the gateway IP of the public subnet. This also happens if I use an internal server which is not connected via OpenVPN. It looks like the backcoming packages are routed on the public gateway ip and not back to the subnet.

I attached 2 pictures which show the gateway configuration and the static route. The selected interface is the interface where both pfSense(s) are connected.

pfsensegw1.jpg
pfsensegw1.jpg_thumb
pfsensesr1.jpg
pfsensesr1.jpg_thumb

There is no reason for it not to be working.

Load Balancing does not combine two circuits into one. The only technology that can do that in pfSense is Multi-Link PPP.

Load balancing distributes states across multiple links with the end goal of getting more of both circuits utilized.

Did you enable sticky connections or anything like that?

A single speed test site has never been a good way to test this. The last time someone said it didn't work I tested it with T-Rex. The results are here:

https://forum.pfsense.org/index.php?topic=124373.msg697215#msg697215

That thread is probably worth reading.

This too: https://portal.pfsense.org/docs/book/multiwan/index.html

I solved the HP part, running the following on the switch CLI:

****************************************************************************** * Copyright (c) 2010-2016 Hewlett Packard Enterprise Development LP          * * Without the owner's prior written consent,                                * * no decompiling or reverse-engineering shall be allowed.                    * ****************************************************************************** <hp>system-view System View: return to User View with Ctrl+Z. [HP]ip ttl-expires enable [HP]ip unreachables enable [HP]</hp>

Reference: https://community.hpe.com/t5/Switches-Hubs-and-Modems/Troubles-with-traceroute-in-Switch-HP-5500g/td-p/5880679

Now, tracing the route to Google Public DNS ( 8.8.8.8 ) my router appears:

# traceroute 8.8.8.8 traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets 1  10.100.132.1 (10.100.132.1)  0.551 ms  0.773 ms  0.940 ms 2  * * * 3  187.86.158.121 (187.86.158.121)  6.439 ms  6.440 ms  6.437 ms 4  172.21.1.133 (172.21.1.133)  7.674 ms  7.676 ms  7.672 ms 5  172.22.100.137 (172.22.100.137)  7.667 ms  7.663 ms  7.659 ms 6  172.22.100.121 (172.22.100.121)  7.654 ms  2.738 ms  2.578 ms 7  ip-187-86-128-93.vetorialnet.com.br (187.86.128.93)  2.638 ms  2.641 ms  3.039 ms 8  177-101-203-189.static.stech.net.br (177.101.203.189)  8.913 ms  9.578 ms  10.419 ms 9  xgborder-rs-pae-01-xe-0-0-0.3300.stech.net.br (200.152.253.252)  11.026 ms  11.136 ms  11.506 ms 10  * * * 11  108.170.245.161 (108.170.245.161)  37.144 ms 108.170.245.129 (108.170.245.129)  36.718 ms * 12  209.85.242.119 (209.85.242.119)  36.232 ms * 72.14.238.221 (72.14.238.221)  36.333 ms 13  google-public-dns-a.google.com (8.8.8.8)  55.787 ms  55.838 ms  55.688 ms

Maybe there is something like ttl-expires and/or unreachables for pfSense?

Anyone else got any input on this? Would it be worthwhile to maybe post a bug-report?

Me, I would make a transit network between the WAN pfSense and the proxy pfSense and disable NAT on the proxy. I would not try to put the same subnet on both sides of the proxy.

| $ route get 10.200.100.0 |

| route to: 10.10.100.0 |
| destination: 10.200.100.0 |
| mask: 255.255.255.0 |
| gateway: 10.10.0.1 |
| fib: 0 |
| interface: re0 |
| flags:<up,gateway,done,static></up,gateway,done,static> |

| recvpipe | sendpipe | ssthresh | rtt,msec | mtu | weight | expire |
| 0 | 0 | 0 | 0 | 1500 | 1 | 0 |

| $ route get 10.200.100.100 |

| route to: 10.200.100.100 |
| destination: 10.10.100.0 |
| mask: 255.255.255.0 |
| gateway: 10.10.0.1 |
| fib: 0 |
| interface: re0 |
| flags:<up,gateway,done,static></up,gateway,done,static> |

| recvpipe | sendpipe | ssthresh | rtt,msec | mtu | weight | expire |
| 0 | 0 | 0 | 0 | 1500 | 1 | 0 |

Resgard,

Rodrigo Prazim

Right.. I've figured it out. Basically I'm committing some sins which are causing unpredictable behaviour. I'm not too stressed about them now as I'll be moving away from this setup relatively soon and I have tested with a new, working setup.

For anyone else reading, my sins are:

Mixing VLAN and untagged traffic on the same interface That interface is a VirtIO (which doesn't really work with VLANs I believe)

Once I tested a new build running under ESXi, with VMXNET3 drivers and all separate interfaces (so to pfSense), problems went away and behaviour was as expected.

Hi,

For starters,  you should have your WANs in separate networks, not the same..
Then, for policy routing, you need to create IP Aliases with the hosts you want to use the specific WANs and set their gateway accordingly, in a LAN rule..

Put that rule on top of the other LAN rules.

Best regards

Kostas

I think I can setup multiple routers and have them online all at the same time.  I will be able to swap real easy and add devices easy.  I worked with EIGRP for 15 years so I have an idea of what a basic routing protocol can do.  BGP is not what I need.

So the big question is how stable is RIPv2?  I know RIP will not work in a large network but at my house it should do what I need.

If I go down this road and spend the money.  I don't want to find out pfsense does not work otherwise I will have to dump pfsense for something else that does work.  I feel like once I spend the money I am committed.

Either make it the default gateway or with the advanced option s

It's something like 'tcp outgoing address'. … Been too long since I bothered with it.
Try browsing the squid documentation

i just wasted an hour after setting up gateway groups wondering why NAT reflection broke…

i strongly suggest this side effect should be mentioned in the pfsense book - which i didnt find or overlooked.

Start by updating to the latest stable version. (read the upgrade notes)

roundrobin

Yes you can!

Create a Rule In a Firewall > rules > Lan, you can specify a gateway in Advanced Options from a specific source (this should be the IPs statically mapped, of your roommates. Create an aliases IPs). Put this rule on top of the generic pass all in LAN.