I searched a while and found some references that the behavior changed in Windows 10, where Windows uses the interface metric to send DNS queries. Traffic still goes out the default interface. Lower metric = higher priority.
I have the exact same settings from two other sites into Site B and it works no problems at all.
Additionally, I had to add port 500 into the firewall rules as otherwise I was getting retransmit 4 of request with message ID 0 errors when attempting to establish a connection. Now that's gone, but the problem persists!
Something must be awry with the IPSec configuration. Getting two tunnels going should be the easy part, just use unique ip's on the vti connections and set one with a higher weight in BGP. The problems I've seen have been packet loss unless mss clamping is on, and a tunnel not re-establishing if the line is down for an extended period of time. What messages are you getting in the ipsec log when the second tunnel comes up?
@topogigio In terms of Bell admitting to anything, good luck with that! It's not like Bell is connected to us directly, our ISP is. They (Bell) will not admit to them... Imagine us. Our location REALLY limits the choice of providers; we're lucky enough that they can provide service @ 1Gbps. The tunnel @ 300Mbps is fine with day-to-day operations, but backups between sites are handled differently (Which was the big headache) as we encrypt the stuff with Veeam and carry it with our QNAPs HBS3 sync functions (With SSL encryption on) out of the IPSec tunnel (Thus at high speed).
In terms of the expectations for the 100E, it is way over 300Mbps for IPSec; granted it's not 4Gbps or something with very small packets (In reference to the specs), but it's over 300. We were able to go as high as 740Mbps with a non-throttling ISP...
@betahelix So now there's an OpenVPN tunnel too?
Can you show the actually topology of both sites?
When I have VPN issues I always use the Packet Capture under Diagnostics menu. Try that on the 2100 while pinging from the WireGuard clients.
@manzanoso correction you dont need a static route per se.
what is the status of the tunnel? Status > IPsec
What I could identify is that when the notebook is on the 192.168.150 network, I can transfer packets on the VPN, however, when the notebook is on the 172.23.0 network it does not work. I'm using a nat for output which is what is needed for the other end of the VPN.
First of all, sorry for my own self reply and thank you for your responses... I'm just very frustated. I've been creating VPNs to Oracle for some years now (even with pfSense Tunnel and VTI with other softwares) but pfSense VTI has never been an option for some reasons. This time I wanted to give a try.
I have just undone everything and just given up pfSense. Firstly I went back to the usual Tunnel IPSEC that works as expected. No modifications are needed to make it work on Oracle's side so the problem might/may/must be related to pfSense. If you guys had some links to post here I'll read them all to try to find out what I've done wrong.
I followed this guide https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/routed-vti.html and another thousand recipes available on the net.
Not even the gateway monitoring works !!!! What on earth could be impeding the gateway monitor to work ??
I know many of you have this setup working but as far as I could find there are a lot of complaints like mine.
I must be having a bad week, even posting to this forum is really hard.... akismet keeps telling my post is a SPAM :\ lost good 60 minutes trying to post.... I was trying to ask about the gateway monitoring thing. I have just given up. As you've said, this is a community forum and I should really no wait so much of it although it has already helped me lots of time (thanks guys).
Problem Solved !!
Thank you viragomann that change to the IPsec Filter Mode did the trick.
Everything works perfectly.
Note to anyone reading this. I did have a second VPN to the site 2 that I had not mentioned in my explanation and after reading "All IPsec tunnels have to be VTI" I changed that to a VTI also. I also made the assignment of the IPsec tunnel rule to any / any and my forwarded port from site 1 began working to site 2 as soon as I changed the IPsec Filter Mode.
@jimp Thank you for your clarification. You saved me time on testing this. I guess I have to try a more difficult way. @luckman212 I found the same tutorial, it looks like it describes pretty much the steps we need to go through to set up dual-wan.
Install the iperf package at both ends. Use that to determine what your baseline end to end speed really is. Now run it over your ipsec tunnel. If there is a substantial difference then that needs looking into.
@mamawe As far as I know that type of NAT has never been valid on an IPsec tunnel. You can do 1:1 or Many:1 but not Many:Some_Other_Size_Many.
Maybe it wasn't clear from my answer.
I used Many:1-NAT and 1 address for our side of the VPN traffic selector.
The last two sentences referred to the peer VPN gateway.
Some implementations allow to negotiate a smaller traffic selector in phase 2 as was configured (1 address instead of a subnet). With these you don't have to change anything at the peer VPN gateway.
If the peer VPN gateway insists on using the correct traffic selector, you have to have the peer VPN configuration changed.
@ibnkamala I am saying: forwarding the ports and protocols needed to make IPSEC work in a double-NAT scenario (such as you have) is not an easy task. It seems you might be a little over your head with it.
So, I suggested changing strategy and trying Wireguard, which is a completely different protocol, to establish the site-to-site tunnel. I posted a link to the guide so you could try to set it up.
If you do give that a try, just stick to the default port (udp/51820) and forward that port from "Simple Internet Box" to your inside SiteA pfSense, and also forward from SiteB Sonicwall to pfSense. It's just another possible solution.