Ah, then you should update unbound:
pkg upgrade unbound
It will pull in a new strongswan version with that.

Or try a 2.4.5 dev snapshot which contains that.

Steve

Well, there's Packet Capture, built into pfSense, that can capture all the traffic on a pfSense interface. However, you'd have to manually start & stop it and then download the capture file. If an interface on another device, you'd also need a managed switch, configured to port mirror.

The Gateway logs does not show any logs during today's outage and also we were are not able to connect out from the firewall during the outage period.

Thanks
Tanner

Exactly! If the cat catches mice, then who cares what it looks like!

@steve_b
okay, I'm seeing inconsistent behavior here and I haven't been able to pin down why.

router 1 (the originally not working one):
While looking for the debug logs, I noticed that the log entries for success had disappeared. I traced this to a check for boot completion that was failing, preventing the backup from starting (this is a different issue. I also don't know why the boot is no longer finishing).

I manually deleted the booting file so that the backup would run, and it started working. I then walked back the firewall and DNS config (and the debugging stuff that I had added in acb.php) that I had done before to try to get it to fail again, and it would not fail. So, I don't know what changed to make it work.

router 2 (the originally working one):
This one started exhibiting the behavior seen before on router 1 (backups report success in web ui, but they are not occurring). This one does have a backupdebug.txt indicating a timeout on the save. I can still ping and curl acb.netgate.com.

Thanks for the hint. I think I got the issue resolved. I was trying to use the console to figure out which nic was active so I can assign my lan/wan to the nics i wanted. I was doing this by using option 1 (assign interfaces) and seeing which nic was showing as up in order to assign things before logging in for the first time.

After going into Interfaces > assignments and adding the extra 4 nics and then clicking on each one and selecting enable, pfsense now seems to be detecting the interfaces as up.

I believed the maximum throughput was 42MBps for PPP lining up with HSDPA but I recently saw a report of >60Mbps so I guess it's possible if your carrier and hardware support it. I've personally seen ~32Mbps using a Sierra m.2 modem and PPP.

Steve

What card are you taking out? What are you replacing it with?

pfSense configures the network again interfaces detected by the OS at boot: em0 em1 igb0 re0 etc. The name given to it depends on the driver it uses and the order it is detected in.

If you had those 4 NICs above and you removed igb0 and replaced it with another igb card then you would likely not have to change anything. The new card would also be named igb0 and the config would match it.
If you replaced it with a different card, say a Broadcom card using the bge driver, then pfSense would stop part way through the boot and ask you to assign the interfaces because igb0 referenced in the config no longer exists. You would just assign the NICs at the console to the same interfaces and it will boot up with all the same settings as before.

It gets more complex if you replaced igb0 with another em card because that new card might be detected it a different order so that while you would not have em0 em1 and em3 the new card could potentially be any of them. You can still assign it at the console in the same way but if you find things don't work as expected after booting you might need to re-assign the cards in a new order or swap the physical network connections.

Steve

Hmm, no. Very odd. 😕
Glad you found it though. Might help someone else hitting that.

Steve

Looks like you cross-posted here as well as Reddit, but I responded over there.

For the benefit of others who may come across this: It's fixed in the most recent version of the OpenVPN Client Export package. It now forms the correct FQDN for Cloudflare hostnames when exporting.

Usually when you see that it's because you're using a bash script and pfSense does not use bash. Some commands do not work.

Though here it looks like it's trying to run { as a command.

Steve

Problem has been resolved. Since these units are MultiWan, we found that when the gateway changed, neither unit could resolve DNS internally, preventing them from reaching the ACB Servers. By adding gateways in the System > General Setup > DNS Server Settings and associating different DNS servers with each gateway, we were able to restore the connection and now all is working.

Not sure what PCIe devices might allow that. I've only ever used PPP with those.

Why do you want to bridge this though? If you only have one public IP I'm not sure why you would. You could use that IP on the bridge interface and be able to access pfSense that way but then there seem little point in bridging.
If you use the public IP on some downstream device you wouldn't be able to access pfSense without going through that device somehow.
There is also the fact that most "modems" will want to use the IP themselves and run in router mode.

Steve

Thank you so much for your insight

you were correct, once I added the -S option for source address it worked :-)

ping -S x.x.y.y x.x.z.z

Thank you.

Thank you! I have just updated the driver with the one you mentioned and I did some iperf tests and it looks like it's stable. Cheers!

It should work like that as long as the settings have been applied correctly and there are no conflicting subnets.

However you will be (at least) double NATing which just makes things hard if you ever have to configure port forwards etc.

If you connect the pfSense console does that have a real WAN IP? Can you ping out from there?

You might have to spoof the MAC address on the WAN if your ISP requires that.

Steve

Yes, that's nothing to worry about. Just your internal clients ACKing a reset after the state has been closed by the firewall.

Steve

wow wow wow
There goes my weekend.......
Merci Muchos Guys

@grimson I don't want to do anything off regarding your Germany knowledge, as indeed the real German word is die Lichter. Du hast voellich recht.

But read this this internet folklore:
http://www.blinkenlichten.info/origin.html