@mgavrila said in IPSec & OSPF, ping YES, TCP No. OpenVPN & OSPF work as expected.:

@cmcquistion_ This is an expected behavior. Take a look here https://docs.netgate.com/pfsense/en/latest/config/advanced-firewall-nat.html#firewall-state-policy

Thank you!

This is the clue that I needed.

I wasn't completely sure how to create the "Rules with Floating Policy Se" referenced on that link, so I instead I just changed my IPSec rule that was already in place for that interface (allow all) and changed the State Policy from Default to "Floating States"

Once I did that and did a Filter Reload, all my traffic is working as expected!

This is good to know. I have a lot of client firewalls that use IPSec and OSPF that are going to stop working when I upgrade their pfSense version unless I implement this change.

@Zotan said in Disable WAN port detection:

package system has detected an IP change or dynamic WAN reconnection - 192.168.90.129 -> 192.168.90.129 - Restarting packages.

system has detected an IP change ... as often as every 2 seconds.

And you don't like that ? That's an understatement.

But if some one is hammering on your head, don't try to remove your head.
Remove the hammer.

First, do the usual tests : hardware :
Check / change WAN cable.
Put a switch between the WAN port the the device at the other side.
Swap WAN and LAN interfaces. if its now the LAN, ditch the NIC.

Software side :
Reset pfSense to default - no, better, re install and do not import your config back in.
Problem solved ? Go have a talk with the admin, as he introduced the issue with one of his 'settings'/'config changes' ^^
More tests are possible, but I don't know how/what you use on your pfSense.

edit : and as I needed 25 minutes to type all this (I'm also supposed to actual 'work') I just see your second post.
You've talked to the admin 👍

@madbrain Well, with no attic or any other type of crawl space, I suppose you don't have many options. I guess one could dig a "trench" close to the wall to hide a cable that goes around the house, unless there are concrete patios or similar, blocking that option... Another possibility would be to use the gutters to hide cables behind them...

Perhaps vent drain is not the correct translation, but what I meant was the vent for your plumbing. When flushing for example, air needs to come in from somewhere. But with no attic, that is a moot point anyway...

@erdeed I'm not entirely sure I'm understanding exactly your thoughts here, but perhaps it's something like this:

You want to have clients using VPN to come in via pfsense and then be directed out on the internet again on each IP depending on which client it is. So their "public IP" is now one of your IP's from the block, not their own? Sort of what you get when subscribing to NordVPN etc.

So you have one physical interface with a block IP's from your ISP with N IP's available. The key here would be that you also need matching interfaces in pfsense.

If you have enough physical ports on your pfsense machine, you could simply put a switch in front of pfsense and connect ISP-cable to port 1 and the other ports 2-N to your WAN ports on pfsense. Each interface will have a unique MAC and therefore get assigned individual IP's from your ISP.

If you only have one WAN port on pfsense, you need to use VLAN instead. So using a managed switch you can create a matching number of VLAN's, and using only two ports on the switch where you basically allow the switch to TRUNK all VLAN's towards pfsense.

Switch port 1 to pfsense (VLAN Trunk ID 1, 10, 11, 12, 13, 14 etc)
So fiber to switch port 2 (fiber/cable in) (set it to VLAN TRUNK untagged I suppose??)

In pfsense you create VLAN's and assign them all to the one WAN interface, and make sure again that they each have a individual MAC addresses. Then you should be getting one IP per virtual WAN interface...

Whether you set up your VPN server in pfsense or have it running on a server on your LAN probably doesn't matter. It's perhaps more a matter of compute resources...

But in pfsense you need to define policy routing rules to make sure each individual VPN-tunnel-IP is routed out the desired interface.

I might have missed something here but I think that should cover it...

@michmoor After extra round with the ISP, they just now admitted now they forgot to inform me of maintance! So it was my ISP!

"who’s MAC address changed and how do you know?"

The log I posted above, you see the ISP's box changed from one mac-address to another. I assume the log line below shows the mac address of the connected device on my WAN-port (igb0). Since their box is directly connected to this port, it can't be anything else than them.

May 13 00:24:06 kernel arp: ISP_BOX moved from d0:d0:4b:66:6c:75 to 30:fd:65:89:4a:1a on igb0

@ThM hey there,
just a sidenote: your (static) IP for pfsense (192.168.100.5) is right in your dynamic dhcp pool (.1 - .254)...
You might want to change that, so that your DHCP Pool is not overlapping with IP reservations or static IP settings...

@mikey_s 2GB is a lot of dns requests... But sure could see that happening.

Lets do some math for curiosity sake. Does your 2GB count both up and down traffic? Lets say a dns query is 500 Bytes total up and down.. Doubled what I saw in simple sniff just for cushion in our math. You would have to

query.jpg

So that is what 4 million queries? My whole network, lots of clients in last 24 hours have done

queries.jpg

Now keep in mind that I change the min ttl to 1 hour, so this will be skewed.. Many ttls these days are short 60 seconds, 5 minutes.. So sure number of queries will be up.. So if you were doing 10x that or 280k queries a day.. Doesn't take long to get to 4million queries.. A 5 minute ttl if something is being asked for all the time would equate to lot of queries, and if something is banging its head looking for something.. Shoot I have had a single alexa do 2Million queries in 24 hours before.

Lets not forget the pings for monitor, default is what 2 a second. Small but there will be some data there. Even with zero byte payload.

So yeah I would think it quite possible to use up a 2GB of bandwidth without really even moving any traffic at all.

I would suggest you do a sniff for say an hour of traffic out your lte interface.. With no clients really even using it.. Then do some math to how long it would take to eat that 2GB up.

With such a low amount data limit to work with.. I would prob make that failover a manual process.. And I wouldn't let it do dns queries out it until such time that is your only connection. And I would for sure limit the min ttl to something less than many sites use these days of those insanely low ttls.. And look to see how much data just monitoring is using..

So just adding up the pings, and have payload set to 0...

500M.jpg

30 seconds is 6KB, so what is that like 17MB a day just in pings, or 30 days like 500MB.. which would be 1/4 of your monthly quota just in monitor if the gateway is up.

LTE can make for a great backup, but if you have a low data quota - it would be quite easy to suck that up all with just background noise like dns and monitoring to be honest. Depending on what counts against your quota..

Did you get that for free? Its not quiet, and it sucks juice.. Prob about 75W just idle..

No it has no "switch" ports.. That is not a box I would recommend to use in a home.. Unless you were a labber/IT hobbiest, etc.

You in the UK, sky broadband is the big one over there so that is what I would guess.. Whats your electric cost like 24p a kwh? So that thing sucking 75 watts idle cost you like a 160 a year.. Your going to spend more than just getting a 200 something box that uses 20 watts (prob way less) in 2 years..

200 box
42 a year for electric
42 2nd year

your at 285 lets call it.. With that box in 2 years you have spent 320 just in electric. If you got it free.

This sort of gear doesn't make a lot of sense for home user.. They are normally loud! they suck a lot of juice compared to other options. They are normally way overkill for home use.

Unless you lab with it, and its not on very often.

Even when you get something like this for free - it can end up costing you more money then if you would of just bought something appropriate for your use case.

Whats your internet speed? Something like a sg1100 be better suited for a home use of pfsense. Its small compared to that thing. It sucks like only 4 watts idle..

@Antibiotic I wait 5 minutes and test again, now looks like start using Tier 1

@rikazkhan Your message was just a quote. Did you mean to add something ?

@manjotsc glad you got it sorted, and thanks for explaining what the actual problem was - this for sure helps the next guy!

@klubar well I would just delete the other route as well to see if they come back. The only thing that I recall where routes like that would be set is if had set dns per interface. With the gateway, see my dropdown that says none above.. And I thought that if you allowed dns to be overriden by dhcp it could add a route there too

Maybe one of those things is where they got added, but never got deleted.. I would just delete the other route, and make sure next time you reboot that they don't come back.

Are they using dns to resolve what they should ping to see if hey have internet? Not a fan of company XYZ using some other service to check if your device you sell has an internet connection.. And you sure and the heck shouldn't hard code IPs.. If you want to check if your device you sell to people can get to the internet it should use what ever dns was provided to your device by either dhcp or set on it and look up some fqdn that you the company controls.. If you then want it to ping that IP, it should be pinging your resource not some other companies IP..

NTP is in line with this as well - if you making a device that will want/use ntp.. And you want to point it to the ntp pool, then get with ntp org and get your own vendor fqdn.. And if your just going to point to ntp in general, don't for F sake point to your country code..When the device is going to be used in different country..

I had one of those smart wifi plug things, still do actually just use it when put out the xmas lights. But clearly the thing is not going to be used in the UK.. We use different power plugs, so there is no freaking way I bought this in the uk and just using it in the us.. But it wants to check its time with the uk.ntp pool.. So I created a host override for what is was looking for..

ntp.jpg

Just pure laziness if you ask me, or they are hiring the cheapest developers they can hire? Or they have some developer and said hey make this work by tmrw we need to start shipping them out next week.. And failed to even give him any parameters or where it might be used.. Hey have it check ntp time while your at it ;)

My other pet peeve is these iot devices that have zero dns cache.. Ok we know you want to talk to fqdn xyz.. But do you really need to ask dns every 2 seconds for it, when you were just given the answer 10 seconds with a ttl of 24 hours ;) I mean it takes really nothing to store the dns record for the next time you want to go there in 2 seconds.. You don't have to store 10k records you need to store the handful you might be wanting to talk too..

Well now I have just gotten off on a rant, sorry ;)

@heliop100 I think you have to give permission - route - to the LAN segment to go out each of the gateways. This is done under firewall, NAT, Outbound. Usually it is recommended before adding rules to select manual then save. Then start adding rules for routing.
Untitled.jpg

@viragomann said in HA, gateway groups and firewall's default route.:

Is it this, why you were looking for localhost?

Not OpenVPN, but the outbound connections from the firewall itself.
In the documentation, localhost should be 'NATted' to interface IP address, which is OK, this part is clear to me.

What wasn't clear to me is if I should have created a new gateway group, using Interface address, to use as the default route of the firewall.

However, since it doesn't translate anything and everything is working, including IPsec, I'll leave as it is right now, everything using the same gateway group, that has the CARP IP there.

Note that without those CARP IPs in the gateway group, IPsec tunnels won't go up.

Gut, denn das sollte einfach so funktionieren, wenn die IP der Fritz im Netz liegt was durch den VPN Tunnel geroutet wird.
Also Route in die Fritz mit Ziel pfSense für das gleiche netzt und auf geht es.