Firewall itself has no internet connection

coolsaet

The outbound NAT is in Hybrid mode.
I have setup 2 rules with NO NAT for 172.17.0.0/16 and 172.18.0.0/17 as this are the VPN clients subnets which are only used for internal access.

I did state the WAN Gateway in the WAN interface settings and also put it as the default gateway in the gateway section.

You are right about the 172.42.42.0/24. Did that on a late night. I already fixed it by changing it to 172.31.42.0/24.

I did packet captures on the WAN interface, but not yet on the VPN gateways. Will try that next.

viragomann

@coolsaet said in Firewall itself has no internet connection:

The outbound NAT is in Hybrid mode.
I have setup 2 rules with NO NAT for 172.17.0.0/16 and 172.18.0.0/17 as this are the VPN clients subnets which are only used for internal access.

There should not be an outbound NAT rule generated automatically for these networks.
The only case I know, where pfSense is doing this is, when you state a gateway in the interface settings. But this should only be done with upstream gateways (WAN).

For the VPN you have to add the gateway only in System > Routing > Gateways and add static routes.

coolsaet

@viragomann said in Firewall itself has no internet connection:

There should not be an outbound NAT rule generated automatically for these networks.
The only case I know, where pfSense is doing this is, when you state a gateway in the interface settings. But this should only be done with upstream gateways (WAN).

I didn't add them on the interfaces.

For the VPN you have to add the gateway only in System > Routing > Gateways and add static routes.

Only added them in the static routing and gateway section, but they are still showing up in the automatic generated Outboun NAT section.

I also did the packet capture on all interfaces and now found it on the WAN interface:

But it seems to have a source IP of 0.0.0.0.

viragomann

@coolsaet said in Firewall itself has no internet connection:

Only added them in the static routing and gateway section, but they are still showing up in the automatic generated Outboun NAT section.

On WAN, I guess. This wouldn't matter, since the packets are routed out on another interface.

I also did the packet capture on all interfaces and now found it on the WAN interface:
But it seems to have a source IP of 0.0.0.0.

So the packets go out on the correct interface according to the routing table, but they are not natted.
What if you switch the outbound NAT in automatic mode?

coolsaet

@viragomann

I set outbound nat to automatic, but it still show 0.0.0.0 as the source IP.

viragomann

@coolsaet
Pretty strange. I've sadly no idea, where the 0.0.0.0 comes from in this case. If it doesn't nat I'd expect to see 127.0.0.1 as source.
Maybe you can override it with a custom outbound NAT rule. But this shouldn't be necessary in normal circumstances.

What's also weird to me is the MAC 26:3c:94:fe:3c:b9 address at the public IP in the routing table.
I don't expect this to be assigned from Proxmox. Did you spoof it?
Otherwise I can't think of what this should tell us.

Maybe @stephenw10 has an idea, what's going on here and can help.

coolsaet

@viragomann said in Firewall itself has no internet connection:

@coolsaet
Pretty strange. I've sadly no idea, where the 0.0.0.0 comes from in this case. If it doesn't nat I'd expect to see 127.0.0.1 as source.
Maybe you can override it with a custom outbound NAT rule. But this shouldn't be necessary in normal circumstances.

Should I than put a rule in the outbound NAT with source 0.0.0.0? Wouldn't that create other problems?

What's also weird to me is the MAC 26:3c:94:fe:3c:b9 address at the public IP in the routing table.
I don't expect this to be assigned from Proxmox. Did you spoof it?

It is assigned by Proxmox. Apparently, it gives completely random MAC addresses when no prefix is specified.

viragomann

@coolsaet said in Firewall itself has no internet connection:

Should I than put a rule in the outbound NAT with source 0.0.0.0? Wouldn't that create other problems?

That's a very good question. It brings me to another idea.
Normally it should be 127.0.0.1, the loopback address. But in your case it seems to be 0.0.0.0, which may indicate, that there is no IP assigned to the loopback interface for whatever reason.

Run

ifconfig -a

on pfSense to show the interface IP settings. Especially the lo0, but possibly there is some more odd with the interfaces, so best to see the whole output.

coolsaet

@viragomann
That was my idea too. But sadly everything looks normal. Or maybe I am missing something?

vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: WAN
	options=800b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
	ether 26:3c:94:fe:3c:b9
	inet6 fe80::243c:94ff:fefe:3cb9%vtnet0 prefixlen 64 scopeid 0x1
	inet 85.27.8.241 netmask 0xffffffe0 broadcast 85.27.8.255
	inet 85.27.8.229 netmask 0xffffffe0 broadcast 85.27.8.255
	inet 85.27.8.233 netmask 0xffffffe0 broadcast 85.27.8.255
	inet 85.27.8.234 netmask 0xffffffe0 broadcast 85.27.8.255
	media: Ethernet 10Gbase-T <full-duplex>
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vtnet1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1420
	description: LAN_DATA
	options=800b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
	ether 5e:3d:cd:15:e1:00
	inet6 fe80::5c3d:cdff:fe15:e100%vtnet1 prefixlen 64 scopeid 0x2
	inet 172.21.0.1 netmask 0xffff0000 broadcast 172.21.255.255
	media: Ethernet 10Gbase-T <full-duplex>
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vtnet2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=800b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
	ether ee:9c:d3:80:80:6a
	media: Ethernet 10Gbase-T <full-duplex>
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vtnet3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: CUSTOMER_VPN
	options=800b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
	ether 26:e2:70:22:2d:37
	inet6 fe80::24e2:70ff:fe22:2d37%vtnet3 prefixlen 64 scopeid 0x4
	inet 172.31.42.1 netmask 0xffffff00 broadcast 172.31.42.255
	media: Ethernet 10Gbase-T <full-duplex>
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vtnet4: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: LAN_SERVICES
	options=800b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
	ether c6:a6:54:fa:4f:06
	inet6 fe80::c4a6:54ff:fefa:4f06%vtnet4 prefixlen 64 scopeid 0x5
	inet 10.99.0.1 netmask 0xffff0000 broadcast 10.99.255.255
	media: Ethernet 10Gbase-T <full-duplex>
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vtnet5: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: VAULT_VPN
	options=800b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
	ether 42:cd:41:82:9d:1e
	inet6 fe80::40cd:41ff:fe82:9d1e%vtnet5 prefixlen 64 scopeid 0x6
	inet 172.31.43.1 netmask 0xffffff00 broadcast 172.31.43.255
	media: Ethernet 10Gbase-T <full-duplex>
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vtnet6: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: VAULT_SERVERS
	options=800b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
	ether 4e:64:ab:25:a9:ae
	inet6 fe80::4c64:abff:fe25:a9ae%vtnet6 prefixlen 64 scopeid 0x7
	inet 172.30.0.1 netmask 0xffff0000 broadcast 172.30.255.255
	media: Ethernet 10Gbase-T <full-duplex>
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
enc0: flags=0<> metric 0 mtu 1536
	groups: enc
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x9
	inet 127.0.0.1 netmask 0xff000000
	groups: lo
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=100<PROMISC> metric 0 mtu 33160
	groups: pflog
pfsync0: flags=0<> metric 0 mtu 1500
	groups: pfsync
tun_wg0: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1420
	description: VIRT_VPN
	options=80000<LINKSTATE>
	inet 10.66.66.1 netmask 0xffffff00
	groups: wg WireGuard
	nd6 options=101<PERFORMNUD,NO_DAD>

vnet2 is disabled in the interfaces section.

viragomann

@coolsaet
No, it seems all well.

You can try it with a custom outbound NAT rule for 0.0.0.0/32, translating it to the WAN IP.
But to be honest, I would rather tear the VM down and start from scratch with a new installation.

coolsaet

@viragomann said in Firewall itself has no internet connection:

You can try it with a custom outbound NAT rule for 0.0.0.0/32, translating it to the WAN IP.

That does seem to work.

But to be honest, I would rather tear the VM down and start from scratch with a new installation.

I agree. NATting 0.0.0.0 doesn't feel right. I already tried reinstalling pfsense and restoring a backup of the config, but without success. I will be starting over from scratch. It will also give me a chance to improve some things.

viragomann

@coolsaet said in Firewall itself has no internet connection:

I agree. NATting 0.0.0.0 doesn't feel right. I already tried reinstalling pfsense and restoring a backup of the config, but without success. I will be starting over from scratch. It will also give me a chance to improve some things.

Yes, I expect that it's somewhere in the config and you would have the same issue, after import it from a backup.

Maybe you can find a hint in the config to whatS' wrong, but 0.0.0.0 generally indicates an unknown IP and as seen in the interface settings, any interface which is up has an IPv4 configuration. So I'm running out of ideas.

coolsaet

@viragomann
Finally got around to reinstalling and reconfiguring from a clean install.

I am happy to say that it solved the problem.