[SOLVED / FEATURE REQUEST] GUI gone with default route set but works without
-
Hi,
I've got an issue with pfsense running in an openstack enviroment.
Currently the openstack network ist set up as followed:
Hoster-GW 192.168.1.1<-> OS external network with floating IP 192.168.1.2 <-> OS dmz network with 10.40.0.0/24 <-> pfSense WAN NIC <->pfSense LAN NIC <-> internal network 10.40.50.0/24
The Firewall has the 10.40.0.3 set in the dmz network as well as the 10.40.50.3 on the lan network.
WAN Gateway is set to 10.40.0.1 witch routes to the external network 192.168.1.0/24.Now, when i set this IP Configuration in the pfsense console setup and add the allowallonwan rule using this php playback stuff, I can ping the firewall vut not access the gui.
Once I delete the default route, I can access the GUI. But only as long as i reboot the firewall or set the gateway 10.40.0.1 as default again.
So far I've tried:
-
Set default gateway to 192.168.1.1 on the wan interface (abnormal gateway option in pfsense)
-
add static route in openstack dmz-subnet for 0.0.0.0/0 and nexthop 10.40.0.1
-
tried the same as a static route in pfsense
When I cant access the GUI but use the vnc console of OS, I can see packets coming from the wan ip hitting the 10.40.0.3, for example from 1.2.3.4
Also I see pfsense responding with packets from 10.40.0.3 to 1.2.3.4. But those packets, when having a default route, seem to get stuck in Openstack right at the 10.40.0.1 gateway.
But without the default route set, still a route tells pfsense which interface servers which subnet. So vtnet1 has the 10.40.0.0 and vtnet0 the 10.40.50.0.
Why does the default route mess everything up? This seems to be an issue with Openstack doing DNAT for exposed IPs.Unfortunately I can't attach the pfsense directly to the external network by design of OS. So any help would be appreciated on this issue.
- netcore
EDIT: Allthough I disabled port security already. With the default gateway set suddenly tcpdump shows packets leaving the pfsense interface having the same tcp sequence number allover again.
-
-
OK, I finally found out what happend and guess what: It wasn't actually the default route. I just thought so because the Problem occurred after the firewall was up for some minutes, even when leaving the default route unset.
I took a look at the packet leaving my OS controlnode using tcpdump and stumbled upon checksum error:
xxx.xxx.xxx.xxx23915 > static.xxx.xxx.xxx.xxx.clients.your-server.de.https: Flags [P.], cksum 0xe00b (incorrect -> 0x70a8), seq 2591:3192, ack 32823, win 257, length 601
I looked up the MTU of my controller which was set to 1500. Also the PFSense operates at the same MTU.
Now, normally in a non-virtualized Network, this seems to work finde since the switches may be able to adjust their MTU or have bigger frames anyway but Openstack just runs at 1500, resulting in fragmented packets.
So I set the MTU to 1400 in PFSense and voila! No more GUI issues even after reboot. ;D -
A Feature-Request for PfSense:
I jut configured Openstack to deploy an mtu using DHCP Option 26, but PFsense seems to ignore this one consequently.
dhcpdump -i tapb8659f7c-df
TIME: 2017-06-29 13:22:00.059
IP: 10.40.50.3 (fa:16:3e:2c:e0:61) > 255.255.255.255 (ff:ff:ff:ff:ff:ff)
OP: 1 (BOOTPREQUEST)
HTYPE: 1 (Ethernet)
HLEN: 6
HOPS: 0
XID: 11ac2ce2
SECS: 0
FLAGS: 0
CIADDR: 0.0.0.0
YIADDR: 0.0.0.0
SIADDR: 0.0.0.0
GIADDR: 0.0.0.0
CHADDR: fa:16:3e:2c:e0:61:00:00:00:00:00:00:00:00:00:00
SNAME: .
FNAME: .
OPTION: 53 ( 1) DHCP message type 3 (DHCPREQUEST)
OPTION: 50 ( 4) Request IP address 10.40.50.3
OPTION: 61 ( 7) Client-identifier 01:fa:16:3e:2c:e0:61
OPTION: 12 ( 9) Host name xxxxxxxxx
OPTION: 55 ( 9) Parameter Request List 1 (Subnet mask)
28 (Broadcast address)
2 (Time offset)
121 (Classless Static Route)
3 (Routers)
15 (Domainname)
6 (DNS server)
12 (Host name)
119 (Domain Search)–-------------------------------------------------------------------------
TIME: 2017-06-29 13:22:00.059
IP: 10.40.50.2 (fa:16:3e:63:19:c0) > 10.40.50.3 (fa:16:3e:2c:e0:61)
OP: 2 (BOOTPREPLY)
HTYPE: 1 (Ethernet)
HLEN: 6
HOPS: 0
XID: 11ac2ce2
SECS: 0
FLAGS: 0
CIADDR: 0.0.0.0
YIADDR: 10.40.50.3
SIADDR: 10.40.50.2
GIADDR: 0.0.0.0
CHADDR: fa:16:3e:2c:e0:61:00:00:00:00:00:00:00:00:00:00
SNAME: .
FNAME: .
OPTION: 53 ( 1) DHCP message type 5 (DHCPACK)
OPTION: 54 ( 4) Server identifier 10.40.50.2
OPTION: 51 ( 4) IP address leasetime 86400 (24h)
OPTION: 58 ( 4) T1 43200 (12h)
OPTION: 59 ( 4) T2 75600 (21h)
OPTION: 1 ( 4) Subnet mask 255.255.255.0
OPTION: 28 ( 4) Broadcast address 10.40.50.255
OPTION: 15 ( 14) Domainname openstacklocal
OPTION: 12 ( 15) Host name host-10-40-50-3
OPTION: 3 ( 4) Routers 10.40.50.3
OPTION: 121 ( 14) Classless Static Route 20a9fea9fe0a2832 .....(2
02000a283203 ...(2.
OPTION: 6 ( 8) DNS server xxxxxxxxxxxxx
OPTION: 26 ( 2) Interface MTU 1450
–-------------------------------------------------------------------------This shouldn't be ignored because it'll result in fragmented packets / incorrect checksums since OS itself adds ~50 Bytes to a VXLAN-paket anyway.