PfSense won't forward traffic form LAN server to internet
-
★ My setup:
Host (ArchLinux, nanoBox):
Physical interfaces: with eth0 (no ip) and wlan0 (hostapd).
Virtual interfaces: br0 (static IP 192.168.7.2 assigned with netctl profile)Guest (pfSense inside KVM):
Guest interfaces:
vtnet0 - bridged to eth0
vtnet1 - bridged to br0 (192.168.7.1)My Host is also a web sever. I do not know if this is good practice, but br0 is the interface which which host services connect to internet.
I assign static IP to br0 for two reasons:
1 - when I set it to DHCP it does not receive any IP. I have no idea why - it worked for a month before, but it just doesn’t anymore.
2 - it allows me to access the host even when pfSense VM is down★ What works:
- accessing internet from Host
- accessing internet from WiFi clients connecting to the internet using hostapd
- port forwarding from the internet to WiFi clients connecting to the internet using hostapd
★★★ What does not work: ★★★
- port forwarding from the internet to the Host! This means that any service running on Host (http, ssh, VNC etc.) is not accessible from the Internet.
I have no idea why, because:
➤ All the NAT and Firewall rules on pfSense are set up and identical (config restore) to when they worked on VirtualBox setup. NAT rules: http://i.imgur.com/jqzqYYu.png Firewall rules: http://i.imgur.com/pfuuwAz.png
(nanoBox alias is assigned to 192.168.7.2 ip. Replacing alias with the ip itself makes no difference)
➤ The static IP of br0 is the same as the IP of the destination host in every IP rule.
➤ Host can access internet.
➤ Host can be pinged on its IP (192.168.7.2) from pfSense
➤ NAT and Firewall rules worked for any other internal IP (for example, a VNC server on one of WiFi clients)★ Configs
Host's ifconfig
# ifconfig br0: flags=4163<up,broadcast,running,multicast> mtu 1500 inet 192.168.7.2 netmask 255.255.255.0 broadcast 192.168.7.255 inet6 fe80::4ccb:a9ff:feb7:5617 prefixlen 64 scopeid 0x20 ether a0:88:69:0d:5c:41 txqueuelen 0 (Ethernet) RX packets 2825 bytes 330247 (322.5 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 3339 bytes 802554 (783.7 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 enp2s0: flags=4163<up,broadcast,running,multicast> mtu 1500 inet6 fe80::5ea1:75a3:7d46:befd prefixlen 64 scopeid 0x20 ether 00:90:27:77:fb:02 txqueuelen 1000 (Ethernet) RX packets 223027 bytes 20719723 (19.7 MiB) RX errors 0 dropped 178 overruns 0 frame 0 TX packets 6747 bytes 2101069 (2.0 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<up,loopback,running> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10 <host>loop txqueuelen 0 (Local Loopback) RX packets 12388 bytes 1341938 (1.2 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 12388 bytes 1341938 (1.2 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 macvtap0: flags=4163<up,broadcast,running,multicast> mtu 1500 inet6 fe80::26f4:1e55:97a0:c0cb prefixlen 64 scopeid 0x20 ether 00:90:27:77:fb:02 txqueuelen 500 (Ethernet) RX packets 217268 bytes 20328935 (19.3 MiB) RX errors 8919 dropped 8919 overruns 0 frame 0 TX packets 6620 bytes 2073711 (1.9 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 vnet0: flags=4163<up,broadcast,running,multicast> mtu 1500 inet6 fe80::5d6b:398c:6b44:d602 prefixlen 64 scopeid 0x20 ether fe:54:00:6f:2e:15 txqueuelen 500 (Ethernet) RX packets 4558 bytes 4062075 (3.8 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 4583 bytes 624983 (610.3 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 wlp1s0: flags=4163<up,broadcast,running,multicast> mtu 1500 inet6 fe80::6e57:fe92:1321:1521 prefixlen 64 scopeid 0x20 ether a0:88:69:0d:5c:41 txqueuelen 1000 (Ethernet) RX packets 6040 bytes 811010 (792.0 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 7038 bytes 4986969 (4.7 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0</up,broadcast,running,multicast></up,broadcast,running,multicast></up,broadcast,running,multicast></host></up,loopback,running></up,broadcast,running,multicast></up,broadcast,running,multicast>
/etc/netctl/br
Interface=br0 Connection=bridge BindsToInterfaces=() #IP=dhcp IP=static Address=('192.168.7.2/24') Routes=('192.168.0.0/24 via 192.168.7.1') Gateway=('192.168.7.1') ## Ignore (R)STP and immediately activate the bridge #SkipForwardingDelay=yes
★ Notes that may or may not be relevant:
-
When KVM starts, it creates an apparently useless vnet0 interface on host. I say it is useless because when I was using VirtualBox in identical setup, it did not create anything beyond the interfaces listed at the beginning of this post, and things worked fine. vnet0 remains IP-less. However, if I destroy br0, vnet0 immediately receives a DHCP IP from pfSense guest.
-
when I look at DHCP logs on pfSense, I can see repetitive entries:
Apr 4 08:12:03 dhcpd: DHCPDISCOVER from 4e:cb:a9:b7:56:17 via vtnet1 Apr 4 08:12:03 dhcpd: unexpected ICMP Echo Reply from 85.28.128.1 Apr 4 08:12:04 dhcpd: DHCPOFFER on 192.168.7.102 to 4e:cb:a9:b7:56:17 (nanoBox) via vtnet1 Apr 4 08:12:52 dhcpd: DHCPDISCOVER from fe:54:00:6f:2e:15 (nanoBox) via vtnet1 Apr 4 08:12:52 dhcpd: unexpected ICMP Echo Reply from 85.28.128.1 Apr 4 08:12:53 dhcpd: DHCPOFFER on 192.168.7.100 to fe:54:00:6f:2e:15 (nanoBox) via vtnet1 Apr 4 08:13:06 dhcpd: DHCPDISCOVER from 4e:cb:a9:b7:56:17 (nanoBox) via vtnet1 Apr 4 08:13:07 dhcpd: unexpected ICMP Echo Reply from 85.28.128.1 Apr 4 08:13:07 dhcpd: DHCPOFFER on 192.168.7.102 to 4e:cb:a9:b7:56:17 (nanoBox) via vtnet1 Apr 4 08:13:56 dhcpd: DHCPDISCOVER from fe:54:00:6f:2e:15 (nanoBox) via vtnet1 Apr 4 08:13:56 dhcpd: unexpected ICMP Echo Reply from 85.28.128.1 Apr 4 08:13:57 dhcpd: DHCPOFFER on 192.168.7.100 to fe:54:00:6f:2e:15 (nanoBox) via vtnet1 Apr 4 08:14:10 dhcpd: DHCPDISCOVER from 4e:cb:a9:b7:56:17 (nanoBox) via vtnet1 Apr 4 08:14:11 dhcpd: unexpected ICMP Echo Reply from 85.28.128.1 Apr 4 08:14:12 dhcpd: DHCPOFFER on 192.168.7.102 to 4e:cb:a9:b7:56:17 (nanoBox) via vtnet1 Apr 4 08:15:00 dhcpd: DHCPDISCOVER from fe:54:00:6f:2e:15 (nanoBox) via vtnet1 Apr 4 08:15:01 dhcpd: unexpected ICMP Echo Reply from 85.28.128.1 Apr 4 08:15:01 dhcpd: DHCPOFFER on 192.168.7.100 to fe:54:00:6f:2e:15 (nanoBox) via vtnet1 Apr 4 08:15:15 dhcpd: DHCPDISCOVER from 4e:cb:a9:b7:56:17 (nanoBox) via vtnet1 Apr 4 08:15:16 dhcpd: unexpected ICMP Echo Reply from 85.28.128.1 Apr 4 08:15:16 dhcpd: DHCPOFFER on 192.168.7.102 to 4e:cb:a9:b7:56:17 (nanoBox) via vtnet1
★ Things I tried:
- disabling DHCP server on pfSense
- changing host's static IP
- in Firewall/NAT ruels, replacing alias nanoBox with the ip itself
- many other things I can't remember
-
Could you try to take a screenshot?
You are saying you can't port forward from the internet to the host, but it should be the other way arround so you are port forwarding your server to the wan interface :)
-
Could you try to take a screenshot?
Yes, but of what?
You are saying you can't port forward from the internet to the host, but it should be the other way arround so you are port forwarding your server to the wan interface :)
Well, I need my server on the LAN to be accessible from WAN. Sorry for the confusion.
-
When you look at the 80 NAT rule, you do NOT have a firewall rule associated with that. WHY?
-
Check your firewall logs.
-
@doktornotor,
Good point. I set it up,
http://i.imgur.com/0LrJCzv.png
but the problem persists.@EMWEE,
In the firewall logs, I can see nothing relevent, ie related to port :80, or the external IP I am trying to access the webserver from. -
Move it above the block all rule and then its fine.
-
Move it above the block all rule and then its fine.
I don't think I have a block all rule. And if you mean "RFC 1918 networks" and "Reserved/not assigned by IANA", then it is not possible.
-
Do you have an outbound nat rule in place that seperate subnet?
-
Do you have an outbound nat rule in place that seperate subnet?
http://i.imgur.com/L5FrSeb.png
Is that it? -
Move it above the block all rule and then its fine.
I don't think I have a block all rule. And if you mean "RFC 1918 networks" and "Reserved/not assigned by IANA", then it is not possible.
You can try unchecking the box “Block private networks” on the screen Interfaces WAN (at the bottom) and see if that solves your problem.
-
You can try unchecking the box “Block private networks” on the screen Interfaces WAN (at the bottom) and see if that solves your problem.
That's one of those things I tried and forgot to mention :(
-
Perhaps you could finally look at the firewall logs?!?!?
-
Nope…
It looks like this...
Menu -> Firewall -> NAT -> Outbound
-
Perhaps you could finally look at the firewall logs?!?!?
What am I looking for? I already said I could see nothing related to the IP I am accessing the web server from.
Nope…
It looks like this...
Menu -> Firewall -> NAT -> Outbound
That's what I have:
http://i.imgur.com/fWWY3XA.png -
You can try unchecking the box “Block private networks” on the screen Interfaces WAN (at the bottom) and see if that solves your problem.
That's one of those things I tried and forgot to mention :(
But the screen shot of the firewall rules shows Block private networks is in effect. I am confused.
-
But the screen shot of the firewall rules shows Block private networks is in effect. I am confused.
Because I re-enabled it after I found out it did not change anything. Anyway, now I have it disabled again.
-
★ My setup:
Host (ArchLinux, nanoBox):
Physical interfaces: with eth0 (no ip) and wlan0 (hostapd).
Virtual interfaces: br0 (static IP 192.168.7.2 assigned with netctl profile)Guest (pfSense inside KVM):
Guest interfaces:
vtnet0 - bridged to eth0
vtnet1 - bridged to br0 (192.168.7.1)My Host is also a web sever. I do not know if this is good practice, but br0 is the interface which which host services connect to internet.
Does this mean pfsense WAN interface is assigned to vtnet1, has a static ip of 192.168.7.1 and pfsense LAN interface is assigned to vtnet0?
-
Does this mean pfsense WAN interface is assigned to vtnet1, has a static ip of 192.168.7.1 and pfsense LAN interface is assigned to vtnet0?
No, the other way around:
eth0-WAN-85.x.x.x-vtnet0
br0-LAN-192.168.7.1-vtnet1 -
Does this mean pfsense WAN interface is assigned to vtnet1, has a static ip of 192.168.7.1 and pfsense LAN interface is assigned to vtnet0?
No, the other way around:
eth0-WAN-85.x.x.x-vtnet0
br0-LAN-192.168.7.1-vtnet1Does 192.168.7.2 have 192.168.7.1 as gateway? I am guessing not because it has internet with pfsense in shutdown.