Problem routing traffic from dedicated Public IP Block (/29) on OPT-2 Int in/out of WAN port assigned IP address via DHCP from provider network.

[popquiz]

• • • U R G E N T H E L P N E E D E D * * *

I am having problems getting IP traffic to flow. Followed "Routing Public IP Addresses" in the guide so I have a good rule under outbound NAT that signifies the traffic for that network is not NATted.

Let me provide some details (IP Addresses altered for security reasons):
WAN interface: 8.240.122.133 /22
WAN Gateway: 8.240.120.1 mask 255.255.252.0
LAN Interface: 10.100.100.1 /24
OPT1 Interface: 10.200.200.1 /24
OPT2 Interface: 204.150.150.145 /29 Network = 204.150.150.144 /29

Trying to route from hosts on 204.150.150.144 /29 network over WAN to no avail. Can ping hosts from pfSense firewall on OPT2 interface but hosts on LAN interface cannot ping hosts on 204.150.150.144 /29 network on directly connected interface OPT2.

To simplify things, I have put the LAN subnet on one simple L2 switch (Netgear GS108) and the OPT2 Interface on a separate simple L2 switch (Netgear GS108). Of course the WAN port is directly connected to my ONT Ethernet Interface.

Conditions present:
Can ICMP ping from pfSense firewall to any device on LAN network interface including the interface.
Can ICMP ping from pfSense firewall to most devices on OPT2 network interface including the .145 used as the OPT2 interface IP address
Cannot ping from devices on the LAN interface to devices on the OPT2 interface, with the exception of the OPT2 interface itself 204.150.150.145.

I have left OPT1 out of the equation in most regards as it is irrelevant to the discussion. If I can get things to work between OPT2 and LAN and/or OPT2 and WAN I should be able to copy configuration details between the LAN and OPT1 interfaces except change the IP addresses to make that work properly.

Another note: I created a rule under "firewall/rules/OPT2 action allow for IPv4 Protocol ICMP ANY, source: ANY, Destination ANY, rule enabled. Second rule: under "firewall/rules/OPT2 action BLOCK, for IPv4 protocol ANY, source: ANY, destination: ANY So this should only allow ICMP traffic to this network.

Could use some helpful suggestions on why routing is not working. Do I need to set a default gateway up for the OPT2 network interface to route packets to the Internet out the WAN interface? Do I need to set up a static route between any of these interfaces to get routing to work? Is there a global IP routing command that I must enable to allow for routing between directly connected networks?

This should be simple routing 101 but it has got me scratching my head. Is the firewall somehow interfering with the routing?

Please help. All useful comments suggestions appreciated.

[johnpoz]

@popquiz said in Problem routing traffic from dedicated Public IP Block (/29) on OPT-2 Int in/out of WAN port assigned IP address via DHCP from provider network.:

I created a rule under "firewall/rules/OPT2 action allow for IPv4 Protocol ICMP ANY, source: ANY, Destination ANY, rule enabled. Second rule: under "firewall/rules/OPT2 action BLOCK, for IPv4 protocol ANY, source: ANY, destination: ANY So this should only allow ICMP traffic to this network.

Huh.. NO that would block traffic FROM that network for anything other than icmp. So yeah that would clearly block this interface from getting to the internet..

Rules are placed on the interface where traffic would enter pfsense from the attached network..

You could have no rules on opt2, and if lan default any any rule is there it would be able to ping anything on the opt2 network.. Return traffic would be allowed by the state.

If you can not ping a device on opt2 from lan, and your lan rules allow it - the problem is most likely firewall on the opt2 device.. Most hosts running local firewall would not allow ping from something other than its local network by default..

[popquiz]

@johnpoz I understand your reply to my rule, but because I have a computer, a Synology NAS, and two servers on this network OPT2 I have only allowed ICMP traffic for the time being until I can l can prove this is working properly and replying to ICMP Pings should populate ARP tables and MAC address tables of switches as well as any interface on the pfSense firewall appliance.

So for the sake of my sanity, I attached two raspberryPi devices each running Debian on each local interface:
on LAN interface igp1 = 10.100.100.101 /24 gayeway = 10.100.100.1 , on OPT2 interface igp3 = 204.150.150.149 /29 gateway = 204.150.150.145

I still cannot ping from LAN to OPT2 eg., from 10.100.100.101 command ping 204.150.150.149
Pinging 204.150.150.149 with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.

ICMP traffic should be able to flow both ways. Where am I wrong?

I do appreciate your prompt response and it did have me thinking that I can simplify things easily by just removing all rules except the NAT rules and see what happens, but this should work... Yes/No???

[johnpoz]

@popquiz said in Problem routing traffic from dedicated Public IP Block (/29) on OPT-2 Int in/out of WAN port assigned IP address via DHCP from provider network.:

all rules except the NAT rules

what nat rules - if you have a public network on your lan side what would you be natting it too?

I still cannot ping from LAN to OPT2 eg., from 10.100.100.101 command ping 204.150.150.149

What are you rules on your LAN - are you policy routing traffic out a gateway?

[popquiz]

@johnpoz This is a home network that is supplied by Service Provider FTTH on the WAN port (igb0) ip address 8.240.122.133 /22 this service is being NAT translated for hosts on the LAN port (igb1) 10.100.100.1 /24. I am not NATting the hosts on OPT2 (igb3) interface 204.150.150.145 /29.

This is done by setting up a hybrid NAT rule, see url: https://docs.netgate.com/pfsense/en/latest/recipes/route-public-ip-addresses.html#figure-wan-ip-and-gateway-configuration

This config allows the Public IP address space on the OPT2 interface to traverse the WAN port without being NAT translated. It does require rules be set up to allow specific traffic.

Essentially, I am avoiding having to rely on DDNS service for separate VM instances running over Private IPs and NAT traversal. This is essentially setting up a DMZ with a Public IP subnet on an interface behind a WAN connection, very similar as would be served if the WAN was a statically assigned IP with a /30 subnet. And yes I get it, why NAT? I still need NAT for the LAN hosts on the LAN interface, so there has to be a rule on the WAN interface for that service and the Hybrid rule to allow for non-NATted traffic on the DMZ using public ip addresses..

See diagram here: https://www.dropbox.com/s/u5xwuo0w5koc6as/my%20home%20network.pdf?dl=0

Again, thanks for your reply! I do appreciate the consideration you are giving this problem.

[johnpoz]

@popquiz said in Problem routing traffic from dedicated Public IP Block (/29) on OPT-2 Int in/out of WAN port assigned IP address via DHCP from provider network.:

This is done by setting up a hybrid NAT rule

I know how its done ;)

Doesn't matter what address you put on opt2 network, rfc1918, public that is actually yours or one you pulled out of thin air..

That has nothing to do with devices on lan talking to this opt2 network - no matter if the range is public or rfc1918.

You need ZERO rules on this opt2 interface to be able to talk to it from lan, and nat rules would not come into play be it your natting this network or not to the internet.

Please post up your lan rules, and do you have any rules in floating.

edit: Upon looking at your diagram, what is a bit confusing is why your using the term gateway

These interfaces on pfsense would not have a gateway set.. Your lan interface on pfsense 10.100.100.1/24, there is no gateway set.. Your devices sure on your lan would point to 10.100.100.1 as their gateway.. But the actual interface on pfsense should have no gateway set.

Same goes for your public IP on opt2, 204.150.150.145/29 would be pfsense IP.. But it would also not have any gateway set.. Only devices on this opt2 network would use the .145 as their gateway.

So your use of this term concerns me that you have set that on the actual interfaces of pfsense.

Doing so would tell pfsense to use those interfaces as "wan" interfaces.

[popquiz]

@johnpoz Thanks again for your reply. I use the term "gateway" as a synomonous term for the interface on the hardware interface for that network. So in general terms I use the following as "gateway" expressions for the following networks:
WAN interface (IGB0): 8.240.122.133 /22
LAN Interface (IGB1): 10.100.100.1 /24
OPT1 Interface (IGB2): 10.200.200.1 /24
OPT2 Interface (IGB3): 204.150.150.145 /29 Network = 204.150.150.144 /29

The only path to the Internet in this scenario is through the IGB0 interface/WAN port. And I agree with you. The LAN and OPT1 interfaces have nothing to do with routing the public IP addresses on interface IGB3/OPT2. So in effect there is only one gateway on this device and that is on the attached Network connection of the WAN Port or the provider end of that connection. In this case that would be 8.240.120.1 /22. Sorry for the mix up, old habits die hard and I've been doing mostly Cisco networks and devices since 1992.

I guess a simple enough test that I could run would be to undo the NAT Outbound rules e.g., Hybrid Rule for 204.150.150.144 /29 applied on WAN interface and the two automatic rules created for ISAKMP and the 127.0.0.0/8 ::1/128 10.100.100.0/24 ANY/ANY ANY/ANY.

The only firewall rules:

I would try to put up a more thorough output, but being rather new to the pfSense family, I am still searching for a "show tech" type command that I can run to give the complete status of the system.

[popquiz]

@johnpoz I have spent nearly the last three hours reading threads from this group and I can see that you have put in a lot of time and effort into helping people with traffic routing and gateway issues. I apologize if my ignorance masked itself as arrogance in any of our conversation. your body of work here certainly shows your competence and generosity and I failed to acknowledge that before posting and throughout this continued dialog. Please accept my sincere apology for any oversteps on my part.

[johnpoz]

@popquiz so with your any rule on lan.. There should be no reason you should not be able to talk to anything on opt2 no matter what the IP is.

Your saying from device on lan you can ping opt2 pfsense address 204.150.150.145?

But you can not ping a device on this network, say .146

Can pfsense itself ping this .146 address?

If so I would suggest a sniff - from your lan device get a ping going to the .146 address, do you see pfsense sending that out on opt2?

You can do a simple packet capture under the diagnostic menu, on the opt2 interface. If you see traffic going out the opt2 interface.. But no response - that points to this opt2 device not pointing to pfsense .145 address as its gateway. Or it is running a firewall.