Weird routing issue

rolandk

i made some other weird observation:

normally, we have added our routing via an ALIAS of network definitions, i.e. ALIASNAME - SUB1/MASK, SUB2/MASK, SUB3/MASK

if i add another network to the ALIASNAME and add that in system->routing->ROUTES-TAB , then the newly added network does not get added.
I need to explicitly add the network in the ROUTES-TAB, though other routing from the alias works.

Maybe i need to flush/reload something elserwere so i can use ALIAS Names for routing?

johnpoz

"Using pfsense in 2 locations which are interconnected via dedicated network interface"

This how you would normally do it..

So in your site 1 you would have routes

172.18.28.0/23 172.16.40.2
192.168.1.0/24 172.16.40.2

in site 2 you would have

172.16.26.0/23 172.16.40.1

Do have gateway to the other side of the transit in each site?

What are your rules on your interfaces. If your forcing traffic out a gateway say your internet gateway before you allow traffic using the normal pfsense routing then yeah your going to have problems.

2sitesviatransit.png_thumb

rolandk

yes, it`s like you visualized!

we have a rule on the 172.16.26.0 lan which does policy based routing into WAN/Internet for ANY destination, but what i don´t understand is why for target 192.168.1.0/24 packets being correctly routed via the interconnect and for target 172.16.28.0/23 they are being routed via WAN/Internet - imho they should go the same way and that is what i don`t understand.

if we add extra-rule for 172.18.28.0 to do policy based routing via Interconnect, it works - but why don`t we need that rule for 192.168.1.0, too ?

johnpoz

dude if you want helping finding what is wrong in you rules - post them..

rolandk

here we go

Rules for LAN interface on network 172.16.26.0/23

SERVERLAN net (172.16.28.0/23)
MGMTLAN net (172.16.37.0/24)
VMLAN net (172.16.30.0/23)
DMZ_TIER2 net (172.16.36.0/24)
DMZ_UM_EXT net (public ip subnet, ip/subnet not shown here because of data protection)

IPv4+6 * LAN net * This Firewall * * none Pass any to this Firewall
IPv4 * LAN net * SERVERLAN net * * none LAN to SERVERLAN
IPv4 * LAN net * MGMTLAN net * * none LAN to MGMTLAN
IPv4 * LAN net * VMLAN net * * none LAN to VMLAN
IPv4 * LAN net * 192.168.81.0/24 * * none LAN to OpenVPN
IPv4 * LAN net * DMZ_UM_EXT net * * none LAN to DMZ1
IPv4 * LAN net * DMZ_TIER2 net * * none LAN to DMZ2
IPv4 * LAN net * * * WAN_UM none everything else to internet

johnpoz

is a screenshot really that hard??

I don't see any rules that allow any traffic to the other sites 192.168.1/24 or 172.18.28/23

is serverlan really 172.18.28??

This is what you said this network was
"172.18.28.0/23 172.16.40.1 UGS igb2"

From those rules I don't see how you could get across to anything..

rolandk

sorry, there is some infos in the description fields i do not want to have on the internet and im new to this forum so it didnt come to my mind.

yes, from the ruleset i should not be able to connect to 192.168.0.x IP adresses, but i CAN.

i can login via ssh to 192.168.1.50 for example and on the remote system, i see client`s ip 172.16.27.45 in netstat. so no nat or anything else in place.

i don`t understand how this is possible as there is no rule which would allow or policyroute this.

johnpoz

Well if wan_um gateway can get there then you could get there.

Do you have any floating rules?

What other routes do you have? When you do a traceroute to this 192.168.1.50 from your client 172.16.27.45 what do you show?

Have seen users have any any in their floating and then wonder why stuff is working even though they have a block on the interface ;)

rolandk

no floating rules in place.

i investigated further and apparently the thing is all about "negate_networks"

https://forum.pfsense.org/index.php?topic=66776.45

i can see with "pfctl -T show -t negate_networks that it contains 192.168.1.0 (and others) but not 172.18.28.0. the question is , why.

will read into it further.

thanks for help so far

Derelict

Because 192.168.1.0/24 is defined in a VPN somewhere, most likely.

You need to bypass policy routing for the 172.18.28.0/23 subnet.

Your problem is not routing in general, it is that you are policy routing out the WAN_UM gateway, which means everything not explicitly exempted in the rules above gets shoved that way without regard to the routing table. Why are you doing that? Is WAN_UM not the default gateway?

rolandk

yes, 192.168.1.0 is also defined in a deactivated ipsec tunnel definition - apparently thats the reason why it exists in negate_networks, though - and thats the reason why 192.168.1.0 is (by chance) being routed the correct way and 172.18.28.0 not

johnpoz

Dude as Derelict said and I stated in post 1 you need to allow rule above your rule that shoves everything down that gateway..
"If your forcing traffic out a gateway say your internet gateway before you allow traffic using the normal pfsense routing then yeah your going to have problems."

rolandk

yes, i know.

but i was more curious why 192.168.1.0 was working THOUGH (i.e. without explicit allow rule).

you should know how your firewall works and how things behave.
you should be able to explain things and do not wonder about miraculous firewall behaviour.
that kann kill your security.

johnpoz

Agreed, but since you have figured that out.. Now its time to correct your rules.

rolandk

Your problem is not routing in general, it is that you are policy routing out the WAN_UM gateway, which
means everything not explicitly exempted in the rules above gets shoved that way without regard to
the routing table. Why are you doing that? Is WAN_UM not the default gateway?

WAN_UM is a gateway group, you cant set that as a default gateway and you can only route to a gw-group via policy routing, cant you ?

johnpoz

if you force a gateway, be it default or a group or whatever.. You have to allow rules above that if you want your clients to talk to other networks off pfsense that are not reachable through that gateway your forcing traffic through. Is that simple!