Routing between Interfaces

Jerrid_C · Mar 7, 2014, 7:30 AM

I have searched and searched the forums and I cannot find an answer. I have seen multiple posts where it has been said that this should just work out of the box, but I am having issues. I have actually reset my pfsense back to factory default just to ensure I start fresh.

My configuration is a virtual system with 3 interfaces

em0 - WAN - 10.0.1.8/24 - Default Route with GW of 10.0.1.1
em1 - LAN - 192.168.1.1/24
em2 - DMZ - 192.168.13.1/24

The problem is I am not able to ping the DMZ subnet from the LAN, and vice versa I am not able to ping the LAN subnet from the DMZ. I have setup firewall rules on both LAN and DMZ to allow any to any for troubleshooting. What am I missing? It must be something right in front of me. Just to state this again, I reset the system and brought it back to a basic setup. Nothing else has been configured or changed beyond basic setup and the firewall rules.

phil.davis · Mar 7, 2014, 10:00 AM

Make sure LAN and DMZ do NOT have a gateway set. Make sure WAN is the default gateway. Otherwise pfSense might be mis-understanding the network topology when generating automatic outbound NAT rules.
Make sure your pass all rules do not specify a gateway - the traffic needs to pass to the ordinary routing table and be delivered locally.
Otherwise, post you interface settings and rules - yes, it works out of the box + adding rule to DMZ.

Jerrid_C · Mar 7, 2014, 4:37 PM

I confirmed everything, can't see the problem. Here are the interfaces and rules:

DMZ 1 Interface

LAN Interface

WAN Interface

Firewall Rules - WAN

Firewall Rules - LAN

Firewall Rules - DMZ

NAT Outbound

johnpoz · Mar 7, 2014, 7:34 PM

There is so many things wrong here its not even funny.

For starters - your wan rules.. How would you have source of lan and dmz into your WAN?

What is the point of a firewall if your going to any any rules inbound from wan? Your first rule in wan.

first rule in lan makes 2nd rule pointless, 3 rule is for ipv6 - but from your configs your not using IPv6, so pointless rule!

dmz rule looks fine if your not wanting to stop a dmz from going anywhere, which kind of really odd for a "dmz"

Your outbound nats show that your set for automatic - but you show rules? And that 1st rule listing is wrong.. Why would you want to nat your dmz talking lan so that it looks like comes from your wan IP?

I suggest you delete all those nat rules and leave on automatic. I would remove your wan rules that allow ALL traffic, Might as well be using a router not a firewall ;)

You can delete the 2 bottom lan rules.

Now on your dmz - can your clients ping your dmz IP 192.168.13.1, you sure your clients mask is set correctly - if they are on say 192.168.13.?/16 then they would think 192.168.1.0/24 is the same network. Same goes for your lan clients.

Do a traceroute from your clients to other network IP, can they ping the IP of the other pfsense IP in the other network.

So for example can client on 192.168.13.0/24 ping 192.168.1.1
Can client on 192.168.1.0/24 ping 192.168.13.1

After you have cleaned up, post your route table from pfsense if not working.

This really is click click automatic out of the box.. The only thing you need to do with a an out of the box configuration is create your allow rule for your 2nd segment.. Thats is - other than that out of the box works. I have 3 lan segments without any issues at all.

Jerrid_C · Mar 7, 2014, 8:33 PM

Well thanks for pointing it out…. I am learning and it has been a while since I have done networking. Crawl, walk, run ;)

As I stated, this is for testing so of course its not going to be "normal" in regards to rules. This is not a production environment, it is a lab. Right now I am trying to just get it to work as I want certain access from DMZ to LAN.

So I set it back to default again to just baseline the configuration. Also the rules you mention are default and the reason NAT rules show up is I was testing it and change the option to manual, and when changing it back to automatic, they don't go away.

DMZ server can ping 192.168.13.1 (its gateway) but it cannot ping 192.168.1.1 (LAN Gateway)

LAN server can ping 192.168.1.1 (its gateway) and it can ping 192.168.13.1 (DMZ gateway)

*I am using the term gateway in the sense its the IP Address on pfsense for that segment. If I am using the wrong terminology, I apologize ahead of time

DMZ or LAN cannot ping a server in the other zone, for instance DMZ server .13 cannot ping 192.168.1.11 and LAN server .11 cannot ping 192.168.13.10

Gateway

DMZ Int

LAN Int

WAN Int

NAT

WAN Firewall Rules

LAN Firewall Rules

DMZ Firewall Rules

johnpoz · Mar 7, 2014, 8:58 PM

"DMZ server can ping 192.168.13.1 (its gateway) but it cannot ping 192.168.1.1 (LAN Gateway)"

Well that makes no sense unless its not pointing to 192.168.13.1 as its gateway, or its mask on its own IP 192.168.13.? is wrong.

Can you post the clients network config. You say for example that lan can ping dmz pfsense IP. This is good, but can not ping something say 192.168.13.42 – couple things that come to mind here, the dmz client does not have its gateway set to point to pfsense, or its mask is wrong. Or it has a firewall preventing ping.

The fact that you have a dmz rules as any any and can not ping the lan pfsense IP, I have to think the clients gateway is not set to point to pfsense, or mask is off.

Your rules are correct.. Other thing that comes to mind is if your saying you did not create that outbound nat rule where you were natting traffic from dmz to lan to the wan IP? It was there from automatic settings, when you turned on manual. That is a problem!! And would cause your issues.

So I see your outbound nat rules are clean and on automatic - if you switch to manual you see the dmz to lan rule??

edit:
See here on my outbound nat rules (switch to manual so you could see them) - they only have source of my lan,wlan and dmz segments, and my openvpn segments - and they all say auto created. While yours below that -- doesn't have comment - so looks to me like you manually created that rule?? That rule would be BAD!

Also just to be sure - you don't have any rules in your floating tab?

Also bottom pic is ipconfig from a 192.168.1.0/24 box - and traceroute to IP on my 192.168.2.0/24 segment.. Can we see this from your dmz segment client.

Jerrid_C · Mar 7, 2014, 8:58 PM

No windows firewall enabled on any server, windows firewall is turned off. No other firewalls installed.

DMZ Server

LAN Server

Nat Set to Manual

johnpoz · Mar 7, 2014, 9:06 PM

Ok that rule is gone now! I showed a picture of your outbound rules showing that rule I was talking about 1st rule on the list.

This second posting of your nat outbound looks fine. So when you try and ping pfsense lan IP 192.168.1.1 from your dmz machine what does the firewall log in pfsense say? If anything if it blocked it it would be logged, unless you turned that off? By default anything blocked by default block rules are logged.

And your floating tab shows what? Have you rebooted pfsense after all the cleanup, reseting, etc.

There should be no reason why you can not ping the pfsense lan IP from dmz box, if your pointing that traffic to pfsense from the rules you have shown. If pfsense was blocking it - it should be logged.

And unless you have some specific routes on your dmz client that looks fine. Tracert would validate your talking to pfsense to get to the lan from the dmz. And post up your pfsense routes

Here is mine for example

Jerrid_C · Mar 7, 2014, 9:09 PM

Nothing is set in floating.

I turned on logging and saw this. Not sure how to explain this or investigate this

LAN Server

DMZ Server

Nothing shows in the log when I did the ping on the DMZ Server

Routing Tables

johnpoz · Mar 7, 2014, 9:37 PM

So what does the traceroute from dmz to 192.168.1.1 look like?

Can you post that dmz box routes.. So example see attached.

So clearly pfsense is sending on the packets to your dmz box 13.10, that your not seeing and answer points to your dmz box having the problem. But very odd that it can ping pfsense IP 192.168.13.1

Are you SURE the IP address for 192.168.13.1 is your pfsense mac?? Maybe you have something else on your 13 network that has IP of pfsense dmz IP?

Or you have a weird routing issue and traffic is not being sent to 13.1 for 192.168.1.0/24 networks… Does this dmz box get internet or access to wan network 10.x.x.x

You can view your mac table with arp -a

So for example you can see below my client on my lan network see my pfsense lan IP mac, and that matches up with my pfsense actual mac.. I would validate this on your dmz client.

edit: and from looking at mine, I have to do something with my dvr, they keep showing up with apipa addresses those 168.254 -- even though they are clearly on my network with the 192.168.1.97 and .98 addresses - notice the same macs.. Odd..

Jerrid_C · Mar 7, 2014, 9:45 PM

Well it seems to have been the DMZ server. I should of known to at least do a route -f to flush the routes to just remove that from being an issue. Once I did a route -f everything started to work. Thanks for helping me along the way. Much appreciated. I will probably be back with more questions as I start to set this up to mimic a real network.

johnpoz · Mar 7, 2014, 9:49 PM

So you had a static route setup? Happy to help - and glad you got it sorted.

Jerrid_C · Mar 7, 2014, 10:35 PM

No, I expect it had to be that it was a template that got deployed and it had some old routing information in there from when it was a vm before a template.

I might not be 100% done yet though, as I can now ping everything from DMZ, but I cannot get out to the internet. I added DNS and I can ping google.com, but when I open up IE and try to go to google.com, it just sits and spins. Tracert from the VM gives me this

johnpoz · Mar 8, 2014, 12:21 PM

Well that is odd as shit.. according to that output google.com is 74.125.225.73.. And you hit it on your 2nd hop?? But you continue on?

What does a trace look like from box in the lan segment? There is no difference to pfsense between lan and dmz other than what you might put on the firewall rules.. They are just 2 segments to pfsense that are not WAN connections that is all. So what does lan client traceroute look like?

jswj · Mar 9, 2014, 3:36 PM

Client DNS IP should point to the gateway address as well, this in turn pfsense will have dns forwarders to resolve the FQDN.

Set auto outbound NAT and disable/remove all nat mappings, also remove static route entry. Make backups of the current config before trying.

I have similar setups set on auto nat, no nat mappings, no static routes, only WAN has gateway. On each LAN interface firewall rules are to allow all traffic generating from the LAN subnet to any destination, tcp and udp, any port.