Traffic blocked even with any/any rules on both interfaces

rjensen

i dont care which IPs by DMZ based servers use for internet access (its blocked anyway…) unless it has an impact on my incoming NAT rules which it seems to have?

If i do a regular HTTPS NAT rule that only works against the external WAN interface that holds the "default gateway"

johnpoz

huh? So you have no inbound traffic on these other IPs? And you don't use them for internet access? Then why do you have them setup? Im confused with your last post.

You might not care, but the server talking to publicIP 3, is going to care if traffic comes back from publicIP 1 – normally not going to like that ;)

rjensen

Sorry, i have incoming traffic (public IP1 > NAT (HTTPS) > server in DMZ) but i dont care if that server in DMZ goes to the internet using public IP1, 2 3 etc. (its blocked to get to the internet)
however if i build these 2 NAT rules only the one public IP (WAN interface) holding "default gateway" works:

public IP1 > NAT (HTTPS) > server01 in DMZ
public IP2 > NAT (HTTPS) > server02 in DMZ

hopes that make sense - it should be no surprise to anyone im new at this :)

johnpoz

"public IP2 > NAT (HTTPS) > server02 in DMZ"

This is my point, so server02 is going to answer that client talking to public IP2 from public IP1 – that client wanting to talk to public IP2 is not going to accept traffic from public IP1 as an answer..

You need to look at your outbound nat rules to make sure that traffic coming in on public IP2 goes back out public IP2..

And not understanding this statement
"but i dont care if that server in DMZ goes to the internet using public IP1, 2 3 etc. (its blocked to get to the internet)"

If you going to want server in the DMZ to answer something from the internet, how could its internet traffic be blocked?

rjensen

OK, i see the Firewall > NAT > Outbound but thats beyond me…

Not sure what source and destination would be? Im looking to achieve being able to nat say port 443 against 1 server and 80 against another server on the same public IP if that makes sense.

I might be wrong but i think i had these multiple NAT rules working before without making any changes.
Under System > Advanced > firewall/NAT i have the options disabled as shown on the attached screenshot: is that part of my issue?

sorry - the servers in DMZ do have internet access, i am just planning on disallowing the servers to browse the internet by blocking port 80/443 outbound to secure the environment. but no such rules at the moment.

![Screen Shot 2013-03-19 at 4.41.48 PM.png](/public/imported_attachments/1/Screen Shot 2013-03-19 at 4.41.48 PM.png)
![Screen Shot 2013-03-19 at 4.41.48 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2013-03-19 at 4.41.48 PM.png_thumb)

johnpoz

You need to look at your outbound nats, change them from automatic to manual so you can see what rules are in place currently.

rjensen

Ive been trying to figure out how to make this work and im not sure if this is a "normal network issues" or related to pfSense? Im at a place now where im thinking going with 5 x pfSense servers - 1 for each external IP as i cant seem to figure out how this is configured in general…

I have switched to manual outbound NAT rules which creates a pre-defined set of rules it seems.
Im also seeing issues when i have multiple WAN interfaces enabled: none of my servers in the DMZ can get to the internet. Before if i only have 1 WAN address set with a default GW that works fine as they can hit the internet and my NAT rules works.

With these 5 WAN IPs (DHCP - Same gateway) how would i configure the outbound NAT rules?
I dont understand if this is set for each host, subnet, interface etc. I understand inbound NAT but as i cant grasp the concepts of outbound NAT i have no idea what to configure.

Is it actually achievable having 1 pfSense with 5 WAN IPs (same gateway) and have regular NAT rules on say HTTPS for each of those 5 WAN IPs to different DMZ targets?

johnpoz

Switch to manual outbound and we can walk through why your having an issue.

So example - now I only have 1 public IP, and here are my outbound nats after switching to manual so you can see them.

So you see the rule I highlighted - this says hey your coming from network 192.168.1.0/24 use the WAN address as public and dynamically do the ports. This is the auto created nat rule..

Lets see yours!

So I created a virtual IP, just example 1.2.3.4 - then I created an outbound nat that says hey if your coming from 192.168.1.42 you use the IP 1.2.3.4 when you go outbound.

outboundnat.jpg_thumb

outboundnatexample.jpg_thumb

rjensen

Attached. Only have 2 of the WAN interfaces active now - thats enough to confuse me so ill add the other 3 in later :)

Right now none of my servers in 192.168.190.0/24 can get to the internet. traceroute etc. does not work.
even the pfSense itself cant do traceroute.
i have WAN1 set with default gateway.

![Screen Shot 2013-03-31 at 6.16.49 PM.png](/public/imported_attachments/1/Screen Shot 2013-03-31 at 6.16.49 PM.png)
![Screen Shot 2013-03-31 at 6.16.49 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2013-03-31 at 6.16.49 PM.png_thumb)

rjensen

I tried the same as you did (i think…) defining that 192.168.190.0/24 has to go through WAN01 but still - even the pfSense cant get to the internet. Im just trying a basic traceroute.
With this approach would i need 5 different subnets/VLANs to make this work? As i have 4 webservers each requiring HTTPS 443 to be available would i need an outbound NAT rule that is mirroring my inbound NAT rule?

johnpoz

Well if you can not even get to the internet your rules are not going to work… If you say you can not get to the internet from pfsense - how is an outbound route going to come into play?

rjensen

agreed - but how do i figure out what the issue is? With a single WAN interface it works - as soon as i add the 2nd all outgoing traffic fails.

podilarius

The rules you have posted for manual outbound NAT looks wrong. You have multiple subnets using multiple interfaces. Simplify the rules a bit by putting different subnets on different external ip addresses. That is unless you are more worried about inbound. Then 1:1 nat would be advisable with the default outbound for all sinners on only one address.

rjensen

i did just try that and the result is that i still cant get to the internet now from any machines in the subnets i have changed outbound NAT for.
pfSense i also tried enabling a second WAN interface and changed all the outbound NAT rules to use that interface, but still no luck.

one thing i keep seeing is that when i add a second interface it monitors the gateway and it gets no status:

Name Gateway Monitor Status Description
GW_WAN xx.xx.xx.xx Gathering data Gathering data

the first interface i have added (default gateway is set on this) works fine and it reports RTT and online status.
can this have an impact?

podilarius

It should report a status if it is setup correctly. If you manually ping the address it should fail also. My guess is that this has a impact if that is the interface you are trying to use. Did you use an IP addresss in the same range as WAN or is that a completely different ISP/conntection?

rjensen

gateway is the same for all the IPs from the same ISP.

my ISP allocates me 5 addresses on my ADSL bridge which i then configure as 1 interface for per IP on the pfSense (DHCP allocated).
i have no static routes or other gateways defined elsewhere. pfSense has a OPT interface in my DMZ and a LAN interface for well - LAN and management.

podilarius

Honestly, it sounds like you have a split route problem which is creating a state problem. I am guessing that you are seeing dropped packets in the firewall logs. Have you tried proxy arp yet?

rjensen

Have not tried proxy ARP - dont know if i can as it seems to require virtual IP (Static IP) which i cant do as the IPs i get from ISP has to be DHCP.

But this is really strange…
Now i deleted all the manual outbound NAT rules, switches back to automatic (i only had 1 WAN interface configured at the time) added a second WAN interface, created HTTPS NAT rules to 2 different machines internally on my DMZ (same subnet) and now it works...
All DMZ machines go out on the same IP even though they have different inbound NAT rules.

When switching back to automatic outbound NAT is that equal to it keeping the settings from when i had a single WAN interface?
But whats even strange is that if i now with 2 WAN interfaces enable manual outbound NAT it continues to work - and i still have those 2 overlapping outbound NAT rules where my 192.168.190.x/24 looks like it has 2 outbound rules - one via each WAN interface.

podilarius

Basically, yes, the AON is equivelant to having only one WAN connection. If you want a server to use a particular IP, try 1:1 NAT and not manual outbound NAT. I would only use MON if you are going map a whole subnet to a particular WAN.

rjensen

and now im back to it not working again :)

decided to just have 1 pfSense for every external IP i have and then map 1:1 with a DMZ VLAN. Not pretty but that i can get working.

Thanks everyone for helping though!