interesting, but after the upgrade I didn't see any arp entries on the WAN with arp proxy, I couldn't even ping the upstream gateway. Here is from your link:

If a particular configuration does not work with IP alias or Proxy ARP type VIPs, try with a CARP VIP instead, or vice versa. Address or wait out the potential ARP concerns before declaring one particular type a failure, and always be on the lookout for IP conflicts.

I didn't see any IP conflicts, but maybe the ARP table became corrupted.

https://doc.pfsense.org/index.php/Port_Forward_Troubleshooting

@truetype
I'm wandering on by here but if you simply redirect port publicip:80 to privateip:443 using NAT that doesn't do a redirect, that would cause an error since the web browser and web server are using two different forms of communication. Let the connection to 80 work and have the web server redirect to https:// so the browser knows to talk https.

If it is only that one FQDN, just set port 80 on the outside of the port forward and 8000 on the inside.

0_1527631920572_Screen Shot 2018-05-29 at 3.11.32 PM.png

Thank you for the link. Seems that I had forgotten to set "Enable automatic outbound NAT for Reflection". After setting this, servers were able to communicate with nodes on the same VLAN

So I got the Managed Switch and now I have several VLANs:

VL10_MGMT VL20_SEC - this is were main clients will connect (mostly via WIFI) and it'll use a VPN_WAN gateway. VL30_CLR - sort of a DMZ where I connected all LAN devices (Freenas and its jails, Receiver, TV, AppleTV, etc) VL40_GUEST - WIFI network only for... guests VL50_IOT - where I'll connect several IoT devices via WIFI (smart lamps, dimmers, climate, etc)

Makes sense?

Why you need manual NAT?
You can just select the desired gateway in each lan rules!
It's under advanced when editing a rule.

it works! Thanks anyway.

Yes, you can setup a LAGG between the firewall and client directly. Or between two firewalls for that matter.

Steve

Hi,
Further informations to this phenomene:

The PFSense runs virtualized on XEN Hosts. After a live-migration of the VM while packets are dropped, everythings works again. We have another PFSense in a completely different setup with similar problems concerning 1:1 natted Systems running on VMWare ESX. On ths system my colleague implemented a cronjob which regularly resaves the WAN-Interface to prevent this phenomene ;-/

Cheers
      Ulli

Ok thanks

Like this:

pfSense-Layer-3-Switch.png
pfSense-Layer-3-Switch.png_thumb

Oh no Derelict I can see a feature request coming to add the magic "unfrack this fracked network design" checkbox.

You think we could get that setup for say 2.6? ;) heheheheh

Hello,

I Have the same issues with 2 Xbox One.
The NAT is open for Xbox Live, but not possible to join a session in warframe (no probleme with rocket league).

https://forums.warframe.com/topic/949122-no-coop-for-2-xbox-same-isp/

Jim,
I am so sorry - I missed your response on this. I know it's been six months, but the problem reared it's head again.

If I understand correctly, you are saying that the combination of NAT port forwarding and 1:1 NAT to my virtual IP's assigned to the CIDR block "could" be causing the issue when you say this "… if something happened to the port forward then it may misbehave.".

It's a weird too as often getting the remote user to clear their browser cache causes the problem to go away - but other times it takes a day.

We had been using NAT port forwarding in conjunction with 1:1 NAT to try and conserve our static IP's  - but it sounds like it might be safer to just do the 1:1 NAT and not port forwards.

Is there any way to further pin this down? I have correlated Chrome browser network requests, with pfSense firewall logs and the request logs on the two web servers involved.  I can pretty clearly see where the first six requests from the browser are all to the IP address of the first web server, but pfSense shows the sixth request gets NATed to a different server - but of course no rationale for why it did that.

UPDATE: Yes we are also using aliases a good bit. What type of issues might that cause?

Thank you again - Richard

Note you don't strictly NEED a VIP if the traffic for those addresses is routed to the WAN interface. All that matters is the traffic arrives. If so, NAT will happen.

If it is an address in the WAN subnet (or some silly, unrouted, secondary WAN subnet) then you must have something that will respond to ARP from upstream in place on WAN, meaning one of the VIP types except Other.

Perhaps. But firewall rules blocking everything but SMTP are far, far easier.

Either way it looks like you want this behavior on whatever Lan2/Router are in your "diagram" and not on pfSense.