Using pfsense with multiple WANs
-
Yes, you could do that. That's what I was suggesting above with the 1:1 NAT rules.
Since you have multiple public IPs on pf1 though you're going to need the same number of IPs on pf3 DCLAN in order to be able to forward the incoming connections to the right servers. Using 1:1NAT on pf1 just makes that easier.Steve
-
This is where my limited knowledge is not helping.
I'm not understanding the part where we're talking about adding VIP's or using a transport network.
I have to clarify to make sure I didn't add complexity that doesn't need to be in the mix.
On pfsense 01, I have only two networks, WAN and LAN.
I have two /29 public IPs set in the Virtual IPs.
I've whittled the use down to one /29 and three IPs being used as part of this change. Two of them are each being used with the HAproxy server routing to a few load balanced servers.On the LAN side, there is only 10.0.0.1/24 and it's DHCP server. Nothing more complex.
One way I can see you could do this would be using multiple VIPs on the pf3 OPT interface to correspond with the /29s on pf1. Then you could use 1:1 NAT rules in pf1 to forward all the traffic to pf3 and from there forwards again the the LAN side servers.
All of this has to be on the DCLAN side as there isn't enough bandwidth to handle the traffic if I sent it to the WAN side of pfsense 03.
Pfsense 01 has to send the traffic to the DCLAN interface of pfsense 03 which would have a private IP.
This is where it's not clear what we're talking about, a transport, a full network or one VIP for each server being redirected?
The public /29's on pfsense 01 and pfsense 03 are completely different of course.
To try and visualize what you're proposing,...
One to one;
pfsense 01
Add 172.16.1.x/24 IPs as needed (VIPs?) per server being moved from this firewalls NAT to pfsense 03.Meaning, a LAN server on pfsense 01 that is 10.0.0.50 for example, would have its rule changed to point to 172.16.1.50 which in turn would be sent to pfsense 03's DCLAN interface and then routed to the new 10.0.0.50 server on the LAN side?
How would DCLAN be configured?Or transport;
Add a new network on pfsense 01, say 172.16.1.1/30 that would be used to forward traffic to DCLAN on pfsense 03?
This means pfsense 03 would need a 172.16.x.x/? network on it's DCLAN NIC?
In this method, similar to above, I would change rules to point from 10.0.0.x servers to point to 172.16.x.x servers but use just one transport to get those to pfsense 03?I feel like I'm complicating all this but maybe once I understand the smaller details, it will all suddenly make sense.
-
If you need to use 10.0.0.0/24 for the LAN side of pf3 then you cannot route the traffic over a transport subnet. You must use NAT.
That means you must have a VIP on pf3 on it's DCLAN interface for each server you are forwarding to.
pf1 only needs a single VIP in the new subnet so it can send traffic on it.It's an ugly setup as we have discussed. But it should work.
Steve
-
I wanted to report back that it's all working but I'm stuck already :(.
Before I bring this to the DC, I need to test it by isolating it on its own.
I have pfsense 03 with its own Internet connection.
I set one IP, 192.168.1.3/24 on DCLAN to simulate an external network connection trying to get to the LAN network servers.I set this IP because I already have 192.168.1.1/24 and even 10.0.0.1/24 used on my network so wanted just a single IP on that interface.
Since I only have one IP set on the DCLAN interface, I'm wanting to set up some port forwards.
I can get to the pfsense firewall but I cannot get to anything behind the LAN switch. For example, I have something at port 8006 so I port forwarded 192.168.1.3, port 7071 to 10.0.0.74, port 8006.
I can see the hit but no luck from PC 192.168.1.205.
DCLAN tcp 192.168.1.205:32491 -> 10.0.0.71:8006 (192.168.1.3:7071) FIN_WAIT_2:FIN_WAIT_2 5 / 4 654 B / 172 B
-
You see a state on the LAN showing it leave?
It looks like it's allowing the traffic in on DCLAN and opening the correct state with NAT there.
The server may not allow that traffic if it's new and hasn't been configured for that.
You will need a gateway on DCLAN for traffic from external sources so incoming states are tagged reply-to correctly. Otherwise it will use the default route which will be the pf3 WAN (I expect).
Steve
-
Ah, darn, then I'm out of luck. I'm already using those networks and wanted to set something up that's isolated yet gives me access from the existing 192.168.1.1/24 network so I can transfer files, set up the servers etc.
The servers on the LAN Switch side are accessible. I have a laptop connected to the LAN Switch and it's able to see everything on that 10.0.0.1/24 network. -
Hmm, not sure what you're saying there. Those subnets look fine.
I'm just saying that pf3 needs to treat the DCLAN connection as a WAN so that it can receive the incoming connections from pf1 and send the replies back that way. In order to do that the DCLAN interface must have pf1 as a gateway.
Steve
-
Basically, I have a simulated condition before it goes to the data center.
I have a pfsense here at my home which has the LAN of 192.168.1.1/24. That pfsense also has two other networks on different interfaces, OPT1 10.0.0.1/24 and OPT2 192.168.254.1.
I've isolated the hardware onto it's own so is using pfsense 03.
It has WAN to Internet.
It has DCLAN with just 192.168.1.3/24 set on it so I can reach it from my network so I have a cable connected to one of the LAN Switch ports and my LAN switch.
It has 10g1LAN which is connected to the LAN Switch and 10g2OPT2 which is not in use.
All devices on the LAN Switch have Internet access and all devices/servers can communicate together.
This is where I wanted to simulate the DCLAN connection and is what I'm playing around with to understand before this goes to the DC.
My options seem to be, set up a transport link between my own pfsense and pfsense 03. However, since I'm not coming in from the Internet, this seems to be a waste of time. I just need to gain access to the LAN Switch side hardware so I can configure all of the hardware. Hence why I just assigned it a fixed IP that wasn't in use, 192.168.1.3. However, that allows me access to the pfsense 03 but nothing on the LAN Switch which is what I need.
I hope I'm explaining all this well. It's amazing how much text there is in writing things where if you are with someone and they are showing you something face to face, you pick it up in minutes.
-
So how are you testing? What's not working?
What you are describing sounds like it should work.
Set up a port forward on your home pfSense (substituting for pf1) from it';s public IP to the pf3 DCLAN IP.
Set up a forward in pf3 from there to the server on it's WAN.If you connect externally that should work as long as it's configured correctly.
Steve
-
Well, I don't know what's not working :). Probably something small I'm not seeing.
I can connect to 192.168.1.3 from any client on the 192.168.1.1/24 network because the IP is basically just another device on the LAN.
I can only reach the pfsense 03 device however, nothing behind it.For example, trying to reach a proxmox server which is at 10.0.0.71:8006. I added this rule to allow 192.168.1.x to reach it.
-
Ok so same as before; that should work from another host in the DCLAN subnet however pf3 is configured because only routing between local subnets is required.
For other sources the DCLAN interface in pf3 needs to be configured correctly (with a gateway).Check the state table in pf3 for a LAN state when you are trying to connect. If it's open and shows traffic it must be the server either refusing the connection or replying incorrectly.
Steve
-
So far, everything works in terms of giving DCLAN the IP of 192.168.1.3. I'm using port forwarding to reach everything.
I have some hardware problems I need to solve then the last step will be trying to set up a transport and simulate the setup once installed at the DC.
-
So it's all working as expected in your home test?
-
Hi, not done yet, things are still weird.
Something keeps happening with the firewall that keeps logging me out. When I connect directly to the pf03 10.0.0.1/24 network using a laptop everything works fine.
When I connect from the 192.168.1.1/24 network, things get weird. Every time I log into something, I get logged out of everything else in the other tabs.
I know it's because I'm forwarding non standard ports all over the place but it's making it really hard to get anything done. Today, I'll set up a PC directly connected so I don't have to use this crazy port forward stuff, then I can move on to the last step which is creating this transport network. -
Yeah, I've hit that a few times if you have several tabs open to the same IP or hostname where only the destination port is different. Conflicting sessions override each other. I've just used separate browsers to prevent that before.
Steve
-
To resolve that, I installed a windows vm on one of the hosts in the new 10.0.0.1/24 network. That seems to be working now.
I guess I can try fully isolating the whole thing soon but I'm not quite sure how I'll properly test this transport.
On my home network, I have a pfsense (call it 00) that handles 192.168.1.1/24 LAN, 192.168.254.1/24 AP and 10.0.0.1/24 to allow remote access to some servers for dev work.
Maybe the best thing is to get the hardware installed at this point and just give DCLAN a 10.0.0.x IP so I can remotely get to everything and once I have failsafe access via the existing 10.0.0.1/24 network and the new Internet connection, then might be the time to test the transport.
The transport will have to take into consideration that there will be other pfsense machines sending data to it as as has been suggested, a very small subnet and this way, I can add as many as needed if it comes to that.
-
So far, so good. I've got a windows-7 vm running on the same hardware. I think it's time to try this transport link.
-
Can anyone offer a url/link/article on how I can go about testing this transport method? I don't want to mess up what I've done to date.
-
You have a diagram of exactly what it is you're planning to do?
-
@stephenw10 Yes, the very first message.
In that image, I show how it's going to be connected at the DC but right now, it's still at my home. I am hoping to get this into the DC over the weekend if I can figure out this transport.
