Using pfsense with multiple WANs
-
If you need to use 10.0.0.0/24 for the LAN side of pf3 then you cannot route the traffic over a transport subnet. You must use NAT.
That means you must have a VIP on pf3 on it's DCLAN interface for each server you are forwarding to.
pf1 only needs a single VIP in the new subnet so it can send traffic on it.It's an ugly setup as we have discussed. But it should work.
Steve
-
I wanted to report back that it's all working but I'm stuck already :(.
Before I bring this to the DC, I need to test it by isolating it on its own.
I have pfsense 03 with its own Internet connection.
I set one IP, 192.168.1.3/24 on DCLAN to simulate an external network connection trying to get to the LAN network servers.I set this IP because I already have 192.168.1.1/24 and even 10.0.0.1/24 used on my network so wanted just a single IP on that interface.
Since I only have one IP set on the DCLAN interface, I'm wanting to set up some port forwards.
I can get to the pfsense firewall but I cannot get to anything behind the LAN switch. For example, I have something at port 8006 so I port forwarded 192.168.1.3, port 7071 to 10.0.0.74, port 8006.
I can see the hit but no luck from PC 192.168.1.205.
DCLAN tcp 192.168.1.205:32491 -> 10.0.0.71:8006 (192.168.1.3:7071) FIN_WAIT_2:FIN_WAIT_2 5 / 4 654 B / 172 B
-
You see a state on the LAN showing it leave?
It looks like it's allowing the traffic in on DCLAN and opening the correct state with NAT there.
The server may not allow that traffic if it's new and hasn't been configured for that.
You will need a gateway on DCLAN for traffic from external sources so incoming states are tagged reply-to correctly. Otherwise it will use the default route which will be the pf3 WAN (I expect).
Steve
-
Ah, darn, then I'm out of luck. I'm already using those networks and wanted to set something up that's isolated yet gives me access from the existing 192.168.1.1/24 network so I can transfer files, set up the servers etc.
The servers on the LAN Switch side are accessible. I have a laptop connected to the LAN Switch and it's able to see everything on that 10.0.0.1/24 network. -
Hmm, not sure what you're saying there. Those subnets look fine.
I'm just saying that pf3 needs to treat the DCLAN connection as a WAN so that it can receive the incoming connections from pf1 and send the replies back that way. In order to do that the DCLAN interface must have pf1 as a gateway.
Steve
-
Basically, I have a simulated condition before it goes to the data center.
I have a pfsense here at my home which has the LAN of 192.168.1.1/24. That pfsense also has two other networks on different interfaces, OPT1 10.0.0.1/24 and OPT2 192.168.254.1.
I've isolated the hardware onto it's own so is using pfsense 03.
It has WAN to Internet.
It has DCLAN with just 192.168.1.3/24 set on it so I can reach it from my network so I have a cable connected to one of the LAN Switch ports and my LAN switch.
It has 10g1LAN which is connected to the LAN Switch and 10g2OPT2 which is not in use.
All devices on the LAN Switch have Internet access and all devices/servers can communicate together.
This is where I wanted to simulate the DCLAN connection and is what I'm playing around with to understand before this goes to the DC.
My options seem to be, set up a transport link between my own pfsense and pfsense 03. However, since I'm not coming in from the Internet, this seems to be a waste of time. I just need to gain access to the LAN Switch side hardware so I can configure all of the hardware. Hence why I just assigned it a fixed IP that wasn't in use, 192.168.1.3. However, that allows me access to the pfsense 03 but nothing on the LAN Switch which is what I need.
I hope I'm explaining all this well. It's amazing how much text there is in writing things where if you are with someone and they are showing you something face to face, you pick it up in minutes.
-
So how are you testing? What's not working?
What you are describing sounds like it should work.
Set up a port forward on your home pfSense (substituting for pf1) from it';s public IP to the pf3 DCLAN IP.
Set up a forward in pf3 from there to the server on it's WAN.If you connect externally that should work as long as it's configured correctly.
Steve
-
Well, I don't know what's not working :). Probably something small I'm not seeing.
I can connect to 192.168.1.3 from any client on the 192.168.1.1/24 network because the IP is basically just another device on the LAN.
I can only reach the pfsense 03 device however, nothing behind it.For example, trying to reach a proxmox server which is at 10.0.0.71:8006. I added this rule to allow 192.168.1.x to reach it.
-
Ok so same as before; that should work from another host in the DCLAN subnet however pf3 is configured because only routing between local subnets is required.
For other sources the DCLAN interface in pf3 needs to be configured correctly (with a gateway).Check the state table in pf3 for a LAN state when you are trying to connect. If it's open and shows traffic it must be the server either refusing the connection or replying incorrectly.
Steve
-
So far, everything works in terms of giving DCLAN the IP of 192.168.1.3. I'm using port forwarding to reach everything.
I have some hardware problems I need to solve then the last step will be trying to set up a transport and simulate the setup once installed at the DC.
-
So it's all working as expected in your home test?
-
Hi, not done yet, things are still weird.
Something keeps happening with the firewall that keeps logging me out. When I connect directly to the pf03 10.0.0.1/24 network using a laptop everything works fine.
When I connect from the 192.168.1.1/24 network, things get weird. Every time I log into something, I get logged out of everything else in the other tabs.
I know it's because I'm forwarding non standard ports all over the place but it's making it really hard to get anything done. Today, I'll set up a PC directly connected so I don't have to use this crazy port forward stuff, then I can move on to the last step which is creating this transport network. -
Yeah, I've hit that a few times if you have several tabs open to the same IP or hostname where only the destination port is different. Conflicting sessions override each other. I've just used separate browsers to prevent that before.
Steve
-
To resolve that, I installed a windows vm on one of the hosts in the new 10.0.0.1/24 network. That seems to be working now.
I guess I can try fully isolating the whole thing soon but I'm not quite sure how I'll properly test this transport.
On my home network, I have a pfsense (call it 00) that handles 192.168.1.1/24 LAN, 192.168.254.1/24 AP and 10.0.0.1/24 to allow remote access to some servers for dev work.
Maybe the best thing is to get the hardware installed at this point and just give DCLAN a 10.0.0.x IP so I can remotely get to everything and once I have failsafe access via the existing 10.0.0.1/24 network and the new Internet connection, then might be the time to test the transport.
The transport will have to take into consideration that there will be other pfsense machines sending data to it as as has been suggested, a very small subnet and this way, I can add as many as needed if it comes to that.
-
So far, so good. I've got a windows-7 vm running on the same hardware. I think it's time to try this transport link.
-
Can anyone offer a url/link/article on how I can go about testing this transport method? I don't want to mess up what I've done to date.
-
You have a diagram of exactly what it is you're planning to do?
-
@stephenw10 Yes, the very first message.
In that image, I show how it's going to be connected at the DC but right now, it's still at my home. I am hoping to get this into the DC over the weekend if I can figure out this transport.
-
I'm sorry I can only hold so many threads at the front of my mind at once.
I'm unsure now exactly what you have configured and what you are going to configure.
Reading back it looks like you added the DCLAN interface to the pf3 device setup on your home network and are now able to access it via port forwards?Steve
-
@stephenw10 LOL, no problem, I don't expect anyone to remember this long thread :).
Yes, my LAN is 192.168.1.1/24 so I gave DCLAN on pfsense-03 the IP of 192.168.1.3/32 so I could reach it.
On my network, I also have a 10.0.0.1/24 and the pfsense-03 has its own 10.0.0.24. I have a laptop connected directly to the LAN Switch.All this hardware needs to go to the DC this weekend so I'm at the point where I'd like to understand how to set up this transport. Because there will be other pfsense firewalls sending traffic to pfsense-03, the idea was to use tiny transport networks so I could add more as needed.
