Using pfsense with multiple WANs
-
Basically, I have a simulated condition before it goes to the data center.
I have a pfsense here at my home which has the LAN of 192.168.1.1/24. That pfsense also has two other networks on different interfaces, OPT1 10.0.0.1/24 and OPT2 192.168.254.1.
I've isolated the hardware onto it's own so is using pfsense 03.
It has WAN to Internet.
It has DCLAN with just 192.168.1.3/24 set on it so I can reach it from my network so I have a cable connected to one of the LAN Switch ports and my LAN switch.
It has 10g1LAN which is connected to the LAN Switch and 10g2OPT2 which is not in use.
All devices on the LAN Switch have Internet access and all devices/servers can communicate together.
This is where I wanted to simulate the DCLAN connection and is what I'm playing around with to understand before this goes to the DC.
My options seem to be, set up a transport link between my own pfsense and pfsense 03. However, since I'm not coming in from the Internet, this seems to be a waste of time. I just need to gain access to the LAN Switch side hardware so I can configure all of the hardware. Hence why I just assigned it a fixed IP that wasn't in use, 192.168.1.3. However, that allows me access to the pfsense 03 but nothing on the LAN Switch which is what I need.
I hope I'm explaining all this well. It's amazing how much text there is in writing things where if you are with someone and they are showing you something face to face, you pick it up in minutes.
-
So how are you testing? What's not working?
What you are describing sounds like it should work.
Set up a port forward on your home pfSense (substituting for pf1) from it';s public IP to the pf3 DCLAN IP.
Set up a forward in pf3 from there to the server on it's WAN.If you connect externally that should work as long as it's configured correctly.
Steve
-
Well, I don't know what's not working :). Probably something small I'm not seeing.
I can connect to 192.168.1.3 from any client on the 192.168.1.1/24 network because the IP is basically just another device on the LAN.
I can only reach the pfsense 03 device however, nothing behind it.For example, trying to reach a proxmox server which is at 10.0.0.71:8006. I added this rule to allow 192.168.1.x to reach it.
-
Ok so same as before; that should work from another host in the DCLAN subnet however pf3 is configured because only routing between local subnets is required.
For other sources the DCLAN interface in pf3 needs to be configured correctly (with a gateway).Check the state table in pf3 for a LAN state when you are trying to connect. If it's open and shows traffic it must be the server either refusing the connection or replying incorrectly.
Steve
-
So far, everything works in terms of giving DCLAN the IP of 192.168.1.3. I'm using port forwarding to reach everything.
I have some hardware problems I need to solve then the last step will be trying to set up a transport and simulate the setup once installed at the DC.
-
So it's all working as expected in your home test?
-
Hi, not done yet, things are still weird.
Something keeps happening with the firewall that keeps logging me out. When I connect directly to the pf03 10.0.0.1/24 network using a laptop everything works fine.
When I connect from the 192.168.1.1/24 network, things get weird. Every time I log into something, I get logged out of everything else in the other tabs.
I know it's because I'm forwarding non standard ports all over the place but it's making it really hard to get anything done. Today, I'll set up a PC directly connected so I don't have to use this crazy port forward stuff, then I can move on to the last step which is creating this transport network. -
Yeah, I've hit that a few times if you have several tabs open to the same IP or hostname where only the destination port is different. Conflicting sessions override each other. I've just used separate browsers to prevent that before.
Steve
-
To resolve that, I installed a windows vm on one of the hosts in the new 10.0.0.1/24 network. That seems to be working now.
I guess I can try fully isolating the whole thing soon but I'm not quite sure how I'll properly test this transport.
On my home network, I have a pfsense (call it 00) that handles 192.168.1.1/24 LAN, 192.168.254.1/24 AP and 10.0.0.1/24 to allow remote access to some servers for dev work.
Maybe the best thing is to get the hardware installed at this point and just give DCLAN a 10.0.0.x IP so I can remotely get to everything and once I have failsafe access via the existing 10.0.0.1/24 network and the new Internet connection, then might be the time to test the transport.
The transport will have to take into consideration that there will be other pfsense machines sending data to it as as has been suggested, a very small subnet and this way, I can add as many as needed if it comes to that.
-
So far, so good. I've got a windows-7 vm running on the same hardware. I think it's time to try this transport link.
-
Can anyone offer a url/link/article on how I can go about testing this transport method? I don't want to mess up what I've done to date.
-
You have a diagram of exactly what it is you're planning to do?
-
@stephenw10 Yes, the very first message.
In that image, I show how it's going to be connected at the DC but right now, it's still at my home. I am hoping to get this into the DC over the weekend if I can figure out this transport.
-
I'm sorry I can only hold so many threads at the front of my mind at once.
I'm unsure now exactly what you have configured and what you are going to configure.
Reading back it looks like you added the DCLAN interface to the pf3 device setup on your home network and are now able to access it via port forwards?Steve
-
@stephenw10 LOL, no problem, I don't expect anyone to remember this long thread :).
Yes, my LAN is 192.168.1.1/24 so I gave DCLAN on pfsense-03 the IP of 192.168.1.3/32 so I could reach it.
On my network, I also have a 10.0.0.1/24 and the pfsense-03 has its own 10.0.0.24. I have a laptop connected directly to the LAN Switch.All this hardware needs to go to the DC this weekend so I'm at the point where I'd like to understand how to set up this transport. Because there will be other pfsense firewalls sending traffic to pfsense-03, the idea was to use tiny transport networks so I could add more as needed.
-
What I would consider a transport subnet would be something without other devices on it that you can route traffic across. You can't really do that here because you need the NAT to allow both firewalls to use 10.0.0.0/24.
What you're doing is simply port forwarding twice in order to reach the servers behind the second firewall. That can be across a separate subnet. So, yes, you should set that up and make sure you can reach the servers from the outside.
I mentioned before that for that to work you will relying on reply-to tagging since pf3 has a separate default WAN connection. So that definitely needs to be verified.Steve
-
Ok, I think the best way to work on this will be after the gear is installed at the DC tomorrow.
I'll give myself remote access via the new WAN connection and that will give me access to this new network and the old one. At that point, it'll be easier because it'll be the real setup instead of trying to simulate it from here.Hope it all goes well and I'll be in touch when I'm back on Monday.
Thanks so much for sticking with me on this. It's been a great learning experience. I'm always floored at what I can do with pfsense and recommend it to many companies that ask me about routing/firewalls. -
Alright, got everything into the data center and it's all working.
The new rack has its own pfsense fw now called 'pfsense 01' since it must become the main firewall. The WAN connection is working and I can see some traffic on the DCLAN interface or (OPT1).
Hopefully, this updated image better explains. Also, I wonder if I should start a new question to benefit someone else who might be interested in learning about transport networks since this will be lost in this long thread.
-
Good to hear. Let us know if you see any issues moving stuff to the new location.
-
(I changed the pfsense numbers to make it simpler and more updated)
It was done over the weekend. The image reflects how things are right now and what I need to do next which is this transport :).
As a refresher, I need to move some of the 10.0.0.x hosts from the pfsense 02 over to a new network behind pfsense 01.
I also need to keep pfsense 02 online, with its own Internet connection, sending/receiving traffic from hosts on the LAN side of pfsense 01.
In addition, there are other pfsense devices in the mix which I need to forward their traffic to pfsense 01. However, I've not shown those to try and keep the question simpler. I figure once I have a way of doing this with what I show, then it's only a matter of duplicating with new transport networks.
