HomeLab & pfsense

Hello,

I don't know in which topic to put my questions so i put them here, correct me if there is a better place.

I bought a bare-metal server to build a homelab on my personal network at home.
I'll install Proxmox on this server as hypervisor.
I'll also build different VMs (Windows / Linux / Windows Server), and one pfsense as router and firewall.

Quickly, here is how my traffic should work :
Internet <-> ISP <-> Home Router (Provided by the ISP) <-> Proxmox Hypervisor <-> pfsense VM <-> other VMs

This is a more visual diagram of the potential structure :

As you can see on this diagram :

I have one Public Static IP from my ISP (which is a French ISP)
My home router is provided by my ISP (It act as Modem/Router/WiFi etc...)
My actual LAN is 192.168.1.0/24 where I have various devices already
I'd like to build a Virtual Private LAN (10.2.0.0/24 for example) with each VM connected to it
Also a Virtual Private LAN between the Hypervisor and the pfsense VM (10.1.0.0/30 for example)
I'd like to connect one interface of my server to this network (with IP 192.168.1.100)
All internet traffic of the VMs need to pass through the pfsense, in both directions

I'd like to expose only the SSH port of the Hypervisor directly to the Home Router and Internet (without going through pfsense)
A VPN will be required on the pfsense to access the Hypervisor Web GUI and everything else (pfsense, all VMs)
To allow traffic from outside to inside of my network, I have to add a NAT/PAT rule in my Home router.

My questions are :

Do you think this is possible ?
For each port I'll open on pfsense for a VM I'll need to open it as well on my Home Router ?
Did I forgot something ? Or made any mistakes ?
If i'd like to talk with my "Laptop 1" to one of the VMs, will it work ?

Sorry for my English mistakes as well...

Thank you very much in advance,

Kevin

stephenw10

It's possible but I would not arrange it like that.

Are you going to be routing between those 3 internal subnets? If so you will need static routes to reach VMs and you will then have asymmetric routing from clients in the 192.168.1.X subnet.
If you are not routing then that's 3 levels of NAT which is..... not a great idea!

You will need port forwards both in the ISP router and in pfSense to reach the internal VMs unless you're bridging the interfaces in pfSense.

You should probably have the NIC in the server shared with the pfSense VM so pfSense has a 192.168.1.X IP on it's WAN. If you want to filter traffic to the hypervisor as well you will need that NIC to be passed through to pfSense and then another internal connection to the hypervisor.

Steve

Thank you very much for your answer @stephenw10 !

I maybe need to make the following communications :

Laptop 1 (192.168.1.30) to a VM (10.2.0.30)
Laptop 1 to pfsense (on 10.1.0.2 ?)
Outside (other Public IP) to a VM (Web Server for example)
Outside to pfsense (VPN for example)

When you mean static routes, it's on the pfsense right ?

Yes I realize that each port that I'll open on the pfsense, I'll need to open the same on my Home Router.
(Small question, when I open a port on my Home Router (192.168.1.1) I need to enter an IP address for destination, should it be the Hypervisor one's (192.168.1.100) or the pfsense one's (10.1.0.2)
(Oooh, if I've the WAN connected directly to the Home Router, I don't have this problem anymore right ??)

My "dream" would be to setup the pfsense before anything in my lab (Hypervisor and VMs), but i'm confused with the idea that the pfsense is a VM of the Hypervisor, so how could it be placed "before"...

Kevin

johnpoz

Confused from your drawing at what would be doing the nat from this 192.168.1 to your pfsense 10.1.0?

I am with stephw all the things you want to do are clearly possible - but wouldn't do it that way.. Your 192.168.1 becomes a transit network once you connect pfsense as downstream. Placing device on this network could create asymmetrical traffic. Pfsense natting would could remove the asymmetrical problems but now you have to port forward and traffic from this 192.168.1 devices traffic into networks behind pfsense.

Can you not just use the 192.168.1 as pure transit and put all your network be them physical and vm behind pfsense? Only issue with this is reboot of your vm host cost you internet connectivity for all devices on your network.

Also if at all possible would change your isp device to bridge/modem mode so that pfsense just get public on its wan. Removes the double natting.

[Edit 1]

About my Home Router, I found some interesting things on the Internet, it might be possible to change...
If so, I need a Modem/Router (TP-Link or Netgear should do the trick ?)
I've an ADSL connection, so I need a Modem+Router to connect to the ISP's network, then i should build (1 or 2 separate, one for my home users, the other for me ?) networks and put pfsense directly after the router

[End - Edit 1]

Thank you as well for your answer @johnpoz !

To be honest with you, I'm still a beginner so I'm looking for the best way to do my networking.

I though it was for communication between the pfsense and the Home Router through the Hypervisor
There is an Ethernet communication from the Home router to the NIC of my bare-metal server, and the other should be a virtual interface.

I'm open to every solution, i just need to have the following things :

At least all VMs on should be behind the pfsense (If the Hypervisor could also, it would be perfect)
I can't run pfsense in an other way than in a VM...

I've several problems...

Since I'm building this at Home, I have all devices of my familly on the network were I'd like to setup my server... I prefer to let them outside of "my network" which means not behind the pfsense...
My ISP doesn't allow me to use it's network with a Custom Modem/Router... I need to use it's own Modem/Router (Yeah french ISPs aren't so cool...) this is why I'm forced to have this nested traffic and natting...

If i understand great, I could communicate from a computer on 192.168.1 to a VM on 10.2.0 ? (Since my default gateway is the Home Router, the computer ask for 10.2.0 to the Home Router, but the Home Router might not know where it is because only pfsense know, right?)

Thanks in advance,

Kevin

johnpoz

@keviiin38 said in HomeLab & pfsense:

I prefer to let them outside of "my network" which means not behind the pfsense...

You can have multiple networks behind pfsense to the edge (internet) doesn't mean they can talk to each other in anyway. Pfsense would be the FIREWALL between your networks.

@johnpoz said in HomeLab & pfsense:

@keviiin38 said in HomeLab & pfsense:

I prefer to let them outside of "my network" which means not behind the pfsense...

You can have multiple networks behind pfsense to the edge (internet) doesn't mean they can talk to each other in anyway. Pfsense would be the FIREWALL between your networks.

(I've edited my last answer)

So I could setup pfsense on the top of the network, don't apply any rules to the existing "Home Network" and apply all my rules to the "HomeLab network" ?
Pfsense could act as a FW and a router for the "HomeLab Network" right ?

Thanks in advance,

Kevin

stephenw10

Yes, and doing so would avoid the routing/access problems. But it also introduce connectivity issues if you ever have to reboot the pfSense VM or the host server. Connectivity would be lost everywhere.

I would just try it and see what works for you. At the very least you will get some useful experience.

I would avoid having the 10.1.0.X subnet entirely. Put the pfSense WAN NIC in the 192.168.1.X subnet directly.

Steve

@stephenw10 said in HomeLab & pfsense:

Yes, and doing so would avoid the routing/access problems. But it also introduce connectivity issues if you ever have to reboot the pfSense VM or the host server. Connectivity would be lost everywhere.

I would just try it and see what works for you. At the very least you will get some useful experience.

I would avoid having the 10.1.0.X subnet entirely. Put the pfSense WAN NIC in the 192.168.1.X subnet directly.

Steve

Okay thanks to you guys @stephenw10 and @johnpoz for your awesome answers !

I'll try every solutions you gave me, and make a feedback later on.

I have a last question, could you explain me if you can, in which network I put the Bare-Metal Server (Hypervisor/Proxmox) NIC, so that also the hypervisor behind the pfsense FW is ? Also on 192.168.1 ?

Thanks in advance,

Kevin

stephenw10

To do that the hypervisor must pass-through the NIC connected to the 192.168.1.X subnet to the pfSense VM only. It must not have an IP in that subnet itself. You can then add an internal NIC so that the hypervisor has connectivity via an IP on the internal subnet.

However that's far more complex to setup. Initially I would just add the pfSense VM and one other VM and concentrate on getting that setup and working with the internal subnet.

Steve

Thank you very much for your answers !

You guys @johnpoz and @stephenw10 are awesome !

I'll make some feedback on this topic later on

Kevin