Creating virtual networks based on proxmox with pfsense

timoonskii · Jul 2, 2020, 10:46 AM

Hi guys,

I am new to pfsense and proxmox and currently I’m stuck. Hopefully, you can help me :).

Environment:
proxmox cluster based on 3 nodes. Setup will be on just one of them. One of the network ports (enp5s0f0) has a bridge (vmbr0) with 192.168.123.23 and defined 192.168.123.1 as gateway. Before that, theres a physical firewall.

What I want to achieve:
Virtualized Pfsense that has connection to WAN (vmbr0) and several network zones (vmbr10 (192.168.10.1), (vmbr11 (192.168.11.1), vmbr12 (192.168.12.1), vmbr13 (192.168.13.1), vmbrX etc.) that are all like DMZ.
VMs in proxmox will attach the specific linux bridge of their zone. For example: Vm1100 attach the vmbr11. VM1100 can just go out in the internet but not to the other zones.

Current state:
I’ve created vmbr11, vmbr12 and so on, on the proxmox node. The pfsense has been connected to vmbr0 (WAN) vmbr11 (ZONE1), vmbr12 (ZONE2) etc. A vm hat vmbr11, got an ip address by the DHCP from pfsense. So it has connection to the pfsense. It’s able to ping 8.8.8.8 and google.ch (dns works!) but I can’t reach any websites by using a browser.
If I connect the VM directrly to the vmbr0, everything (ping, dns and reach websites) works!
Pfsense config: in networking I disabled Hardware Checksum Offloading, Hardware TCP Segmentation Offloading and Hardware TCP Segmentation Offloading.
On the Nework interfaces “Block private networks and loopback addresses”

Screenshots

Many thanks in advice! :)

viragomann · Jul 2, 2020, 11:25 AM

Something in the firewall log?

NogBadTheBad · Jul 2, 2020, 12:10 PM

@timoonskii said in Creating virtual networks based on proxmox with pfsense:

Hi guys,

I am new to pfsense and proxmox and currently I’m stuck. Hopefully, you can help me :).

Bet your a Cisco guy, using the term Zones

timoonskii · Jul 2, 2020, 1:01 PM

Hi viragomann,

oh yes, interesting point, sorry forgot that to mention:

i can see my request here (google.com with 216.58.215.238)

I think the response dind't came back... ?

@NogBadTheBad
We have no cisco in use

viragomann · Jul 2, 2020, 1:01 PM

@timoonskii said in Creating virtual networks based on proxmox with pfsense:

I think the response dind't came back... ?

You can easily investigate that by using Diagnostic > Packet Capture.
Take a capture on WAN interface while you try to access an internet site from the testdevice.

Here you can verify, that the outbound NAT translates source addresses correctly into the WAN address.
If there are no response packets, the problem may possibly be outside of pfSense.

NogBadTheBad · Jul 2, 2020, 1:03 PM

@timoonskii said in [Creating virtual networks based on proxmox with pfsense]

@NogBadTheBad
We have no cisco in use

Zones tend to be a term Cisco firewall guys use.

timoonskii · Jul 21, 2020, 8:02 AM

Hey guys,

so sorry for not getting back to you earlier!

I've done the pcap: go on google.com from a VM, IP192.168.10.54

....

...

..

The TCP traffic has ECN and CWR, so there might be a congestion that is related to layer3?

many thanks in advice!
regards

viragomann · Jul 21, 2020, 11:14 AM

@timoonskii
So 192.168.195.60 is your WAN address. Is it the only one or do you have a subnet?
When you connect the VM to vmbr0 and you're able to browse the internet which IP does it use?

Can you access the WAN gateway normally from pfSense?

timoonskii · Jul 21, 2020, 12:45 PM

192.168.195.60 is my WAN address (sorry had to change it in the meanwhile)

yes when i use vmbr0 browsing is working.

i can reach the pfsense from my local browser (not in the 192.168.x.x Network)

thanks for your help
regards