Drastically Slow internet Speed for VMs/CTs Behind pfSense on Proxmox
-
@stephenw10 so what should I do, upstream for LAN to none and add NAT outbound for WAN
-
@zikou Normally one does not need to change Outbound NAT routing unless you're trying to do something specific like have certain traffic use an alias IP.
It also looks like you're trying to map traffic on the WAN interface in your screenshot?
I suggest leaving Outbound NAT at Automatic.
A LAN network does not have a gateway because pfSense should be routing any traffic for other networks to its gateway, i.e. towards the Internet. So unless you have two Internet connections there should be only one gateway.
Can you answer about the checksum setting?
-
@SteveITS yeah the checksum is already configured
I set the upstream of LAN to none still low upload
speedtest Speedtest by Ookla [error] Error: [0] Cannot read from socket: Server: WEMACOM Telekommunikation GmbH - Frankfurt (id: 54577) ISP: OVHcloud Idle Latency: 1.56 ms (jitter: 0.12ms, low: 1.53ms, high: 1.76ms) Download: 2525.11 Mbps (data used: 1.6 GB) 3.73 ms (jitter: 0.82ms, low: 1.54ms, high: 7.64ms) Upload: FAILED cy: 1.55 ms [error] Cannot write: -
@zikou said in Drastically Slow internet Speed for VMs/CTs Behind pfSense on Proxmox:
I set the upstream of LAN to none
What about on the WAN interface config? Does that have the upstream gateway set?
If so you should then see the auto outbound NAT rules generated for the LAN subnet to WAN translation.
But, as I said, the rules you have in proxmox make it look like it's NATing the LAN subnet there and if so you don't need any outbound NAT in pfSense. However that would require a route in Proxmox.
So check in Proxmox does it have a route to the LAN subnet via the pfSense WAN? If it's routing via something else that may cause the asymmetry. -
@stephenw10 here the route table in proxmox host
route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 141.55.x.x 0.0.0.0 UG 0 0 0 vmbr0
10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 vmbr1
10.1.0.0 0.0.0.0 255.255.255.0 U 0 0 0 vmbr2
141.75.x.0 0.0.0.0 255.255.255.0 U 0 0 0 vmbr0WAN interface
here the output of traceroute in vm
traceroute google.com
traceroute to google.com (172.217.168.78), 30 hops max, 60 byte packets
1 * * 10.1.0.252 (10.1.0.252) 0.217 ms
2 10.1.0.1 (10.1.0.1) 0.076 ms 0.144 ms 0.136 ms
3 141.75.x.x (141.95.x.x) 0.540 ms 0.569 ms 0.603 ms
4 10.164.161.142 (10.164.161.142) 0.416 ms 10.164.161.144 (10.164.161.144) 0.327 ms 10.164.161.142 (10.164.161.142) 0.404 ms -
Ok, two problems there:
The gateway is not set on the WAN interface in pfSense. Which means no auto outbound NAT rules are generated for it.
The traffic route from the client is not going through pfSense at all. It goes to the pfSense LAN and is redirected to 10.1.0.1 which is some other router in the LAN subnet. It never goes to the pfSense WAN side.
And that is where your asymmetric route is because reply traffic just goes from that other router to the client directly which means pfSense never sees it and hence will be blocking traffic.
What is 10.1.0.1? I don't see it in the Proxmox routing table.
Unless there's some other subnet behind that you need access to you should remove that as a gateway in pfSense. Make sure the WAN gateway is set as default and is on the WAN dircetly.
However as I said before it looks like whoever set this up expected hosts to use the LAN directly because it has a MASQ rule for that subnet. Normally that would not exist is clients had to use pfSense to get external access.
-
@stephenw10 there is a dhcp server the gateway for vm that are in vmb2 is 10.1.0.252
10.1.0.1 is the IP address assigned to your vmbr2 bridge on the Proxmox host. It's not a separate physical router; it's the Proxmox host's interface that directly connects to the 10.1.0.0/24 network.
ip route show on proxmox host
ip route show
default via 141.75.x.x dev vmbr0 proto kernel onlink
10.0.0.0/24 dev vmbr1 proto kernel scope link src 10.0.0.1
10.1.0.0/24 dev vmbr2 proto kernel scope link src 10.1.0.1
141.75.x.x/24 dev vmbr0 proto kernel scope link src 141.75.x.x -
Ok if that subnet on vmbr1 should be isolated by the pfSense VM then I would not expect to see an IP address assigned to it in Proxmox.
The way you have it setup now clients in that subnet can just use Proxmox directly and bypass pfSense.
-
@stephenw10 I dont know what to do to solve the issue because there are some production vm if I touch something it will affect those vms
-
The first thing to do is make sure pfSense itself is using the WAN side gateway as it's default route. That will stop it sending redirects to the client to the LAN side gateway. As I said I would remove/disable the LAN side gateway entirely since it should never be used as far as I can see.
Of course that is assuming that the expected route of traffic from LAN side clients is through pfSense.
I would guess that those client VMs/containers existed before pfSense was added and used that LAN gateway. Then someone added pfSense to get more control over the traffic but never finished setting up the routing.
-
OP, if you follow this you cannot go wrong, plain and simple: https://docs.netgate.com/pfsense/en/latest/recipes/virtualize-proxmox-ve.html
