VM's take 'forever' to get network connectivity.
-
Re-asking in here, as I learn my way around these forums.
I'm running pfsense on a proxmox install.
99% of it is running great.
However, when I reboot a vm (havent' checked with real hardware yet) it takes over 10+ minutes (sometimes much more) to get network connectivity outside it's lan.
All my vm's are on 192.168.5.xxx, gateway .5.1 and are containers using a debian 8 templater.
Immediately on boot I can ping any other .5.xxx address, but nothing on .0.xxx (real machines), or the internet. Dns does resolve (if ping google.com it gives an IPv4 address, no pings).
Once connectivity comes up, when I do 'ping google.com' it prefers IPv6 and I get good results.
vbr0 = eth0
vbr1 = eth1
vbr2 = eth2
vbr5 = virtual only for VM's.I track ipv6 with a /56.
What other info can I give that will help with resolution?
ReplyQuoteNotify -
Just to show that I really do read . . .
I found the posts about tx checksum offloading. I undid that when I originally set up pfsense IN pfsense.
Just now I added "post-up /sbin/ethtool -K $IFACE tx off" to /etc/interfaces for all of my vmbr interfaces. No apparent change.
-
I virtualize PFSense on KVM (Ubuntu 14.04.3 LTS) which is the underlying virtualization layer used in Proxmox and havn't had any such issues.
My immediate concern is how your networks are configured in Proxmox and how they are cabled together on the physical boxes.
Could you give me an example of how your network is setup?
For example mine is configured like this:
WAN = Internett
Modem = Cable modem from ISP
Router = Home Wireless router
KVM-01 = Ubuntu 14.04.3
Switch = Netgear switch (Switch Connects other physical hosts to same network as Br1 and gives them WAN Access through PFSense)
PFSense = VM running on KVM-01
Br0 = Phyisical NIC on KVM-01 With a dedicated link from Router
Br1 = Phyisical NIC on KVM-01 With a dedicated link to SwitchWAN <–-> Modem <---> Router <---> Br0 (192.168.1.2 Static)<---> PFSense <---> Br1 (192.168.2.0/24 DHCP) <---> Switch
All my VMs are conencted to Br1 which is PFsense LAN interafce. All other physical hosts Connected to the same switch get internet Access through PFSense via Br1
Then on PFSense I have a NAT rule from Internal network (Br1) to WAN network (Br0) and firewall rules to allow the network traffic.
-
Following your example:
WAN = Internet
Modem = Motorala SB6183 I bought
Buffalo = Buffalo n300 in switch (dummy) mode acting as an AP and 10/100 switch.
Proxmox = that's the physical box running ProxMox VE 4
OldServer = the machine proxmox replaced that I haven't reconfigured yet
RasPi = My pi running OSMC for tv
Win7 = win7 desktopPFSense = KVM VM running on Proxmox
eth0 = nic on motherboard
eth1-4 = add on nic w/ 4 ports.
vmbr0 = proxmox's virtual switch that is connected to eth0
vmbr1 = proxmox's virtual switch that is connected to eth1
vmbr2 = same but eth2
vmbr5 = virtual only switchPhysical:
WAN <–-> Modem <----> eth0
eth1 <---> Buffalo <---> RasPi (dhcp 192.168.0.xxx)
<wifi>Win7 (dhcp 192.168.0.xxx)
<wifi>phones/tablets/etc
eth2 <--> OldServer (dhcp 192.168.2.xxx)Virtual:
pfsense:
Wan (dhcp from ISP) <--> Lan (vmbr1) 192.168.0.1 <-->buffalo (and follows from above)
<--> VMNet (vmbr5) 192.168.5.1 <---> Plex (192.168.5.101
<--> DVR (192.168.5.103)
<---> Others following same idea, all static.
<--> AltNet (vmbr2) 192.168.2.1 <---> Old server (DHCP)Something that JUST struck me as I was looking at things to tell you (and this is why I ask questions!) my network lists in proxmox. See pic.
No address for the ethX is normal. The no address for vmbr0 means that it gets one from DHCP from my ISP.
But I have no gateways for VMBR2 or 5. It won't let me assign .0.1 as a gateway b/c it's already the gateway for vmbr1.
So . . . maybe I need to change the bridge ips to .2.2 & .5.2 and then put the gateways at .2.1 & .5.1, which I have statically assigned in pfsense?
I'd like feedback before I make this change.
EDIT: I attempted to make the change on a new vmbr and it wouldn't let me set anything for gateway. So only the one gateway, apparently.
</wifi></wifi>
-
The IPs in your picture are the IPs of the interfaces on the Proxmox box - The physical interfaces.
eth0 (vmbr0) seems like it's not configures to have an IP at all –---
eth1 (vmbr1) seems like it has a DHCP address - 192.168.0.42
eth2 (vmbr2) seems like it has been assigned a static IP - 192.168.2.1
vmbr5 isn't bound to a NIC, but has IP 192.168.5.1Those addresses above cannot be assigned to your PFSense box or it will get all confused. And they are not going to be Gateway addresses
Consider those addresses simple clients on your network.
Your PFSense VM IPs:
WAN: DHCP from ISP = OK
LAN: Static 192.168.0.1
LAN2: Static 192.168.2.2
LAN3: Static 192.168.5.2Those addreses are also the GATEWAY for your respective LANs.
Example client on each LAN
LAN
IP: 192.168.0.50
Mask: 255.255.255.0
Gateway: 192.168.0.1
DNS: 192.168.0.1LAN2
IP: 192.168.2.50
Mask: 255.255.255.0
Gateway: 192.168.2.2
DNS: 192.168.2.2LAN3
IP: 192.168.5.50
Mask: 255.255.255.0
Gateway: 192.168.5.2
DNS: 192.168.5.2Hope that helps.
Have to sleep now :D
-
To be perfectly honest, that didn't seem to make a difference, but when I put it all back, to my original settings, things seem to be working much better.
Granted, there were a few reboots in there, and one ip conflict (forgot my buffalo is on 192.168.0.2).
I'm not calling it solved, but I am thinking I may be done for now.
-
To be perfectly honest, that didn't seem to make a difference, but when I put it all back, to my original settings, things seem to be working much better.
Granted, there were a few reboots in there, and one ip conflict (forgot my buffalo is on 192.168.0.2).
I'm not calling it solved, but I am thinking I may be done for now.
Definitely a fundamental design flaw in your physical and logical network design. The fault doesn't lie With PFSense at least.
If I was you I would create a single WAN to LAN network with PFSense and get that working properly. Then add in the other LANs 1 by 1.
Also keep in mind that the IPs on your proxmox Box (Ips in your attachment) are bound to the physical Ethernet adapters and are in no way responsible for directing network traffic.
-
Definitely a fundamental design flaw in your physical and logical network design. The fault doesn't lie With PFSense at least.
If I was you I would create a single WAN to LAN network with PFSense and get that working properly. Then add in the other LANs 1 by 1.
Also keep in mind that the IPs on your proxmox Box (Ips in your attachment) are bound to the physical Ethernet adapters and are in no way responsible for directing network traffic.
I actually did that. Single Wan to Lan worked fine. And still does. Opt1 (VMNet) is what's slow to be able to get outside it's own subnet now.
I dunno. It's working well now. The only issue I have now is reverse proxy, which should be simple, but I'm apparently less knowledgeable than I thought. :)
-
Definitely a fundamental design flaw in your physical and logical network design. The fault doesn't lie With PFSense at least.
If I was you I would create a single WAN to LAN network with PFSense and get that working properly. Then add in the other LANs 1 by 1.
Also keep in mind that the IPs on your proxmox Box (Ips in your attachment) are bound to the physical Ethernet adapters and are in no way responsible for directing network traffic.
I actually did that. Single Wan to Lan worked fine. And still does. Opt1 (VMNet) is what's slow to be able to get outside it's own subnet now.
I dunno. It's working well now. The only issue I have now is reverse proxy, which should be simple, but I'm apparently less knowledgeable than I thought. :)
If you are using Squid3 there is a weird bug where it cannot bind to ports lower than port 1024. So the only way to solve that is to force Squid3 to listen on a highport (higher than 1024) and NAT port 80/443 to that highport on localhost.
https://forum.pfsense.org/index.php?topic=88191.0
-
That's not it then. My ISP blocks 80, so I use 8080 from outside, and try to redirect it to 80 inside.
What I want ideally, is from outside to be able to type dvr.mydomain.net and get to mydomain.net:XXXX for sonarr. And other things too. wiki, htpc stuff, etc etc.
I'll happily accept <service>.mydomain.net:8080 redirecting internal-ip:port for each service tho.
I mean, I could just NAT so that mydomain.net:XXXX goes to internal-ip:xxxx, but I want to idiot proof some of this for others.
Search my post history, you'll see where I ask for hand holding. </service>
-
Finally figured this out.
In my network config I had used netmask 255.255.255.0
Should have had something closer to 255.255.240.0. :)
Everything seems to work normally now. Wonder why it worked at all before?
-
What??? Why would you mask have to do with anything.. You should be able to use whatever netmask you want.. /20 that you state would put all those networks on the same network /20 is x.x.0.1 to x.x.15.254
-
I'm sorry, it was the netmask on the proxmox set up, not in pfsense. I thought that part at the screen REAL HARD, but it didn't make it for some reason.
lol
