Help! Totally lost on complex virtual install!



  • Let me start off by saying hello and thank you  to anyone who spends time helping me with this. I've tooled around with pfsense for about a month now to get where I am (with some heavy googling and lurking here mind you :P) and while it's working in one sense, I'd really like it to be done correctly. So now that that's out of the way! Here is my setup.

    I have a FiOS router with DHCP enabled dishing out IP's on my home LAN (192.168.1.X). All my computers/wifi are in this LAN. One of those machines is a server (192.168.1.10) running Hyper-V. There is a pfsense VM running that shares an internal only virtual NIC with the host as well as a standard virtual NIC. So the IP's so far are as follows:

    Host Machine FIOS LAN: 192.168.1.10
    Host Machine Internal Shared Virtual NIC: 192.168.0.2
    pfsense VM WAN IP  (Configured on the shared virtual NIC):  192.168.1.22
    pfsense vm LAN IP (Configured on the shared internal NIC): 192.168.0.1

    On my FIOS Router, I have forwarded all traffic on numerous ports over to the pfsense box (192.168.1.22) and then from there the pfsense box dishes it out as needed (for example, all traffic on port 80 is forwarded to the internal shared IP of the host machine [192.168.0.2] as it also running an IIS websever.)

    I know that was a giant mouthful, I'm just trying to provide as much information as possible. So here in lies my problem. I am getting very erratic behavior with this configuration. After hours and hours of troubleshooting, I've pinpointed that what seems to be happening is that I can only get to one of the two following states:

    1.In it's current state (the preferable of the two right now) pfsense loses the ability to resolve DNS for some reason, and sometimes even fails to ping as if there is no internet connection at all. I have all my DNS servers pointed to 8.8.8.8/8.8.4.4. While in this state however, external WAN requests  work (meaning I can resolve my website externally, and log in to pfsense as well from outside the 192.168.1.X LAN)

    2. In this state, it's pretty much the opposite. The pfsense VM can resolve hostnames, I can check for upgrades and install them, etc. However all WAN requests seem to fail. I can't hit anything from outside the LAN (I can manage fine from the host machine over the internal LAN). This is obviously the worse of the two as it renders my web server useless.

    In order to "switch" between these two states, I typically need to manually assign the  interface IP's. (If I assign the WAN IP, WAN requests work and it breaks pfsense's internet) or sometimes  even a reboot of the VM will do it.

    I should also note that it seems spending decent amount of time logged into the pfsense VM via the hyper-v terminal window (just double clicking on the VM in hyper-v manager) usually ends up with what appears to be a hard freeze. I can no longer input commands to the VM such as assigning interface IP's etc. However whichever of the two states I happen to be in above is still stuck, and I can manage pfsense via the webconfigurator. The only way I've found to fix that is to reboot the machine.

    If anyone has any insight as to what kind of horrible mistakes I've made here I'd be more than grateful for some insight.



  • A diagram would have been a lot clearer.  What netmasks are you using for all these connections?



  • I'm even lost on simple virtual install  8)

    ( ;D )

    But that's mostly because I am both retarded and the eternal noob, even with the kind help of BB and JFL  :-[

    (Windows server 2012 works virtually, 'though. But that's easy, that's Windows; don't cost you nothing, don't gets you anything  ;D )



  • @KOM:

    A diagram would have been a lot clearer.  What netmasks are you using for all these connections?

    Sorry, I'll try to throw together a diagram (not really my forte).

    The netmask on both the WAN and LAN interfaces is a /24 (255.255.255.0)



  • In order to "switch" between these two states, I typically need to manually assign the  interface IP's. (If I assign the WAN IP, WAN requests work and it breaks pfsense's internet)

    Can you elaborate on what you mean by 'assign the interface IPs'?  Do you use Interfaces - (assign) to remap the NICs to what they were already set to, or did you mean setting their IP addresses again?  Also, since your pfSense is behind another router, do you have Interfaces - WAN - Private networks - Block private networks unchecked?



  • Sorry for the delay, got caught up with a project at work and put this on the backburner.

    When I say reassign the interface IP's, you are correct, I meant to say I set their IP addresses again. Everytime I do this it seems to break one side of the network. Here is my best attempt at a diagram for you (apologize for how terrible I am at Visio.)


  • LAYER 8 Netgate

    And on Hyper-V you have a VM-only vSwitch, not tied to any physical NIC that you want to use for pfSense LAN and VMs "behind" pfSense?

    (sorry, no experience with Hyper-V.  Just vSphere and XenServer).

    Should work fine.



  • Sorry to bump this thread, but I was hoping someone may have some insight as to what is going on. I'm still pretty lost and I don't really get what's happening :(



  • Wow, believe it or not, it looks like my issue was caused by my DNS server settings. I was using Google DNS in the pfsense config (8.8.8.8) and for some reason it appears that it was breaking things. I've switched to my FiOS router as the DNS (192.168.1.1) and it appears to be working both externally (web requests) and internally now. How bizarre. Thanks for everyones help, here's hoping this thread helps someone else in need lol.

    Edit: I also upgraded to 2.2 during this process…I don't THINK that has anything to do with it, but I suppose it's possible.


Log in to reply