pfSense on Hyper-V WAN IP Issue

lispeedyg

Hi,
I've been using fpSense on a number of installs over the past years that have worked flawlessly.

Howeve, as of a few months ago, the pfSense installs that require DHCP with Optimum Cable (Altice) no longer work. Since the problems have risen, I've been trying to debug the issue. Note, I have similar installs (i.e., Hyper-V VM's) where they use Static IP's and have not had any issues.

In the faulting installs pdFense in unable to pull the correct WAN addresses. LAN clients are still routed correctly INTERNALLY. But, they have no access to the internet.

Problems:

Cannot ping outside network (only internal IP's resolve)
Cannot open Cable Modem page, i.e. Arris TM1602A @192.168.100.1
Packet Capture of the WAN provided only Discover requests without any DHCP Offers (to be expected as I cannot open modem page)

Things I've tried:

Rebuilt (from scratch) Hyper-V server
Used 2 new Intel Pro/1000 Dual port Network cards
Installed a fresh copy pfSense 2.4.5-P1
I replaced the pfSense router with a very standard minimal router (Netgear N300) and was able to immediately resolve the DHCP
With successful Netgear connection, I tried spoofing it's MAC on pfSense (No DHCP response)
I've tried using the known Dynamic IP and inserted it as static -- No Go
I've tried switching out Network cards with existing working ones from active installs -- No Change
I've tweaking Network adapter settings to try spoofing the working Netgear N300 -- No Go
I've disabled Network Adapter Hardware acceleration -- No Go

I've probably tried a more actions that I've forgotten over the months of trials. But, at this point I'm out of ideas.

Any Help would be appreciated.

Gertjan

@lispeedyg said in pfSense on Hyper-V WAN IP Issue:

Packet Capture of the WAN provided only Discover requests without any DHCP Offers

This DHCP Discovers is coming from pfSense, right ?
And if the pfSense WAN is set to static, there is no issue ?
Is the modem connected to the ISP - the link is up ?
The ISP should reply to the DHCP Discover from pfSense (and not the modem - as it is a modem and not a router).

@lispeedyg said in pfSense on Hyper-V WAN IP Issue:

Used 2 new Intel Pro/1000 Dual port Network cards

One of these, the WAN, is reserved and used only by the VM of pfSense, and not the host, right ?

lispeedyg

@gertjan

Yes the DHCP discoveries are from pfSense
Static IP's are resolving correctly
The ISP link is up
Yes
Both network cards are Dual port cards. I use one of the cards for the pfSense VM and the other is used to access the VM host.

I just ran another test to eliminate the Hyper-V component, i.e. I completely deleted the Hyper-V Host and configured a new pfSense installed without any other complications on the same hardware.

Result: The same issue exists. I'm still unable to pull an IP from the ISP.

Gertjan

Can't this be a MAC issue ? The ISP only gives a replies to the DHCP discover if it recognizes the MAC ?

lispeedyg

@gertjan
I don't think so. As I mentioned I used a very simple router which worked perfectly. And, once I switched back to the pfSense box, I spoofed the MAC address of the simple router, i.e., NETGEAR N300 without success.

lispeedyg

Does anyone have any ideas?
Here's a view into the adapters Hyper-V Adapters:
vmconnect6.2_08d6kZCqHw.png

provels

@lispeedyg
Stating the obvious, I know, but is your WAN vSwitch set to 'External'?

Do you have any working VM pfSense instance that works that you can pull a config from?

lispeedyg

@provels said in pfSense on Hyper-V WAN IP Issue:

@lispeedyg
Stating the obvious, I know, but is your WAN vSwitch set to 'External'?>

It is.

Do you have any working VM pfSense instance that works that you can pull a config from?

This "was working" for years. And, I have 2 other installs at other locations with the same ISP that are showing the same issues. But, only when I need to pull an IP. the ones with static IP's are working fine.

But, it seems that I can't even ping the modem ip where I'm perfectly able to ping and open the modem page when using a simple router.

provels

@lispeedyg said in pfSense on Hyper-V WAN IP Issue:

same ISP that are showing the same issues

Prob need to work with ISP, maybe they can assign you a temp static for one instance to prove it's their problem.

But, it seems that I can't even ping the modem ip

Since the WAN has no address. Bet you could if you set the FW WAN IP address to 192.168.100.2...

lispeedyg

@provels said in pfSense on Hyper-V WAN IP Issue:

@lispeedyg said in pfSense on Hyper-V WAN IP Issue:

same ISP that are showing the same issues

Prob need to work with ISP, maybe they can assign you a temp static for one instance to prove it's their problem.

They are of no use. The only way for them to issue a static IP is to sign up for an account.

But, it seems that I can't even ping the modem ip

Since the WAN has no address. Bet you could if you set the FW WAN IP address to 192.168.100.2...

Your suggestion: No Go.

Also, just swapped out modem with new one -- No Go
Swapped Network cards -- No Go

I can see the WAN IP from the Hyper-V Host interface but cannot get it to work on the pfSense VM. There has to be something different with the base pfSense code for acquiring dynamic IP's.

I see another thread from 2018 that seemed to try and address a similar issue. But the thread ended with a comment for the user to "Work upstream to why offer is not being sent.". I don't know where or how this would work.

Hopefully someone here can point me to possible paths to follow. This is making no sense to me at this point as to why it's not working.

Thanks in advance.

provels

@lispeedyg said in pfSense on Hyper-V WAN IP Issue:

Your suggestion: No Go.

Can't even reach the modem's inside interface? Weird. Would double NAT be an option as last resort? Delete/recreate the WAN vSwitch?

Just throwing stuff at the wall now. How many nodes can you threaten to pull from the ISP to engage their cooperation?

Bob.Dig

Seems to be not a hyper-v problem but a problem with that specific ISP. If you use Windows-Server you could DDA one NIC to the vm directly, but probably won't help either.

lispeedyg

@provels said in pfSense on Hyper-V WAN IP Issue:

@lispeedyg said in pfSense on Hyper-V WAN IP Issue:

Your suggestion: No Go.

Can't even reach the modem's inside interface? Weird. Would double NAT be an option as last resort? Delete/recreate the WAN vSwitch?

Just throwing stuff at the wall now. How many nodes can you threaten to pull from the ISP to engage their cooperation?

Thanks for the responses. I can't count the number of times I've deleted and re-created the vSwitches.

As far as double NAT, I've been avoiding that option since the speed and security would be inherently compromised.

And, unfortunately, the ISP could care less unless. They've recently (last year) been purchased by a French company Altice. And, they've been dramatically reducing staff count. Service as definitely degraded since the acquisition.

Any other suggestions would be appreciated.

lispeedyg

@bob-dig said in pfSense on Hyper-V WAN IP Issue:

Seems to be not a hyper-v problem but a problem with that specific ISP. If you use Windows-Server you could DDA one NIC to the vm directly, but probably won't help either.

Thanks for the response. But, I already tried installing pfSense without the use of a VM -- No Go.

I really think this may be a pfSense DHCP bug. If so, can you tell me how to report and the info needed to support the bug resolution? Or, should I bite the bullet and get a Static IP?

Thanks for any suggestions.

provels

@lispeedyg
Any Reddit groups for your ISP?
Tried a Gen1 VM? I only have 2012R2 as far as Hyper-V goes... But I've had no issues with either Realteks or my current Intel i340-T4 using Gen1.
Tried swapping ports and reconfiguring the NICS? I know you said 2 "new" cards, but I doubt any "new " cards are being made with 13-year old tech except grey market. Did you just mean "different"?
Are there other approved modems for this provider?

Again, just throwin'...

lispeedyg

@provels said in pfSense on Hyper-V WAN IP Issue:

@lispeedyg
Any Reddit groups for your ISP?
Tried a Gen1 VM? I only have 2012R2 as far as Hyper-V goes... But I've had no issues with either Realteks or my current Intel i340-T4 using Gen1.
Tried swapping ports and reconfiguring the NICS? I know you said 2 "new" cards, but I doubt any "new " cards are being made with 13-year old tech except grey market. Did you just mean "different"?
Are there other approved modems for this provider?

Again, just throwin'...

Thanks for the suggestion I just created a post on Reddit

Gen1's tried -- No Go

The replacement modem i installed yesterday was replaced with the same model Arris TM1602A. After install I hooked up my laptop directly to the modem and was able to get to the status page (see below) and the internet. But, still unable to have pfSense acquire the IP of the Modem or the WAN.:

provels

@lispeedyg
~~Maybe spoof the laptop's MAC on the WAN interface now that it has connected once?~~ Guess you tried with the Netgear.
Try double NAT just to see if it connects?
See if one of your team has a spare/different modem you can borrow.
Does the host share the WAN or LAN NIC for management?

How are you seeing the WAN address on the host NIC?

lispeedyg

@provels said in pfSense on Hyper-V WAN IP Issue:

@lispeedyg
Maybe spoof the laptop's MAC on the WAN interface now that it has connected once?
Try double NAT just to see if it connects?
See if one of your team has a spare/different modem you can borrow.

How are you seeing the WAN address on the host NIC?

Hi, I've already tried spoofing using the NetGear router that did work on both the WAN vSwitch -- No Go

As far as seeing the WAN IP, I have 2 dual port Intel 1G Network Cards. One for the Optimum (issues) ISP. ANd the other using FIOS (primary) Fiber channel ISP. So, in the VM host I actually have 2 VM's one for pfSense both ports connected to the OPT card. And, another VM (windows) machine that I can switch between either card as the network port. Additionally, the VM Host is connected to the FIOS AND the OPTIMUM cards give me a look into the IP's for each card:

provels

@lispeedyg
I have no exp w/ multi WAN, sorry. Can the FIOS be disabled to see if it matters w/o bringing the company down? Are all your locs multi WAN?

Not to insult, but you've probably memorized these pages... https://docs.netgate.com/pfsense/en/latest/multiwan/index.html

Maybe a bug, but you've exhausted me! You could open a bug report at https://redmine.pfsense.org/projects/pfsense

lispeedyg

@provels said in pfSense on Hyper-V WAN IP Issue:

@lispeedyg
I have no exp w/ multi WAN, sorry. Can the FIOS be disabled to see if it matters w/o bringing the company down? Are all your locs multi WAN?

Not to insult, but you've probably memorized these pages... https://docs.netgate.com/pfsense/en/latest/multiwan/index.html

Maybe a bug, but you've exhausted me! You could open a bug report at https://redmine.pfsense.org/projects/pfsense

Most of the installs do not have multi-WAN environments. But, i only use the second card as my gateway into the primary LAN environment. And, as I've mentioned, I've tried only pfSense installed on a PC (single network card) -- No Go.

Thanks for the input. It's always useful to get another point of view.